Hyperspectral imaging analysis and applications for food quality 9781138630796, 1138630799

Table of contents :
Content: Imaging systems. Fundamentals / P.J. Williams and K. Sendin
Optimization of hyperspectral image cube acquisition : a case study on meat and bone meal / Cecilia Riccioli, Ana Garrido Varo, and Dolores Pérez Marin
Image segmentation / Sylvio Barbon Jr., Ana Paula Ayub da Costa Barbon, N.A. Valous, and D.F. Barbin
Data extraction and treatment / Yao-Ze Feng and Hai-Tao Zhao --
Chemometrics. Multivariate analysis and techniques / Mohammed Kamruzzaman
Principal component analysis / Cristina Malegori and Paolo Oliveri
Partial least squares regression / Leo M.L. Nollet
Linear discriminant analysis / Leo M.L. Nollet
Support vector machines / Leo M.L. Nollet
Decision trees / Leo M.L. Nollet
Artificial neural networks and hyperspectral images for quality control in foods / Luis Condezo-Hoyos and Wilson Castro --
Applications. Recent advances for rapid detection of quality and safety of fish by hyperspectral imaging analysis / Chao-Hui Feng, Yoshio Makino, Masatoshi Yoshimura, and Francisco J. Rodríguez-Pulido
Applications of hyperspectral imaging for meat quality and authenticity / Mohammed Kamruzzaman
Hyperspectral imaging : applications in analysis of fruits for quality and safety / Anoop A. Krishnan and S.K. Saxena
Applications in vegetables / Leo M.L. Nollet, Hong-Ju He, and Hui Wang
Applications in medicinal herbs and pharmaceuticals / Leo M.L. Nollet, Hong-Ju He, and Hui Wang
Hyperspectral imaging in dairy products analysis / Basil K. Munjanja
Hyperspectral imaging : application in quality and safety of beverages / N.C. Basantia
Raman hyperspectral imaging : application in food additives' quality and safety / Rajesh Kumar R. Singh and N.C. Basantia.

Citation preview

Hyperspectral Imaging ­Analysis and Applications for Food Quality

Food Analysis and Properties Series Editor: Leo M.L. Nollet University College Ghent, Belgium This CRC series Food Analysis and Properties is designed to provide a state-of-art coverage on topics to the understanding of physical, chemical, and functional properties of foods including (1) recent analysis techniques of a choice of food components, (2) developments and evolutions in analysis techniques related to food, and (3) recent trends in analysis techniques of specific food components and/or a group of related food components.

Flow Injection Analysis of Food Additives Edited by Claudia Ruiz-Capillas and Leo M.L. Nollet

Marine Microorganisms: Extraction and Analysis of Bioactive Compounds Edited by Leo M.L. Nollet

Multiresidue Methods for the Analysis of Pesticide Residues in Food Edited by Horacio Heinzen, Leo M.L. Nollet, and Amadeo R. Fernandez-Alba

Spectroscopic Methods in Food Analysis Edited by Adriana S. Franca and Leo M.L. Nollet

Phenolic Compounds in Food: Characterization and Analysis Edited by Leo M.L. Nollet, Janet Alejandra Gutierrez-Uribe

Testing and Analysis of GMO-containing Foods and Feed Edited by Salah E. O. Mahgoub and Leo M.L. Nollet

Fingerprinting Techniques in Food Authenticity and Traceability Edited by K.S. Siddiqi and Leo M.L. Nollet

Hyperspectral Imaging Analysis and Applications for Food Quality Edited by N.C. Basantia, Leo M.L. Nollet, Mohammed Kamruzzaman For more information, please visit the Series Page: https://www.crcpress.com/ Food-Analysis--Properties/book-series/CRCFOODANPRO

Hyperspectral Imaging ­Analysis and Applications for Food Quality

Edited by

N.C. Basantia Leo M.L. Nollet Mohammed Kamruzzaman

CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2019 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed on acid-free paper International Standard Book Number-13: 978-1-138-63079-6 (Hardback) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all materials reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged, please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com

To my wife, Niharika, daughter, Nivriti, and son, Devansh. Without their support, cooperation, encouragement, sacrifice of their happy moments, and affection, I wouldn’t have completed this work in time. To all the scientists who aim to bring one of the green, fast, and nondestructive types of technology into application for food quality and safety and to help all the stakeholders of the food processing sector. Dr. N.C. Basantia

Contents Series Preface ix Preface

xi

Editors xiii List of Contributors xv

Section I  IMAGING SYSTEMS Chapter 1

Fundamentals 3 P.J. Williams and K. Sendin

Chapter 2

Optimization of Hyperspectral Image Cube Acquisition: A Case Study on Meat and Bone Meal

21

Cecilia Riccioli, Ana Garrido Varo, and Dolores Pérez Marin Chapter 3

Image Segmentation 35 Sylvio Barbon Jr., Ana Paula Ayub da Costa Barbon, N.A. Valous, and D.F. Barbin

Chapter 4

Data Extraction and Treatment 45 Yao-Ze Feng and Hai-Tao Zhao

Section II  CHEMOMETRICS Chapter 5

Multivariate Analysis and Techniques

61

Mohammed Kamruzzaman Chapter 6

Principal Component Analysis 85 Cristina Malegori and Paolo Oliveri

Chapter 7

Partial Least Squares Regression

109

Leo M.L. Nollet Chapter 8

Linear Discriminant Analysis 115 Leo M.L. Nollet

vii

viii

Chapter 9

Contents

Support Vector Machines

119

Leo M.L. Nollet Chapter 10

Decision Trees

123

Leo M.L. Nollet Chapter 11

Artificial Neural Networks and Hyperspectral Images for Quality Control in Foods 125 Luis Condezo-Hoyos and Wilson Castro

Section III  APPLICATIONS Chapter 12

Recent Advances for Rapid Detection of Quality and Safety of Fish by Hyperspectral Imaging Analysis 159 Chao-Hui Feng, Yoshio Makino, Masatoshi Yoshimura, and Francisco J. Rodríguez-Pulido

Chapter 13

Applications of Hyperspectral Imaging for Meat Quality and Authenticity 175 Mohammed Kamruzzaman

Chapter 14

Hyperspectral Imaging: Applications in Analysis of Fruits for Quality and Safety 195 Anoop A. Krishnan and S.K. Saxena

Chapter 15

Applications in Vegetables 207 Leo M.L. Nollet, Hong-Ju He, and Hui Wang

Chapter 16

Applications in Medicinal Herbs and Pharmaceuticals 233 Leo M.L. Nollet, Hong-Ju He, and Hui Wang

Chapter 17

Hyperspectral Imaging in Dairy Products Analysis

239

Basil K. Munjanja Chapter 18

Hyperspectral Imaging: Application in Quality and Safety of Beverages 245 N.C. Basantia

Chapter 19

Raman Hyperspectral Imaging: Application in Food Additives’ Quality and Safety 263 Rajesh Kumar R. Singh and N.C.Basantia

Index 277

Series Preface There will always be a need for analyzing methods of food compounds and properties. Current trends in analyzing methods include automation, increasing the speed of analyses, and miniaturization. The unit of detection has evolved over the years from micrograms to picograms. A classical pathway of analysis is sampling, sample preparation, cleanup, derivatization, separation, and detection. At every step, researchers are working and developing new methodologies. A large number of papers are published every year on all facets of analysis. So, there is a need for books that gather information on one kind of analysis technique or on analysis methods of a specific group of food components. The scope of the CRC Series on Food Analysis & Properties aims to present a range of books edited by distinguished scientists and researchers who have significant experience in scientific pursuits and critical analysis. This series is designed to provide state-of the-art coverage on topics such as 1. Recent analysis techniques on a range of food components 2. Developments and evolution in analysis techniques related to food 3. Recent trends in analysis techniques of specific food components and/or a group of related food components 4. The understanding of physical, chemical, and functional properties of foods The book Hyperspectral Imaging Analysis and Applications for Food Quality is the eighth volume of this series. I am happy to be a series editor of such books for the following reasons: • I am able to pass on my experience in editing high-quality books related to food. • I get to know colleagues from all over the world more personally. • I continue to learn about interesting developments in food analysis. A lot of work is involved in the preparation of a book. I have been assisted and supported by a number of people, all of whom I would like to thank. I would especially like to thank the team at CRC Press/Taylor & Francis, with a special word of thanks to Steve Zollo, Senior Editor. Many, many thanks to all the editors and authors of this volume and future volumes. I very much appreciate all their effort, time, and willingness to do a great job.

ix

x

Series Preface

I dedicate this series to • My wife, for her patience with me (and all the time I spend on my computer) • All patients suffering from prostate cancer; knowing what this means, I am hoping they will have some relief Dr. Leo M.L. Nollet (Retired) University College Ghent Ghent, Belgium

Preface Professor A. Goetz of Colorado Boulder University was the first to use the term “hyperspectral imaging.” It was initially limited to use on aircrafts or spacecrafts for remote sensing applications and earth observation. Then, it was used to identify surface materials for geological purposes. It was only by the early 1990s that it was first reported for use in the laboratory and it is now almost ubiquitous in agricultural and food analysis. Hyperspectral imaging is a technique that enables the acquisition of both spatial and spectral information from an object. This provides researchers with spatially resolved data, permitting localization and visualization of chemical and biochemical compounds within samples. Where conventional spectroscopy provides information from a single spot at various points across a sample, hyperspectral imaging allows data acquisition of the entire sample in one swift measurement. Image analyses involving mathematical, statistical, and software programming appro­aches are the essential elements of any computer-integrated hyperspectral imaging system. Interests in food quality are driven from the essential need to supply consumers with consistent products at an affordable price. The ideal way to enhance food supplies and consider food safety issues is to regularly monitor food products during all stages of the handling chain to detect the onset of potential issues and to take timely action once a problem has been identified. Effective monitoring systems and an intelligent approach to providing reliable information about the content, composition, and safety properties of food materials are required. The processes of food quality assurance and overall food quality monitoring are traditionally performed by human panels, chemical analysis, and/ or mechanical methods. Unfortunately, these methods are destructive, time-consuming, laborious, and costly and require specific sample preparation. Hyperspectral imaging analysis is a novel technique to monitor food quality offering a lot of benefits. The reader finds in this book three sections. Section I discusses the fundamentals of Imaging Systems: How can hyperspectral image cube acquisition be optimized? Also, two chapters deal with image segmentation, data extraction, and treatment. One of the advantages in developing hyperspectral imaging systems is the wealth of spectral data residing in the images. These data are multivariate in nature due to a large number of data variables, one at each wavelength, for each sample. Multivariate data analysis is thus an indispensable part of this novel detection technique. It is required to appropriately extract meaningful information from the spectra to correlate with the measured attribute under investigation.

xi

xii

Preface

Seven chapters in Section II deal with Chemometrics. Chapter 5 explains the fundamentals of multivariate analysis and techniques. In six chapters, the reader may find information on and applications of a number of chemometric techniques: principal component analysis, partial least squares analysis, linear discriminant model, support vector machines, decision trees, and artificial neural networks. In the last section, Applications, numerous examples are given of applications of hyperspectral imaging systems in fish, meat, fruits, vegetables, medicinal herbs, dairy products, beverages, and food additives. The three editors like to thank all contributors for all work, energy, and time spent to deliver on-time, excellent contributions.

“Men moet een boek niet beoordelen aan de kaft.” Dutch proverb “One should not judge a book by its cover”

Editors N.C. Basantia is a senior lead scientist at Cavinkare Research Centre of Cavinkare Private Ltd., in Chennai, India. Dr. Basantia graduated with a PhD in Dairy Chemistry at the National Dairy Research Institute, Karnal, India, in 2002, and with an MS in Dairy Chemistry at the National Dairy Research Institute, Karnal, India, in 1997. His technical expertise and experiences are analysis of dairy foods and others, HPLC and GC, and laboratory management as per ISO/ IEC:17025. He is member of a number of technical committees such as the Bureau of Indian Standard for cosmetic ­products. He is the author and coauthor of numerous papers and book chapters. Leo M.L. Nollet earned an MS (1973) and PhD (1978) in biology from the Katholieke Universiteit Leuven, Belgium. He is an editor and associate editor of numerous books. He edited for M. Dekker, New York— now CRC Press of Taylor & Francis Publishing Group—the first, second, and third editions of Food Analysis by HPLC and Handbook of Food Analysis. The last edition is a two-volume book. Dr. Nollet also edited the Handbook of Water Analysis (first, second, and third editions) and Chromatographic Analysis of the Environment, third and fourth editions (CRC Press). With F. Toldrá, he coedited two books published in 2007 and 2017, respectively: Advances in Food Diagnostics (Blackwell Publishing—now Wiley) and Advanced Technologies for Meat Processing (CRC Press). With M. Poschl, he coedited the book Radionuclide Concentrations in Foods and the Environment, also published in 2006 (CRC Press). Dr. Nollet has also coedited with Y. H. Hui and other colleagues on several books: Handbook of Food Product Manufacturing (Wiley, 2007), Handbook of Food Science, Technology, and Engineering (CRC Press, 2005), Food Biochemistry and Food Processing (first and second editions; Blackwell Publishing—now Wiley—2006 and 2012), and the Handbook of Fruits and Vegetable Flavors (Wiley, 2010). In addition, he edited the Handbook of Meat, Poultry, and Seafood Quality, first and second editions (Blackwell Publishing—now Wiley—2007 and 2012). From 2008 to 2011, he published five volumes on animal product-related books with F. Toldrá: Handbook of Muscle Foods Analysis, Handbook of Processed Meats and Poultry Analysis, Handbook of Seafood and Seafood Products Analysis, Handbook of Dairy Foods Analysis, and Handbook of Analysis of Edible Animal By-Products. Also, in 2011, with F. Toldrá, he coedited two volumes for CRC Press: Safety Analysis of Foods of Animal Origin and Sensory Analysis of Foods of Animal Origin. In 2012, they published the Handbook of Analysis of Active Compounds in Functional Foods. In a

xiii

xiv

Editors

coedition with Hamir Rathore, Handbook of Pesticides: Methods of Pesticides Residues Analysis was marketed in 2009; Pesticides: Evaluation of Environmental Pollution in 2012; Biopesticides Handbook in 2015; and Green Pesticides Handbook: Essential Oils for Pest Control in 2017. Other finished book projects include Food Allergens: Analysis, Instrumentation, and Methods (with A. van Hengel; CRC Press, 2011) and Analysis of Endocrine Compounds in Food (Wiley-Blackwell, 2011). Dr. Nollet’s recent projects include Proteomics in Foods with F. Toldrá (Springer, 2013) and Transformation Products of Emerging Contaminants in the Environment: Analysis, Processes, Occurrence, Effects, and Risks with D. Lambropoulou (Wiley, 2014). In the series Food Analysis & Properties, he edited (with C. Ruiz-Capillas) Flow Injection Analysis of Food Additives (CRC Press, 2015) and Marine Microorganisms: Extraction and Analysis of Bioactive Compounds (CRC Press, 2016). With A.S. Franca, he coedited Spectroscopic Methods in Food Analysis (CRC Press, 2017), and with Horacio Heinzen and Amadeo R. FernandezAlba, he coedited Multiresidue Methods for the Analysis of Pesticide Residues in Food (CRC Press, 2017). Mohammed Kamruzzaman is a professor and head at the Department of Food Technology and Rural Industries of Bangladesh Agricultural University, Mymensingh, Bangladesh. Professor Kamruzzaman received B.Sc. in Chemical Engineering from Bangladesh University of Engineering and Technology (BUET), MS in Food Technology from Bangladesh Agricultural University, and M.Sc. in Food Science, Technology and Nutrition from a consortium of the four European Universities (Belgium, Germany, Ireland and Portugal). He obtained Ph.D. in Food and Biosystems Engineering from University College Dublin (UCD), Ireland. He conducted JSPS postdoctoral research at the University of Tokyo, Japan. He also conducted post-doctoral research at the University of California, Davis (UC Davis), USA. His research interests focus on hyperspectral imaging, computer vision, spectroscopy, and multivariate analysis applied in determination of meat quality, safety, and authenticity. His research not only has been published in numerous prestigious international journals and book chapters but has also been useful for non-destructive meat quality evaluation for the meat industry. He is an editorial board member of Journal of Biosystems Engineering. He is a member of Institute of Food Technologists (IFT) and a Fellow of the Institution Engineers, Bangladesh.

List of Contributors D.F. Barbin Department of Food Engineering University of Campinas Campinas, Brazil Ana Paula Ayub da Costa Barbon Department of Animal Science State University of Londrina Londrina, Brazil Sylvio Barbon Jr. Department of Computer Science State University of Londrina Londrina, Brazil N.C. Basantia Cavinkare Research Center Cavinkare Pvt. Ltd Chennai, India Wilson Castro School of Industrial Engineering Universidad Privada del Norte Cajamarca, Perú Luis Condezo-Hoyos Scientific Analyst and Consultant Lima, Perú

Chao-Hui Feng Graduate School of Agricultural and Life Sciences The University of Tokyo Bunkyō, Tokyo, Japan and College of Pharmacy and Biological Engineering Chengdu University Chengdu, Sichuan, China and College of Food Science Sichuan Agricultural University Ya’an, Sichuan, China Yao-Ze Feng Key Laboratory of Agricultural Equipment in Mid-lower Yangtze River Ministry of Agriculture Huazhong Agricultural University Wuhan, China Ana Garrido Varo ETSIAM Universidad de Cordoba Córdoba, Spain Hong-Ju He Agro-Food Nondestructive Detection & Analysis Team School of Food Science Henan Institute of Science and Technology (HIST) Xinxiang, Henan, China

xv

xvi

List of Contributors

Mohammed Kamruzzaman Department of Food Technology and Rural Industries Bangladesh Agricultural University Mymensingh, Bangladesh Anoop A. Krishnan Export Inspection Agency- Kochi (Laboratory) Kerala, India Yoshio Makino Graduate School of Agricultural and Life Sciences The University of Tokyo Bunkyō, Tokyo, Japan Cristina Malegori Department of Pharmacy University of Genova Genova, Italy Basil K. Munjanja Department of Chemistry University of Pretoria Pretoria, South Africa Leo M.L. Nollet (Retired) University College Ghent Gent, Belgium Paolo Oliveri Department of Pharmacy University of Genova Genova, Italy Dolores Pérez Marin ETSIAM Universidad de Cordoba Córdoba, Spain Cecilia Riccioli ETSIAM Universidad de Cordoba Córdoba, Spain

Francisco J. Rodríguez-Pulido Food Colour & Quality Laboratory, Department of Nutrition and Food Science University of Seville Seville, Spain S.K. Saxena Export Inspection Council New Delhi, India and School of Food Science Henan Institute of Science and Technology (HIST) Xinxiang, Henan, China K. Sendin Department of Food Science Stellenbosch University Stellenbosch, South Africa Rajesh Kumar R. Singh Prem Henna Private Limited Nasik, India N.A. Valous Department of Food Engineering University of Campinas Campinas, Brazil Hui Wang Agro-Food Nondestructive Detection & Analysis Team, School of Food Science Henan Institute of Science and Technology (HIST) Xinxiang, Henan, China P.J. Williams Department of Food Science Stellenbosch University Stellenbosch, South Africa Masatoshi Yoshimura Graduate School of Agricultural and Life Sciences The University of Tokyo Bunkyō, Tokyo, Japan

List of Contributors

Hai-Tao Zhao Key Laboratory of Agricultural Equipment in Mid-lower Yangtze River Ministry of Agriculture Huazhong Agricultural University Wuhan, China

xvii

Section

I

Imaging Systems

Chapter

1

Fundamentals P.J. Williams and K. Sendin Stellenbosch University

CONTENTS 1.1 Introduction 3 1.1.1 T  he Need for Speed in Food Analysis 4 1.2 Introduction to Spectroscopy 5 1.3 Integration of Spectroscopy and Imaging 7 1.4 Introduction to Hyperspectral Imaging 8 1.4.1 B  rief History 8 1.4.2 P  rinciples of Hyperspectral Imaging 8 1.4.3 B  asic Hyperspectral System Components and Set-Up 9 1.5 Multispectral Imaging 11 1.6 Analysing Hyperspectral Images 12 1.7 Advantages and Disadvantages 14 1.7.1 A  dvantages 14 1.7.2 D  isadvantages 15 1.8 Conclusion 15 References 16

1.1 INTRODUCTION Hyperspectral imaging, originally developed by Goetz et al. (1985) for earth observation and remote sensing at NASA’s Jet Propulsion Laboratory, has recently become a topic of much research interest in the fields of agriculture and food science (Elmasry et al., 2012). It was known as imaging spectrometry in the 1980s until Goetz et al. (1985) first used the term “hyperspectral imaging”, and it was limited to use on aircrafts or spacecrafts for remote sensing applications and earth observation. It was then used to identify surface materials for geological purposes and was especially advantageous as it could “see through” vegetation covering the land (Sendin et al., 2018). It was first reported for use in the laboratory in the early 1990s (Robert et al., 1991), and is now almost ubiquitous in research applications related to agricultural and food analyses (quality and safety) (Gowen et al., 2007; Gowen et al., 2015). Hyperspectral imaging is a technique that enables the acquisition of both spatial and spectral information from an object (e.g. cereal grains, fruit, meat or any agrofood product) (Burger & Geladi, 2005; Gowen et al., 2007). This provides researchers with spatially resolved data, permitting localisation and visualisation of chemical and

3

4

P.J. Williams and K. Sendin

biochemical compounds within samples. Conventional spectroscopy provides information from a single spot at various points across a sample, whereas the hyperspectral imaging allows data acquisition of the entire sample in one swift measurement (Manley, 2014). This makes it an ideal analytical tool for the evaluation and analysis of food and agricultural products. Consequently, there has been a surge in the number of publications dedicated to the application of hyperspectral imaging, particularly near infrared (NIR) hyperspectral imaging, to analyse, evaluate and monitor various food and agricultural products, such as meat (Elmasry et al., 2011); fruits and vegetables (Lorente et al., 2012); cereals (Sendin et al., 2018); and fish and seafood (Cheng & Sun, 2014). Hyperspectral imaging has been described as an instrument well suited for food quality assessment and safety inspection using an array of spectral imaging modalities such as NIR, fluorescence, and Raman hyperspectral imaging (Feng & Sun, 2012). Of the hyperspectral imaging techniques available, NIR hyperspectral imaging is the most applicable for rapid analyses because the other modalities are time-consuming and require sample preparation prior to measurement. NIR hyperspectral imaging and its various imaging configurations have proven to be a valuable instrument with applications specific to food (Gowen et al., 2007), pharmaceutical (Gowen et al., 2008), and agricultural domains (Elmasry et al., 2012). This chapter introduces the fundamentals of hyperspectral and multispectral imaging, including the principles, system components and set-up, potentials, limits, advantages and disadvantages. An introduction to spectroscopy, with some theoretical aspects, and the integration of spectroscopy and imaging are briefly discussed. Section 1.6 gives an outline of the key steps involved in analysing hyperspectral images. 1.1.1 The Need for Speed in Food Analysis Food analysis is vital for the entire food industry and all those affiliated with it. Determining and assessing the quality, safety, nutrition, sensorial properties and stability of food products are the primary concerns of food chemists working in the food industry, academia and government laboratories associated with food programs (McGorrin, 2006; Rychlik, 2015). It forms an integral part of a quality management program, as products require analysis throughout the development process, production and even after a ­product is in the market (Nielsen, 2010). The chemical analysis of food is the subject area concerning the development, ­application and study of analytical techniques to characterise food and agricultural products (McGorrin, 2006; Nielsen, 2010). These provide information regarding a wide variety of characteristics of foods, including their structure, composition, physicochemical properties and sensory attributes. This data is critical as it enables new product development and quality control, while troubleshooting product problems and customer complaints. However, with the exponentially growing global population and the everincreasing consumer awareness and concern about the safety, origin, and authenticity of foods, there is exaggerated pressure on food quality managements systems. In addition, traditional (proximate analyses: moisture, ash, fat, protein, carbohydrate) and modern analytical methods (chromatographic, mass spectrometry, and molecular based) employed to characterise food are destructive, require well-trained analysts, are time-consuming and expensive, and often generate harmful waste products. For example, to determine the protein content of food samples, the most commonly used techniques are the Kjeldahl

Fundamentals

5

digestion (Kjeldahl, 1883; Sáez-Plaza et al., 2013) and the Dumas combustion (Dumas & Boussingault, 1844) methods. Both these methods are considered the standard, official methods; however, both are destructive, require skilled analysts, and present potentially hazardous situations if not operated correctly. In addition, the Kjeldahl method requires the use of corrosive reagents (such as concentrated sulphuric acid and sodium hydroxide) and produces toxic waste. Similarly, conventional methods for food microbial analyses are regarded as the “gold standard” in many food microbiology laboratories. The conventional culture techniques, although laborious, are still used in the food industry and food-related research because they offer cost effectiveness, ease of use, and familiarity (Gracias & McKillip, 2004). These include a combination of, but are not limited to, culture and colony techniques (Beuchat, 1992; Baird & Lee, 1995), molecular (Feng, 1997; Elmerdahl Olsen, 2000) and immunological methods (Notermans & Wernars, 1991), such as the polymerase chain reaction (PCR) and enzyme-linked immunosorbent assay (ELISA). This multistep approach can be tedious and laborious and is more likely to introduce errors throughout the process. Furthermore, specialised growth media, supplementary staining and microscopy may also be required, which extends the identification of timeframe, prohibiting shipment and sale of the products. Thus, there is a need for rapid, non-destructive analytical techniques for food quality and safety analysis. Vibrational spectroscopy methods, particularly NIR spectroscopy, have been the subject of research for many years in this regard (Polesello et al., 1983; Davies & Grant, 1987; Scotter, 1990; Osborne et al., 1993; Cen & He, 2007). Although high in spectral resolution, NIR spectroscopy is limited in the spatial dimension, offering no information regarding the location of the constituent or contaminant investigated. Measurements are made across a small area on the sample and averaged. This may lead to discrepancies between predicted and measured constituents. In contrast, NIR hyperspectral imaging is a technique capable of incorporating localization, thus measuring entire samples, rapidly and accurately (Gowen et al., 2007; Boldrini et al., 2012).

1.2  INTRODUCTION TO SPECTROSCOPY Spectroscopy permits scientists to study a sample by describing the energy transfer between light and matter. It is based on the concept that atoms within molecules possess defined energy levels, which may be excited with light radiation (Pasquini, 2003). When a molecule absorbs light of a frequency that supplies exactly the same energy between two levels, it is absorbed causing excitation to a higher energy level. The ultraviolet (UV), visible (VIS), near infrared (NIR) and infrared (IR) regions of the electromagnetic spectrum can be measured with spectral instruments. The UV and VIS regions are categorised as electronic spectroscopic techniques, and the NIR and IR regions are categorised as vibrational techniques. Vibrational spectroscopy, specifically NIR spectroscopy, has gained huge popularity as a tool for determining the chemical and physical properties of food products. The NIR region covers a small part of the electromagnetic spectrum between the VIS and IR regions from 780 to 2,500 nm. The most prominent bands observed during NIR analyses are due to overtones of stretching vibrations and stretching-bending combinations involving major X–H bonds (Ozaki et al., 2006). All molecules ­containing these bonds will give rise to a measurable NIR spectrum, such as O–H bonds (e.g. ­moisture, carbohydrate and fat), C–H bonds (most organic compounds), N–H bonds (e.g. proteins and amino acids) and S–H bonds (e.g. proteins and

6

P.J. Williams and K. Sendin

amino acids). Consequently, NIR analysis is highly suited to the study of biomaterials such as food and food products. Vibrational spectroscopy is based on the concept that the bonds between atoms within molecules vibrate with defined frequencies (Pasquini, 2003). When a molecule absorbs light that corresponds with the frequency of vibration, it is excited to a higher vibrational energy level (v). The energy gap from the ground state (v = 0) to the first level, the fundamental state (v = 1), may be measured as one quantum. If light carries a photon with the energy of one quantum, it is absorbed and the molecule is excited from the ground state to the fundamental state. If a photon possesses two quanta, the first overtone is excited (v = 2), and so on to higher overtones. In addition to the ability of an overtone to be produced at multiples of the fundamental vibrational frequency, combination tones occur where two or more fundamental and/or overtone vibrations combine (through addition and subtraction of energies) to give a single band. Fundamental vibrations of molecules generally originate from absorption in the IR region, while spectra in the NIR region comprise of overtones and combination tones (Manley, 2014). Overtones and combination tones are much less likely to occur than the fundamentals, and thus absorption is usually 10–100 times lower for first overtones with subsequent overtones and combinations continuing to decrease in strength. Electronic spectroscopy involving the UV and VIS regions (160–780 nm) is based on concepts similar to vibrational spectroscopy; however, the excitation by the higher energy UV–VIS radiation gives rise to transitions through electronic energy levels (Sathyanarayana, 2001). This occurs when electrons move from an occupied orbital in the ground state to an appropriate orbital in a higher energy state. Information regarding the covalent bonds within the sample molecules is revealed and can be used for the identification of inorganic and organic species, such as transition metal ions and highly conjugated organic compounds. Spectroscopic measurements use the interactions between electromagnetic radiation and the sample material to provide a detailed fingerprint of the sample. As the energy from the light must be equal to the energy difference between two energy levels, there is a selective response from the molecular system in the sample to the incident light. In the wavelength or spectral range used, frequencies corresponding to bonds present will be absorbed, others that do not correspond will not be absorbed, and some will be partially absorbed. Plotting this absorption versus wavelength produces the spectrum of a sample, which allows qualitative and quantitative assessment of chemical and physical features. However, extracting meaningful information from the spectra, especially in the more popular NIR region, is often not straightforward. As food products contain thousands of X–H bonds, an NIR spectrum contains a large number of bands from overtones and combination tones that overlap heavily with one another. This is referred to as the multicollinearity problem and results in a spectrum with smooth, broad bands (Manley, 2014). It is difficult to assign specific spectral features to specific chemical components in the sample, and visually interpreting the spectrum is not as simple as with other forms of spectroscopy. It is due to this complexity that NIR spectroscopy is currently the analytical technique in which chemometrics is most often applied. Chemometrics is the use of mathematical and statistical techniques for extracting relevant information from analytical data by recognising certain patterns and associating these with the chemical and physical properties under investigation (Esbensen & Geladi, 1989). Chemometrics and NIR technology have evolved together, and as NIR spectroscopy achieves more robust identification and extends its applicability, new challenges arise that motivate the improvement of chemometric techniques.

Fundamentals

7

1.3  INTEGRATION OF SPECTROSCOPY AND IMAGING The food industry has widely applied both imaging and spectroscopic technologies for quality and safety evaluation. Spectral imaging integrates the two technologies by combining their main features to acquire spatial and spectral information simultaneously. Conventional imaging technology is one of the mostly widely used alternatives to manual inspection and has become an integral part of the food industry’s move toward automation (Sun, 2010). Imaging systems usually include computer vision by a camera utilizing either monochromatic (black and white) or polychromatic (color-based) light, complemented with image processing and analysis, involving mathematics, computer programming, and software programming. These systems acquire either two- or threedimensional spatial information and can often assess several objects per second, leading to very high online throughput (ElMasry & Sun, 2010). Imaging methods allow the quantification or classification of a sample’s spatial and colour attributes. External characteristics are often important factors in consumers’ perception of a food product, such as assessing quality and maturity. Thus, imaging technologies are widely used for monitoring the color, shape, size, surface texture, and external defects in food and agricultural products (Lu & Park, 2015). However, as these imaging technologies only use VIS wavelengths, an image of the external view is acquired that it cannot be used to detect or assess the intrinsic physical and chemical properties of a sample. Furthermore, imaging systems may have difficulty performing complex classifications such as distinguishing similar colours or detecting invisible defects. As discussed, spectroscopy offers superior detection results and is used extensively in the food industry to perform a wide range of qualitative and quantitative analyses. Diffusely reflected light is recorded as the spectral response, which contains information about absorbing chemical groups near the surface of a sample. These analyses are conducted based on correlation that exists between the spectral response and a specific chemical or physical property of the food product. Spectroscopy is widely used in industry to replace current conventional analytical methods, as it is non-invasive, non-destructive, rapid, and easy to implement online (Sun, 2010). Furthermore, once spectral information has been gathered, the data has the potential to be used for evaluating multiple chemical or physical properties. A constraint of conventional spectroscopic methods is that the information is not spatially resolved. Certain applications require the spatial distribution of chemical or physical properties. Instead, only a small portion of the sample is scanned and an average spectrum or, if a calibration model exists, a prediction value is acquired. However, the scan may not be representative of the entire sample or sample group, especially if it is heterogeneous. If a defect or contaminant is confined to a small area on the sample, the area may not be scanned at all and will remain undetected. Furthermore, if the aim of the analysis is to sort or separate defective food products, an average difference in spectral response may be observed but the individual defective objects will not be identified. The respective merits and shortcomings of conventional imaging and spectroscopic techniques highlight the advantages of combining the two platforms into one analytical system. Acquiring spatial positional information with spectra gives hyperspectral imaging the capability to measure, inspect, sort, and grade food products with remarkable accuracy and efficiency (Sendin et al., 2018). The applicability in the food industry is largely due to the fact that food and agricultural products exhibit large variations in properties and composition within an individual sample or in a sample group. Spatially resolved spectral information can be used, for example, to identify or separate individual samples

8

P.J. Williams and K. Sendin

in a sample group based on chemical or physical properties, or map spatial distributions (e.g. distribution of moisture) throughout a sample or group of samples. Hyperspectral imaging systems provide the food industry with a real-time, non-destructive alternative to manual or conventional methods that delivers cost effective, consistent, rapid and accurate results.

1.4  INTRODUCTION TO HYPERSPECTRAL IMAGING 1.4.1 Brief History Hyperspectral imaging techniques were first adopted in the field of remote sensing for ­airborne detection and mapping (Goetz et al., 1985). Prominent initial applications include military use for detection of hidden vehicles in dense vegetation, and the geological uses for the detection and analysis of a variety of surface minerals, vegetation, ice and snow, as well as for characterising soil properties including moisture, organic matter content and salinity. The key development spurring the advancement of hyperspectral imaging was the invention of the first hybrid area array detectors, using mercury–cadmium–telluride on silicon charge-coupled devices (CCDs), by Boyle and Smith (1970). This allowed a camera to record both spatial and spectral information of wavelengths beyond 1,000 nm. Due to the advancement of electronics, hardware, computing and software through the 1980s and 1990s, the scientific community began to embrace the technique in a wider range of applications. Much of the advancement was due to work conducted in the mid-1980s by Alexander Goetz and his colleagues at the National Aeronautics and Space Administration Jet Propulsion Laboratory (NASA/JPL) during the development of the hyperspectral Airborne Visible/Infrared Imaging Spectrophotometer (AVIRIS). As data increased in quality, statistical techniques adapted from established chemometric methods for NIR spectroscopy were first adopted for hyperspectral image analysis. During this progression, spectral imaging researchers were posed with the challenge of datasets becoming too large and the merit of information extraction by statistical techniques was recognised. This continually increasing spectral image quality paired with the incredible simultaneous advances in desktop computing allowed the application of image processing and statistical analysis, and opened doors for hyperspectral imaging research to flourish. 1.4.2 Principles of Hyperspectral Imaging Hyperspectral imaging is the simultaneous acquisition of spatial images in numerous spectrally congruent bands (Burger & Geladi, 2006). A hyperspectral image comprises hundreds of two-dimensional black and white or greyscale image planes, with one image plane for each waveband. The hyperspectral image thus consists of three dimensions, two spatial dimensions, length (x) by width (y), and one spectral dimension (λ). This threeway data matrix is generally termed the hypercube, or alternatively spectral cube, data cube or spectral volume (Manley, 2014). The hypercube can be examined in various useful ways (Figure 1.1), including as an image plane of the sample at a single waveband, for example, to examine moisture distribution throughout a sample at 1,450 nm, or as the full spectrum of a single

Fundamentals

9

FIGURE 1.1  A hyperspectral image or hypercube is comprised of two spatial dimensions

(x and y) and one spectral dimension (λ). This may be examined as an image plane of the entire sample at a chosen wavelength (e.g. at 2,000 nm), or as a spectrum of a pixel in the sample. pixel to investigate the chemical composition of a specific point. Pixels in hyperspectral image data are digitised grey values, or intensities for each waveband, and may be expressed as integers. These intensity values are acquired with 8-, 12-, 14- or 16-bit grey values, where 12-bit is generally adequate for most applications (ElMasry & Sun, 2010). Each pixel possesses a full spectrum or spectral signature, which is generated by plotting the recorded light intensity against each waveband in the range used. With appropriate reference materials and data analysis, these spectral signatures can be used to routinely classify, detect or differentiate the given materials in every pixel of a sample’s hypercube. 1.4.3 Basic Hyperspectral System Components and Set-Up Typical hyperspectral imaging systems consist of four main parts, namely an imaging unit, a light source, a sample stage, and a computer. The specific parts used, configuration, and image acquisition mode will be dictated by the desired application. One of the most important considerations is the spectral range, which is the wavelength region covered by the instrument (ElMasry & Sun, 2010). Hyperspectral imaging systems may operate in the UV, VIS, NIR or IR regions, with the VIS and NIR regions being most

10

P.J. Williams and K. Sendin

popular in food analysis. In addition, the spectral and spatial resolution must be considered. The spectral resolution relates to the ability to distinguish two adjacent wavebands, while spatial resolution describes the minimum size of a distinguishable object in the image. Both of these resolution parameters will affect the quality of the acquired image dataset and will also determine its size. One must consider that while having high quality data is important for achieving accurate calibrations and results, large datasets may not be suitable for applications requiring rapid analysis times. The imaging unit, arguably the most important element, is made up of a zoom lens, a spectrograph and a camera (Figure 1.2). The function of a spectrograph is to disperse the light reflected from the sample through the lens into a continuous spectral range, which will be recorded by the camera. Most spectrographs achieve this by including optical devices, each suited for different acquisition modes, such as a prism, a diffraction grating, or a liquid crystal tunable filter (LCTF) or acousto-optic tunable filter (AOTF) (Pu et al., 2015). When operating in the VIS and NIR range of 400–1,000 nm, cameras will use a lower cost CCD or complementary metal oxide semiconductor (CMOS) siliconbased camera to collect both spectral and spatial information (Sendin et al., 2018). If NIR wavelengths of 1,100–2,500 nm are required, an expensive indium gallium arsenide (InGaAs) or mercury cadmium telluride (HgCdTe) array detector is used. The light source used will also be determined by the desired spectral range. When operating in the VIS and NIR regions, tungsten–halogen lamps are most commonly used (Sendin et al., 2018). This is due to their durability, stability and capability to emit light of a broad ­spectral range (400–2,500 nm). Other possible light sources include quartz–halogen lamps, light emitting diodes (LEDs), tunable lasers and heated xenon lamps. However, LED sources are restricted to a narrow range of 400–900 nm. Three instrumentation approaches are used to acquire hyperspectral images: ­staring scan, push-broom (line) scan, or whisk-broom (point) scan (Dale et al., 2013). Hyperspectral image data are a three-dimensional hypercube, with two spatial dimensions

FIGURE 1.2  Schematic of a push-broom hyperspectral imaging system, illustrating the

line-by-line data acquisition of the samples on the linear translation stage.

Fundamentals

11

and one spectral dimension. There is currently no means of acquiring all three dimensions simultaneously, and thus there are different combinations in which one or two of these dimensions are gathered either simultaneously or sequentially. The simplest approach is staring, in which the image field of view is fixed and an image plane is collected at one waveband after another. In other words, both spatial dimensions are acquired simultaneously and the spectral dimension is acquired sequentially. Acquiring an image at different wavelengths using this configuration requires a tunable filter, such as an LCTF or an AOTF (ElMasry & Sun, 2010). As this process is inherently slow, it is only preferable when fewer wavebands are used, and is well suited to multispectral imaging. In addition to the obvious issue of time restraints in industrial uses, lengthy image acquisition times (often several minutes) may be a concern when working with certain food samples which may undergo changes due to heating by the continuous illumination from light source. Lastly, the sample must remain in a fixed position during image acquisition, and thus this mode would not be suitable for production line implementation. The second approach is whisk-broom or point scan, where the full spectrum of each point is acquired individually. All the three dimensions are acquired separately as the sample is moved in a zigzag pattern on a grid of points covering the whole image. As the entire spectrum is gathered for an individual pixel at once, a tunable filter is not required; instead, an optical grating, prism or a similar dispersing element is used. This acquisition mode is popular for microscopic imaging where the acquisition time is usually not a problem. However, since a double scan (i.e. spatial and spectral) is required, this mode is also not suitable for implementation on a production line. The third mode, and currently most popular in hyperspectral imaging, is pushbroom or line scan (Figure 1.2). This method acquires the full spectrum of one line of pixels at a time. Thus, the spectral dimension and one spatial dimension are acquired simultaneously, while the second spatial dimension is acquired sequentially. Similar to whisk-broom instruments, a dispersing element is used in the spectrograph. However, as an entire line of pixels is recorded at once, a two-dimensional dispersing element and a two-dimensional detector array are required. As this method does not require the changing of filters and only requires the sample to be moved in one direction (the direction of the second spatial dimension), it is well suited to implementation on a conveyor system at relatively high speeds (5–10 s/m conveyor belt).

1.5  MULTISPECTRAL IMAGING Hyperspectral imaging is a powerful analytical tool for non-destructive food analysis by providing a full spectral measurement over a continuous wavelength range for every pixel in an image. This data contain a wealth of information concerning the sample’s chemical components, for example, moisture, carbohydrates, proteins and other hydrogen-­bonded constituents, as well as physical properties, such as size, shape, and texture. Hyperspectral imaging is ideal for research purposes, as there is little constraint of time and computing power in a research laboratory, and acquiring a large and complete hyperspectral dataset allows for more sophisticated analysis to be conducted. As a result, an abundance of hyperspectral imaging research in food science and beyond has ­demonstrated the ability of hyperspectral imaging to evaluate the properties of food product vital to the food industry (Pu et al., 2015; Sendin et al., 2018; Cheng et al., 2017). Despite demonstrating this huge potential for food industry implementation, there are two primary constraints

12

P.J. Williams and K. Sendin

of the technique that discourage its widespread use. First, hyperspectral systems are high in cost, and second, they generate extremely large amounts of data in a short time, resulting in high computational loads that hinder the development of efficient real-time applications. The best way to overcome both problems is to reduce the number of wavelengths recorded by the hyperspectral system, thus developing a multispectral system. Multispectral imaging systems are based on the same concept as hyperspectral imaging, but use far fewer wavebands (~2–20) which are irregularly spaced, as opposed to the continuous spectra in hyperspectral image data. While a typical hyperspectral instrument acquires ~250 wavebands, only a few are highly related to certain quality attributes. Therefore, one may select a handful of optimal wavebands and preserve only the most relevant information. By minimising the size of the hypercube, redundant information is eliminated from the chemometric prediction model and real-time predictions are possible due to improved analysis speeds. Furthermore, due to the issue of multicollinearity associated with spectroscopic techniques utilising the NIR region, it has been found that the high spectral resolution of hyperspectral images does not necessarily lead to increased analytical precision, and the accuracy of predictions may be improved by removing redundant wavebands (Elmasry et al., 2012). If a waveband does not carry useful spectral information, it will only add noise to a chemometric model, and thus reducing the high dimensionality of the hyperspectral image dataset may help overcome this issue. Lastly, the cost of the system is also significantly reduced if only a handful of key wavelengths are used as an instrument can be built using simpler, and thus less expensive, components. Multispectral systems are generally developed for a single specific inspection application, for example, use of four wavebands to separate Arabica and Robusta coffee beans (Calvini et al., 2017). The process of developing a multispectral system usually begins with spectral variable selection derived from hyperspectral image data. The simplest method for wavelength selection is based on the maximum or minimum intensity difference in the spectra (Pu et al., 2015). However, the selected wavebands might not be the most significant ones, for example, if the high response is due to a major chemical component that does not vary between the classes under investigation. Instead, a host of waveband extraction methods based on data analysis methods have been reported, including spectrum derivatives, principal component analysis (PCA), partial least squares (PLS), back propagation neural networks (BPNN), and artificial neural networks (ANN) (Elmasry et al., 2012; Pu et al., 2015). Once the waveband selection is made, the multispectral system will be built. A staring instrument is the less-expensive set-up option, utilizing an LCTF or an AOTF. Alternatively, one can inexpensively assemble a simple multispectral imaging system using commercially available filters that best match the selected key wavebands that are sequentially changed in front of the camera. These staring systems could be used in a facility that does not require testing to take place on- or in-line and can be built as cheaply as ~$5,000 (US). Applications requiring on- or in-line assessment will require a push-broom set-up to facilitate the movement of samples on a conveyor belt, which can cost more due to the need of a more complex spectrograph (~$15,000), but will cost significantly less than a typical hyperspectral push-broom system (~$400,000).

1.6  ANALYSING HYPERSPECTRAL IMAGES Hyperspectral imaging produces enormous amounts of data from a single sample, and with thousands of samples that require analysis daily, as would be the norm in the food

Fundamentals

13

FIGURE 1.3  Schematic indicating the steps involved in hyperspectral image analysis [Adapted from Dorrepaal et al. (2016)].

industry, this poses a potential data analysis problem. This magnitude of data requires a combination of chemometrics and visualisation tools to adequately mine for meaningful information. Furthermore, in order to automate these methods in an online system, one requires powerful data analysis tools. Multivariate image analysis (MIA) offers a platform to do this in an efficient way. The MIA is a data analysis technique employed for the exploration and interpretation of hypercubes (Esbensen & Geladi, 1989; Geladi & Grahn, 1996; Burger & Geladi, 2007a). This approach consists of a number of steps (Figure 1.3) that ultimately results in rapid sample classification, object identification, sample constituent prediction, and/or visualisation. After image acquisition, the raw cube is corrected according to Equation 1.1, where I is the relative reflectance image, I 0 is the raw reflectance image, D is the dark reference image, and W is the white reference image. This is commonly referred to as ­reflectance ­c alibration and is performed to account for the background spectral response of the instrument and the “dark current” camera response (Gowen et al., 2007).

Reflectance =

I0 − D (1.1) W −D

However, since absorbance is directly proportional to concentration according to the Beer–Lambert law (Swinehart, 1962), the analysis of NIR hyperspectral image data often involves a conversion to absorbance values (Equation 1.2), where the logarithm of reflectance is taken.

 I −D Absorbance = log10  0 (1.2)  W − D 

Once the hypercube has been corrected and unfolded, an array of pre-processing techniques may be applied to remove effects of scattering, to improve signal-to-noise ratio and to eliminate physical inconsistencies caused by non-uniform lighting. The most widely used spectral pre-processing techniques are the Savitzky–Golay derivatives (Savitzky & Golay, 1964), standard normal variate (SNV) (Barnes et al., 1989) and multiplicative scatter correction (MSC) (Geladi et al., 1985). These techniques were shown to be useful on NIR hyperspectral image data (Burger & Geladi, 2007b). Additional procedures may include thresholding and masking to remove redundant background information from the hypercube. Because of the size of these hypercubes and the succeeding dataset, d ­ imensionality reduction techniques are essential. PCA is well suited for this purpose and is crucial for data reduction and examination of NIR spectra (Cowe & McNicol, 1985) and exploration of hyperspectral images (Esbensen & Geladi, 1989; Burger & Geladi, 2006).

14

P.J. Williams and K. Sendin

However, for hyperspectral images, an unfolding step is first required. Prior to any data treatment, the hypercube is reorganised into a matrix X, where the pixel information (x, y) are the observations and the wavelengths (λ) are the variables. NIR hyperspectral imaging is ideal for defect or contamination detection. It is perfectly suited for discriminant studies, to compare products/samples with each other in order to identify irregularities or impurities by means of classification. In addition, if quantitative results are required, hyperspectral images can be used to predict and visualise various constituents (moisture, protein, and fat) within a sample, provided that reference measurements have been performed. These are achieved by calculating PLS discriminant analysis (PLS-DA) (Chevallier et al., 2006) or PLS regression (Burger & Geladi, 2006) models on the unfolded cube. PLS-DA is similar to PLS, that uses the latent variable approach to find fundamental relations between two matrices (X  and  Y) (Martens, 2001; Wold et al., 2001; Liu & Rayens, 2007). PLS uses the y-data structure to decompose X so that the outcome constitutes an optimal regression vector. PLS-DA operates similarly; however, instead of measured y data, dummy variables are used, which are indicators of groups (Chevallier et al., 2006). This allows for prediction of group membership, and thus classification of pixels to classes. Similar to PCA, it is also possible to build synthetic images showing the predicted groups. There are numerous other chemometric techniques and data handling methods for hyperspectral images (Burger & Gowen, 2011; Elmasry et al., 2012) and will be discussed in Chapters 5–11.

1.7  ADVANTAGES AND DISADVANTAGES 1.7.1 Advantages Many of the obvious advantages of hyperspectral imaging stem from the technique’s basis in conventional spectroscopy, particularly NIR spectroscopy. The first advantage is the low cost per analysis. The technique requires little sample preparation and does not make use of chemicals and solvents, which increases safety, reduces negative environmental impact and saves money by avoiding both reagent and waste-treatment costs (Dale et  al., 2013). Furthermore, once a calibration has been built and validated, analysis is simple and rapid, and labor cost is reduced. When captured, hyperspectral image data contain all chemical information in a given spectral range. Thus, with appropriate data analysis, it is capable of simultaneously determining several quantitative or qualitative properties with a single scan. In addition, the non-destructive and non-invasive technique allows specific samples to either be germinated after scanning (e.g. analysing seeds) or rescanned at different stages through a study. By integrating conventional spectroscopy with imaging, further advantages are achieved. With conventional techniques, one scan gives one average spectrum, thus indicating only the bulk composition. By measuring pixels in an image, thousands of spectra are obtained during hyperspectral imaging. Not only does this allow more pixels from a selected region of interest to be included in a spectral database, but also allows for chemical maps to be generated. Chemical maps visualise the spatial distribution and concentration of various chemical constituents in the food sample based on their spectral signatures, where regions of similar spectral properties should have similar chemical composition.

Fundamentals

15

1.7.2 Disadvantages Despite many advantages of hyperspectral imaging, and the wide use in food research, its application is limited largely due to the cost of the instrument, particularly of the spectrograph and camera. The required computing power and storage further increases this high cost. While the technique offers numerous benefits to the food industry, the initial cost associated with implementing a hyperspectral imaging system often seems to outweigh the benefits to the production facility. As discussed, multispectral imaging is a potential cheaper alternative ($15,000 vs. $400,000), but the system must be tailor-made for the application by selecting the most appropriate wavelengths for the specific sample. While desktop computing and software have improved substantially since the advent of hyperspectral imaging, some production lines simply move samples too quickly for the image acquisition and analysis times currently offered by push-broom hyperspectral imaging instruments. This is another factor majorly limiting its use. These times are restricted as hyperspectral image datasets are huge and include large amounts of redundant information. Acquiring the dataset and extracting the important information to generate a result requires a relatively long time using today’s computing power. Again, multispectral imaging may be used to overcome this issue (2 vs. 10 s/m conveyor belt). As in conventional NIR spectroscopy, multicollinearity problems may occur in hyperspectral image datasets. This is due to the large number of overtones and combination tones that occur in the contiguous wavebands, which begin to overlap and give rise to spectra with broad bands as opposed to sharp peaks. Due to this overlap, it is difficult to ascertain whether a band is related to the chemical constituent under investigation. Despite the use of complex chemometric methods to reduce these issues, hyperspectral models will often contain some degree of repeated, redundant or irrelevant information. Lastly, the main analytical drawback of hyperspectral imaging, and spectroscopy in general, is that it is an indirect method. For instance, instead of directly measuring the content of moisture (e.g. measuring evaporative weight loss), one uses a measurement of the reflected light to give an accurate but indirect indication of the moisture content. This requires the initial calibration of a model using references, which are samples that have also been measured using direct methods. By the laws of mathematics, the error associated with an indirect method will always be equal to or greater than the error of the reference method used. An accurate calibration generally requires a large database with a wide range of spectral signatures and reference values over multiple seasons, harvests, etc., which will require a significant amount of time, labor and cost in reference analytical methods to build. Once calibrated, model transfer to other instruments poses further issues. It entails routine standardisation between instruments, which can be especially difficult if separated geographically as standards must be transported unchanged between all facilities.

1.8 CONCLUSION With more focus being placed on food authenticity, food assurance and lately, food reassurance, consumers and governments alike demand increasingly more information regarding the origin, composition and safety of their food. This places immense pressure on the food industry to provide swift, reliable and accurate results. Hence, rapid analytical methods are required to perform routine analyses that ensure accurate, reliable data. This is essential in the food industry where routine analyses are performed daily. With

16

P.J. Williams and K. Sendin

the ability to identify and indicate location of possible contaminants, hyperspectral imaging is an excellent tool for food analyses. Hyperspectral imaging is a rapid, non-destructive technique that is capable of capturing data regarding a sample in the form of images. The data are arranged in three-way matrices with two spatial pixel coordinates and one wavelength dimension. Not only does this permit identification and localisation of chemical components within samples, it also enables classification and discrimination of samples into various predefined categories. The relevance to the food industry is obvious, as spatially resolved information can be used to identify or separate individual samples in a sample group or map spatial distributions (e.g. distribution of moisture or protein) throughout a sample or group of samples. Hyperspectral imaging systems provide the food industry with a real-time, nondestructive alternative to conventional methods that delivers cost effective, consistent, rapid and accurate results.

REFERENCES Baird, R.M. & Lee, W.H. (1995). Media used in the detection and enumeration of Staphylococcus aureus. International Journal of Food Microbiology, 26, 15–24. Barnes, R.J., Dhanoa, M.A. & Lister, S.J. (1989). Standard normal variate transformation and de-trending of near-infrared diffuse reflectance spectra. Applied Spectroscopy, 43, 772–777. Beuchat, L.R. (1992). Media for detecting and enumerating yeasts and moulds. International Journal of Food Microbiology, 17, 145–158. Boldrini, B., Kessler, W., Rebner, K. & Kessler, R. (2012). Hyperspectral imaging: A review of best practice, performance and pitfalls for inline and online applications. Journal of Near Infrared Spectroscopy, 20, 438–508. Boyle, W.S. & Smith, G.E. (1970). Charge coupled semiconductor devices. Bell System Technical Journal, 49, 587–593. Burger, J. & Geladi, P. (2005). Hyperspectral NIR image regression part I: Calibration and correction. Journal of Chemometrics, 19, 355–363. Burger, J. & Geladi, P. (2006). Hyperspectral NIR imaging for calibration and prediction: A comparison between image and spectrometer data for studying organic and biological samples. Analyst, 131, 1152–1160. Burger, J. & Geladi, P. (2007a). Hyperspectral image data conditioning and regression analysis. In: Techniques and Applications of Hyperspectral Image Analysis (edited by H.F. Grahn & P. Geladi), pp. 127–153. Chichester, West Sussex, England: John Wiley & Sons, Ltd. Burger, J. & Geladi, P. (2007b). Spectral pre-treatments of hyperspectral near infrared images: Analysis of diffuse reflectance scattering. Journal of Near Infrared Spectroscopy, 15, 29. Burger, J. & Gowen, A. (2011). Data handling in hyperspectral image analysis. Chemometrics and Intelligent Laboratory Systems, 108, 13–22. Calvini, R., Amigo, J.M. & Ulrici, A. (2017). Transferring results from NIR-hyperspectral to NIR-multispectral imaging systems: A filter-based simulation applied to the classification of Arabica and Robusta green coffee. Analytica Chimica Acta, 967, 33–41. Cen, H.Y. & He, Y. (2007). Theory and application of near infrared reflectance spectroscopy in determination of food quality. Trends in Food Science and Technology, 18, 72–83.

Fundamentals

17

Cheng, J.-H., Nicolai, B. & Sun, D.-W. (2017). Hyperspectral imaging with multivariate analysis for technological parameters prediction and classification of muscle foods: A review. Meat Science, 123, 182–191. Cheng, J.-H. & Sun, D.-W. (2014). Hyperspectral imaging as an effective tool for q ­ uality analysis and control of fish and other seafoods: Current research and potential applications. Trends in Food Science & Technology, 37, 78–91. Chevallier, S., Bertrand, D., Kohler, A. & Courcoux, P. (2006). Application of PLS-DA in multivariate image analysis. Journal of Chemometrics, 20, 221–229. Cowe, I. & McNicol, J. (1985). The use of principal components in the analysis of nearinfrared spectra. Applied Spectroscopy, 39, 257–266. Dale, L.M., Thewis, A., Boudry, C., Rotar, I., Dardenne, P., Baeten, V. & Pierna, J.A.F. (2013). Hyperspectral imaging applications in agriculture and agro-food product quality and safety control: A review. Applied Spectroscopy Reviews, 48, 142–159. Davies, A. & Grant, A. (1987). Near infra‐red analysis of food. International Journal of Food Science & Technology, 22, 191–207. Dorrepaal, R., Malegori, C. & Gowen, A. (2016). Tutorial: Time series hyperspectral image analysis. Journal of Near Infrared Spectroscopy, 24, 89–107. Dumas, J.-B. & Boussingault, J.B. (1844). The Chemical and Physiological Balance of Organic Nature: An Essay. New York, NY: Saxton & Miles. Elmasry, G., Barbin, D.F., Sun, D.-W. & Allen, P. (2011). Meat quality evaluation by hyperspectral imaging technique: An overview. Critical Reviews in Food Science and Nutrition, 52, 689–711. Elmasry, G., Kamruzzaman, M., Sun, D.W. & Allen, P. (2012). Principles and applications of hyperspectral imaging in quality evaluation of agro-food products: A review. Critical Reviews in Food Science and Nutrition, 52, 999–1023. ElMasry, G. & Sun, D.-W. (2010). Principles of hyperspectral imaging technology. In: Hyperspectral Imaging for Food Quality Analysis and Control (edited by Professor S. Da-Wen), pp. 3–43. San Diego: Academic Press. Elmerdahl Olsen, J. (2000). DNA-based methods for detection of food-borne bacterial pathogens. Food Research International, 33, 257–266. Esbensen, K.H. & Geladi, P. (1989). Strategy of multivariate image analysis (MIA). Chemometrics and Intelligent Laboratory Systems, 7, 67–86. Feng, P. (1997). Impact of molecular biology on the detection of foodborne pathogens. Molecular Biotechnology, 7, 267–278. Feng, Y.-Z. & Sun, D.-W. (2012). Application of hyperspectral imaging in food safety inspection and control: A review. Critical Reviews in Food Science and Nutrition, 52, 1039–1058. Geladi, P. & Grahn, H.F. (1996). Multivariate Image Analysis. Chichester, West Sussex, England: John Wiley & Sons Ltd. Geladi, P., MacDougall, D. & Martens, H. (1985). Linearization and scatter-correction for near-infrared reflectance spectra of meat. Applied Spectroscopy, 39, 491–500. Goetz, A.F., Vane, G., Solomon, J.E. & Rock, B.N. (1985). Imaging spectrometry for Earth remote sensing. Science, 228, 1147–1153. Gowen, A.A., Feng, Y., Gaston, E. & Valdramidis, V. (2015). Recent applications of hyperspectral imaging in microbiology. Talanta, 137, 43–54. Gowen, A.A., O’Donnell, C.P., Cullen, P.J. & Bell, S.E.J. (2008). Recent applications of chemical imaging to pharmaceutical process monitoring and quality control. European Journal of Pharmaceutics and Biopharmaceutics, 69, 10–22.

18

P.J. Williams and K. Sendin

Gowen, A.A., O’Donnell, C.P., Cullen, P.J., Downey, G. & Frias, J.M. (2007). Hyperspectral imaging: An emerging process analytical tool for food quality and safety control. Trends in Food Science and Technology, 18, 590–598. Gracias, K.S. & McKillip, J.L. (2004). A review of conventional detection and enumeration methods for pathogenic bacteria in food. Canadian Journal of Microbiology, 50, 883–890. Kjeldahl, J. (1883). Neue Methode zur Bestimmung des Stickstoffs in organischen Körpern. Zeitschrift fur Analytische Chemie, 22, 366–382. Liu, Y. & Rayens, W. (2007). PLS and dimension reduction for classification. Computational Statistics, 22, 189–208. Lorente, D., Aleixos, N., Gómez-Sanchis, J., Cubero, S., García-Navarrete, O.L. & Blasco, J. (2012). Recent advances and applications of hyperspectral imaging for fruit and vegetable quality assessment. Food and Bioprocess Technology, 5, 1121–1142. Lu, R. & Park, B. (2015). Introduction. In: Hyperspectral Imaging Technology in Food and Agriculture (edited by B. Park & R. Lu), pp. 3–7. New York, NY: Springer. Manley, M. (2014). Near-infrared spectroscopy and hyperspectral imaging: Nondestructive analysis of biological materials. Chemical Society Reviews, 43, 8200–8214. Martens, H. (2001). Reliable and relevant modelling of real world data: A personal account of the development of PLS Regression. Chemometrics and Intelligent Laboratory Systems, 58, 85–95. McGorrin, R.J. (2006). Food analysis techniques: Introduction. In: Encyclopedia of Analytical Chemistry. Hoboken, NJ: John Wiley & Sons, Ltd. Nielsen, S.S. (2010). Introduction to food analysis. In: Food Analysis, pp. 3–14. Boston, MA: Springer. Notermans, S. & Wernars, K. (1991). Immunological methods for detection of foodborne pathogens and their toxins. International Journal of Food Microbiology, 12, 91–102. Osborne, B.G., Fearn, T. & Hindle, P.H. (1993). Practical NIR Spectroscopy with Applications in Food and Beverage Analysis, pp. 29–33. Essex, England: Longman Scientific & Technical. Ozaki, Y., McClure, W.F. & Christy, A.A. (2006). Near-Infrared Spectroscopy in Food Science and Technology. Hoboken, NJ: John Wiley & Sons. Pasquini, C. (2003). Near infrared spectroscopy: Fundamentals, practical aspects and analytical applications. Journal of the Brazilian Chemical Society, 14, 198–219. Polesello, A., Giangiacomo, R. & Dull, G.G. (1983). Application of near infrared spectrophotometry to the nondestructive analysis of foods: A review of experimental results. Critical Reviews in Food Science and Nutrition, 18, 203–230. Pu, Y.Y., Feng, Y.Z. & Sun, D.W. (2015). Recent progress of hyperspectral imaging on quality and safety inspection of fruits and vegetables: A review. Comprehensive Reviews in Food Science and Food Safety, 14, 176–188. Robert, P., Devaux, M. & Bertrand, D. (1991). Near infrared video image analysis. Sciences des aliments, 11, 565–574. Rychlik, M. (2015). Challenges in food chemistry. Frontiers in Nutrition, 2, 11. Sáez-Plaza, P., Michałowski, T., Navas, M.J., Asuero, A.G. & Wybraniec, S. (2013). An overview of the Kjeldahl method of nitrogen determination. Part I. Early history, chemistry of the procedure, and titrimetric finish. Critical Reviews in Analytical Chemistry, 43, 178–223. Sathyanarayana, D. (2001). General introduction. In: Electronic Absorption Spectroscopy and Related Techniques, pp. 1–21. Hyderabad, India: Universities Press.

Fundamentals

19

Savitzky, A. & Golay, M. (1964). Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry, 36, 1627–1639. Scotter, C. (1990). Use of near infrared spectroscopy in the food industry with particular reference to its applications to on/in-line food processes. Food Control, 1, 142–149. Sendin, K., Williams, P.J. & Manley, M. (2018). Near infrared hyperspectral imaging in quality and safety evaluation of cereals. Critical Reviews in Food Science and Nutrition, 58(4), 575–590. Sun, D.-W. (2010). Hyperspectral Imaging for Food Quality Analysis and Control. San Diego, CA: Academic Press. Swinehart, D.F. (1962). The Beer–Lambert law. Journal of Chemical Education, 39, 333. Wold, S., Sjöström, M. & Eriksson, L. (2001). PLS-regression: A basic tool of chemometrics. Chemometrics and Intelligent Laboratory Systems, 58, 109–130.

Chapter

2

Optimization of Hyperspectral Image Cube Acquisition A Case Study on Meat and Bone Meal Cecilia Riccioli, Ana Garrido Varo, and Dolores Pérez Marin Universidad de Cordoba

CONTENTS 2.1 Introduction 21 2.2 Material and Methods 22 2.2.1 Hyperspectral Imaging Equipment 22 2.2.2 Comparison of Dark Reference Materials 23 2.2.3 Scanning Frequency for Dark and White References 24 2.2.4 Ambient Light 24 2.2.5 Sample Presentation 25 2.2.6 Mathematical Data Treatment 25 2.3 Results and Discussion 26 2.3.1 Dark Reference Material 26 2.3.2 Scanning Frequency of Dark and White References 28 2.3.3 Ambient Light 30 2.3.4 Sample Presentation 31 2.4 Conclusions 31 References 32

2.1 INTRODUCTION Hyperspectral chemical imaging (HCI) is gaining popularity as a non-destructive tool for the quality monitoring of agricultural and food products.1,2 HCI, combining spectral and spatial information, is one of the candidate methods for rapid species identification in meat and bone meal (MBM), since current legislation bans the use of MBM and allows the use of fishmeal only for non-ruminants. Compared to traditional near infrared (NIR) spectroscopy, HCI has the advantage of providing the opportunity to investigate localized micro-domains within a product. A physical or chemical abnormality that makes a minimal contribution to the bulk sample may go undetected by traditional NIR spectroscopy. The same abnormality could dominate a micro-domain and be detected in the NIR image.3

21

22

Cecilia Riccioli, et al.

A generic problem in HCI is the very low spatial and spectral repeatability of image cubes. Increasing attention is being paid to this issue by researchers investigating the use of HCI in remote sensing. A number of authors have recently published papers on procedures for quantifying and improving spectral repeatability.4 Manufacturers of HCI systems use sophisticated equipment to accurately calibrate their instruments, under optimal illumination and constant environmental conditions. From a user’s perspective, these calibration procedures are only of marginal interest, since repeatability is “target dependent”. The analyst of HCI images is primarily interested in the reliability of the end-product, that is, the repeatability of two image cubes consecutively acquired over the same target. 5 When the magnitude of non-repeatability variance is similar to the variance of the spectral or spatial information of interest, it would be impossible to use it for classification or quantification of prediction modeling. Another key factor for the successful real-time application of HCI to food production is the acquisition of spectral images at high speeds. HCI systems should be designed and built to utilize all possible efficiencies in the illumination and detection systems, in order to optimize system’s signal-to-noise (S/N) ratio. The requirements of speed also place a special burden on the need for high-speed software, which can manipulate the hyperspectral images in real time and apply algorithms for measurement and discrimination.6 There are several factors that influence the speediness and efficiency of an HCI system. Researchers and instrument producers have different views about how often dark and white references should be analyzed for time and image-quality optimization.7 This is a crucial issue, especially when changes in temperature occur. Most cameras appearing on the market over the last 10 years are thermally stabilized using different cooler systems.8 The tolerance of NIR calibration to temperature variations can be improved by correction of the image as often as possible during sample image acquisition. The challenge is to reach a compromise between the analysis speed and the elimination of ­temperature-change effects. Another factor to be considered for optimal image acquisition is the material used for the dark reference, since reference reflectance/absorbance features influence the range intensity values of the signals from the detector.9 Sample presentation features also affect the quality of the image; in fact, the ability to discriminate sample pixels from non-sample pixels depends on the support used during analysis. Finally, the control of external light must be borne in mind during hyperspectral image acquisition, since any i­mpairment of the uniformity of lighting conditions can affect image quality. The following sections describe the implementation of a methodology for the correct acquisition of hyperspectral images, with a view to subsequently ensuring the development of statistically robust models for the qualitative and quantitative characterization of MBM using HCI.

2.2  MATERIAL AND METHODS 2.2.1 Hyperspectral Imaging Equipment An NIR camera (MatrixNIR, Malvern Instruments, Maryland, USA) with liquid crystal tunable filters (LCTF) was used to acquire images of samples, which were illuminated by four halogen light sources. Incident light of the lamps passed through polarizing

Optimization of Hyperspectral Image Cube Acquisition

23

filters oriented in the perpendicular direction so that specular reflectance was reduced. The camera, filter and light sources were controlled through a Dell computer (Dell Inc., Round Rock, TX, USA). The camera had an Indium Gallium Arsenide (InGaAs) focalplane array, thermoelectrically cooled detector able to acquire 240 × 320 pixel images with a resolution of 97.8 μm/pixel in the NIR region. Instrument parameters were set as shown below: • Integration time (IT): 32 ms. This camera setting, also known as exposure time, is defined as the time during which the photodiode array accumulates the signal. The greater the IT is, the better the S/N ratio. However, the photodiode array starts saturating if the signal exceeds the dynamic range of the detector.10 • Coadds: 16. This value represents the number of individual frames collected from the detector and averaged at each step. • Scans: 1. Complete scans that will be averaged to produce the final spectrum. The scan number is normally left at one, since it is much more time-efficient to increase the number of coadds to improve the S/N ratio. • Step: 10 nm. This is the spectral resolution. • Wavelength range: 1,100–1,600 nm, 51 wavelength values. • Light intensity: 75%. Voltage: 5.501 V. Current: 7.500 A. A high-diffuse reflectance target (white ceramic plate; Malvern Instruments, MD, USA) was used as white reference in all experiments. Its surface is very spatially homogeneous, it produces no spectral features and exhibits negligible thermal expansion. The reflectance level is about 90%. All the experiments were performed at a constant temperature of 22°C ± 1°C achieved using an air-conditioning device and constant measurement by external thermometer. The time for each image scan was ­approximately two minutes. 2.2.2 Comparison of Dark Reference Materials The features of three different dark materials were analyzed. The first was the stainless steel support recommended as a light specular reflectance target by instrument manufacturers and suggested by other authors.11 The second was the use of a cap covering the lens as recommended by Gowen et al.,12 and the third was a piece of silicon carbide sandpaper. This last option was studied since some of this material is used as a sample holder13 and, at the same time, it has very high absorbance values similar to the stainless steel support. Furthermore, the sandpaper material is cheap and easy to reach. All the three candidate materials were analyzed in “raw mode,” that is, a data cube for each material was obtained and data were treated in terms of intensity values, without considering dark and white references. The analysis was performed under dark ambient illumination. The three dark candidates (cap, sandpaper and stainless support) were analyzed. No pre-processing treatments were applied, and all the pixels (81,920) of each image were considered. Pixel saturation level checks were followed up by visual examination of spectra for the three materials. Finally, the Mahalanobis distance (MD), the distance of each pixel spectrum from the center of the population, was also calculated.

24

Cecilia Riccioli, et al.

2.2.3 Scanning Frequency for Dark and White References Using the lens cap as dark reference, the following reference scanning frequency options were tested:

1. Every sample: dark and white references were acquired before every image acquisition, that is, every 10 min. 2. Every five samples (30 min): dark and white references were acquired for every five image acquisitions. 3. Every ten samples (~1 h): dark and white references were acquired for every ten image acquisitions. The material used as a sample was a non-perishable acetal standard material (Unity Scientific, USA), hereafter termed as “standard material.” Ten scans of the standard material were performed for each reference analysis frequency, as follows:

1. Dark and white reference (R) - standard material (SM) - R - SM - R - SM - R - SM R - SM - R - SM - R - SM - R - SM - R - SM - R - SM. 2. R - SM - SM - SM - SM - SM - R - SM - SM - SM - SM - SM. 3. R SM - SM - SM - SM - SM - SM - SM - SM - SM - SM. All ten images for each reference-frequency mode were assembled using a concatenate function resulting in one image for every mode. A 4 × 4 binning was applied to the images so that an area of 16 adjacent pixels was averaged, thus improving spectral S/N ratio at the expense of spatial resolution. Three different statistics were performed in order to determine the optimum reference frequency. The first was the standard deviation, calculated at 1,520 nm, for the ten samples comprising each reference-frequency mode, in order to detect the concatenated image with the lowest variability. This wavelength was selected because the detector was calibrated at 1,520 nm by the instrument manufacturers using a rare earth oxide standard. The second was the root mean square deviation (RMS), calculated using the formula indicated in Section 2.2.6, for all spectra obtained from the ten scans per reference-frequency mode, in order to obtain information on the variability of concatenated images taking into account the contribution of all 51 wavelengths. Finally, the MD was calculated for all pixels belonging to each scan, in order to determine the distance of the concatenated image from the center of the population. The scans of all the images were taken in dark ambient illumination, that is, without any external light source with the exception of the computer screen. 2.2.4 Ambient Light Since ambient light is one of the factors that affect image quality, different ambient light levels were examined. Tests were performed using the same material used for the analysis of dark and white references. Ten scans of the standard material were performed, using dark and white references before every scan and covering the lens during dark image acquisition. Three illumination levels were studied: • Light: room lights (halogen lamps) switched on. • Darkness: room lights switched off.

Optimization of Hyperspectral Image Cube Acquisition

25

• Mixed conditions: simulated uncontrolled conditions, i.e. light switched on and off while the instrument was analyzing the standard material. 2.2.5 Sample Presentation Four possible sample holders were tested: a stainless steel support, a porcelain plate, a piece of sandpaper and a white Teflon® plate. A sample of MBM was analyzed using all four sample holders under dark ambient illumination. The four sample images thus obtained were visually inspected in order to identify differences between sample and non-sample zones. Intensity histograms for each sample were examined to determine the maximum distance between pixels belonging to the sample and those belonging to the sample holder. Dark and white references were taken every time a sample was analyzed; dark reference was taken by covering the lens.

2.2.6 Mathematical Data Treatment All analyses were carried out using ISYS 4.0 (Malvern, Inc., Olney, MD, USA) and MATLAB® R2017A (The Mathworks, MA, USA). Data analysis was based on several indices, which were used to quantify the spectral repeatability of hyperspectral image cubes. In all experiments except the dark material reference experiment, dark and white data cubes were used to calculate the absorbance value (log 1/R) for each pixel at each ­wavelength λi according to the following equation:

 R − Rdark   1 log   = log  white  , (2.1)  R  Rsample − Rdark 

where Rwhite = Intensity of the remitted radiation for the instrument white standard. Rdark = Intensity of the remitted radiation for the instrument dark standard. R sample = Intensity of the remitted radiation for the sample cell. During the course of images analysis, it is possible to generate infinities by dividing a scalar by 0 and undefined numbers by dividing 0 by 0. These values were removed through a pre-processing treatment included in the ISYS 4.0 software. In order to determine the repeatability of the scans obtained with the three ­reference-frequency modes studied, three computations were used: the standard deviation, the RMS to test the spectral matching of the images and the MD from pixels of one image to the rest of the images belonging to the same group. The RMS parameter is the averaged root mean square of differences corrected for the bias between spectra obtained in two scans of an image, at N wavelengths: i=N

∑ (Y − Y )

2

ij



RMS j = 106

i =1

N

i

, (2.2)

26

Cecilia Riccioli, et al.

where RMSj = root mean square deviation for spectrum j. N = number of wavelengths (λi) in each spectrum j. Yij = absorbance for spectrum j at wavelength λi. Yi = mean absorbance at wavelength λi averaged over J tests. The MD is defined according to elliptical contours spreading out from the group centers. The mathematics and applications of MD has been described by Mark.14 The distance D, from a point X to the center of a group i, is described by the matrix equation: D2 = (X − X i )′ M(X − X i ), (2.3)



where D is the distance, X is the multidimensional vector describing the location of point x, X i is the multidimensional vector describing the location of the group mean of the ith group, (X − X i ) is the transpose of the vector (X − X i ), and M is a matrix determining the distance measures of the multidimensional space involved.

2.3  RESULTS AND DISCUSSION 2.3.1 Dark Reference Material Selection of the best dark reference was based on the assumption that the dark material should have a very low pixel intensity level, a low reflectance value and stable reflectance values for each pixel. The quality of three candidates was tested. Results are shown in Figures 2.1 and 2.2, and Table 2.1. Figure 2.1 represents an oscilloscope-like graph (trace) that shows the distribution of pixel intensity across the image. The Y-axis represents the dynamic range of the camera A/D converter (scale: intensity values from 0 to 4,095); the X-axis is the number of pixels. In order to maximize the use of the available camera dynamic range, the imaging parameters should be set in such a way that a dark image produces an output signal close to zero. It is also important that the output image be uniform. This feature can be monitored by examining the thickness of the oscilloscope trace, which should ideally

FIGURE 2.1  Saturation levels (from 0 to 212 levels) for three different materials.

Optimization of Hyperspectral Image Cube Acquisition

27

FIGURE 2.2  Intensity values for pixels derived from sandpaper (black triangles), stain-

less steel (green circles) and cap (red line). TABLE 2.1  Mean of MD Values Calculated for Each Candidate as Dark Reference Material MD

Stainless Steel

Sandpaper

Cap

0.505

1.100

0.475

be narrow and horizontally flat as specified in the camera user manual. The lens cap appears to be the best candidate: as shown in Figure 2.1, the pixels lie in a flat line in the three procedures studied, however, the lowest saturation level was achieved using the cap, which allowed a higher number of intensity levels. Intensity values for pixels belonging to sandpaper, stainless steel and cap are shown in Figure 2.2. Spectra from pixels derived from sandpaper and stainless steel materials displayed similar features, while cap spectra had lower intensity values, which also varied little along the X-axis (wavelengths). MD values calculated between pixel spectra were used as a similarity criterion for pixels within an image. This additional test sought to verify the homogeneity of the dark image, since in the ideal dark reference all pixels must have the same intensity value at a specific wavelength, that is, the homogeneity must exist in the spatial domain as well as in the spectral domain. Results are tabulated in Table 2.1. Similar MD values were recorded for stainless steel and the lens cap, indicating that both were highly homogeneous. However, lower values were observed using the cap, probably because it enabled non-uniform light effects to be avoided. MD values for sandpaper were high, indicating bigger variability, probably due to the irregular surface; this material is thus less suitable, despite its lower cost and the very low reflectance background.15

28

Cecilia Riccioli, et al.

2.3.2 Scanning Frequency of Dark and White References As previously indicated, acquisition of a good white reference is imperative for proper correction of sample data. The white data cube is collected from a high-reflectance standard and must be acquired using the same instrumental parameters and conditions as those used for the sample data.16 The intensity of a raw signal from the sample is influenced by the system components. The output of the light source is not completely uniform over the full spectra range and neither is the response of the optics because of optical aberrations. The sensitivity of the detector will also vary with wavelength. An additional non-samplerelated contribution to the overall system response is the dark response. There are no clear recommendations about how often reference spectra should be acquired in order to maintain stable results. Thermal drift or ambient temperature changes may cause an improvement in the frequency with which the reference should be taken. As indicated in Section 2.2, three options were evaluated analyzing the standard material and changing the frequency of the reference analysis (every sample, every five samples and every ten samples). After binning treatment, all ten images for each reference frequency mode were concatenated as shown in Figure 2.3. Although the material used was a uniform, sealed white reference, surface irregularities were quite evident. Absorbance values varied depending on pixel position: values were highest at the edges and declined toward the center of the images. These results are clearly linked to the light source, which—despite the use of polarizing filters—caused reflection.

FIGURE 2.3  Image result (log 1/R) for concatenation of the images of ten standardmaterial scans at 1,520 nm.

Optimization of Hyperspectral Image Cube Acquisition

29

As stated by Geladi et al.,17 the ideal illumination system would have several properties as homogeneous illumination, intense polychromatic light, polarized light, deep transmission through samples, and controlled reflection from deep into the sample. Furthermore, biological samples such as MBM may be sensitive to heating caused by the continuous illumination from source lamps. ElMasry et al.18 tried to solve the problem by a cubic tent made from white nylon fabric to diffuse the light and provide a uniform lighting condition. In this work, the inhomogeneous illumination was considered constant, since i­ llumination conditions were maintained unvarying and no shading corrections were applied. Results for the four indices used to estimate spectral repeatability for each of the three reference frequencies studied are shown in Table 2.2. Although RMS values were similar for all the three options, the lowest values were obtained when scanning reference material before “every sample.” To give an idea of typical RMS values for hyperspectral images, it should be noted that the RMS value for pixels from a single scan of the standard reference material is about 2.0 × 104 μLog 1/R. This would suggest that the higher values recorded here are due to properties inherent in the material itself rather than to differences between different scans of that material. Similar behavior was observed when calculating the MD from each pixel of a concatenated image to the center of the population: the “every sample” mode was associated with the shortest distances between pixels, although differences were minor. The MD was also calculated from each image of a single scan to the center of the concatenated-image population, in order to ascertain whether the stability of instrument image acquisition was influenced by reference frequency. The results are shown in Figure 2.4. TABLE 2.2  RMS, MD, and Standard Deviation Values for the Three Reference-Frequency Modes Every Sample RMS μLog 1/R MD Standard deviation at 1,520 nm Standard deviation at 1,200 nm

Every Five Samples

Every Ten Samples

2.298 × 104 0.865 0.025 0.029

2.253 × 104 0.816 0.025 0.028

2.305 × 104 0.820 0.025 0.029

0,95

Mahalanobis distance

0,9

0,85

MD every sample MD every 5 samples MD every 10 samples

0,8

0,75

0,7 1

2

3

4

5

6

7

8

9

10

Images

FIGURE 2.4  MD from each scan to the center of the population for every referencefrequency mode.

30

Cecilia Riccioli, et al.

Differences between distances for each scan were less marked within the “every sample” reference frequency, with mean absolute deviation values of 0.014 compared with 0.025 for the “every five samples” mode and 0.032 for the “every ten samples” mode. Standard deviation among all pixels at 1,520 nm was the same for all frequency modes. The lack of information provided by this test prompted calculation of the standard deviation for a different wavelength, to check the strength of the result. The 1,200-nm band was chosen since this is a crucial wavelength for discriminating between different animal species in MBM.19 Values for the 1,200-nm band were very similar, confirming that differences in standard deviation were very minor. Even so, the lowest value was found for the “every sample” frequency, thus confirming the results for other statistics. 2.3.3 Ambient Light To detect potential variations in spectra due to external light, a test similar to that performed in the reference-frequency analysis was carried out. RMS and MD statistics were used to detect and measure variability between scans of the same material used in the previous test. First, the RMS value was calculated for the group of 10 samples scanned under the same light conditions. All samples were analyzed acquiring reference-material scan before every sample, and maintaining an external temperature of 22°C ± 1°C. The MD from standard-material scans to the center of the population of the concatenated image was also calculated. Results are shown in Table 2.3. The RMS values indicated that the lowest variability values were obtained when analyzing samples in dark ambient conditions. Figure 2.5 shows the MD from every TABLE 2.3  RMS and MD for Three Ambient Light Modes RMS μLog 1/R MD

Light

Darkness

Mixed

2.081 × 104 0.741

1.772 × 104 0.767

2.559 × 104 1.190

1,6

Mahalanobis distance

1,4 1,2 LIGHT

1

DARKNESS MIX

0,8 0,6 0,4 1

2

3

4

5

6

7

8

9

10

Images

FIGURE 2.5  Mean of MD from every image analyzed in given light conditions to the center of the population.

Optimization of Hyperspectral Image Cube Acquisition

31

image to the center of the population of the ten images belonging to each group: “light,” “darkness,” and “mixed.” MD values showed that the “darkness” mode was the most stable, with mean absolute deviation values of 0.028, compared with 0.103 for the light mode and 0.110 for the mixed mode. Interestingly, uncontrolled external lighting increased inter-image variability, prompting a considerable decline in the stability of analysis. This confirmed the link between spectrum quality and external lighting, as well as confirming that the absence of room light is to be preferred during HCI analysis. 2.3.4 Sample Presentation Sample presentation is another key factor for obtaining a high-quality hyperspectral image. The sample holder should be chosen with a view to maximize the differences between sample and non-sample zones and eliminate edge and shadow effects. The advantage of placing the sample on a non-diffuse reflecting metal surface, such as, a stainless steel plate, is that it will appear as totally absorbing in corrected spectral images. This will provide a high-contrast background for the non-sample areas in the image. This assumption nevertheless depends on the characteristics of the sample; sometimes, moreover, the use of other materials more readily available in the laboratory may be preferred. The present experiment analyzed a real MBM sample supported on four different sample holders: porcelain, sandpaper, stainless steel support and Teflon®. Further research is shortly to be carried out to determine the importance of edge definition for spectral and shape information. The incorporation of spatial information improves pixel-based spectral classifications, as shown in remote sensing studies, 20 and in medicine-related research;21 it may also be a key aspect of MBM analysis. Images from all the sample holders used for the MBM sample are shown in Figure 2.6. Spectra and pixel intensity histograms were used to best option in terms of minimizing shadow effects and edgepixel misclassification. The intensity histograms of HCI images offer a concise representation of the global intensity characteristics of an image and facilitate the determination of global features. 22 The use of the stainless steel support and the silicon carbide sandpaper gave rise to two peaks, one corresponding to the sample and the other corresponding to the sample holder. This means that differences between pixels belonging to the sample holder and those belonging to the sample are perfectly distinguishable. The greatest separation between the two peaks was achieved with the stainless steel sample holder, confirming the suitability of this material, as reported by Geladi et al.17 Analysis of spectrum plots indicated a clear separation of sample holder vs. sample pixels using both sandpaper and a stainless steel support. With the latter, however, edge pixels were distinguishable both from sample pixels and sample holder pixels, making this the most suitable material for correct edge definition. In practical terms, given the powdery texture of MBM samples, stainless steel also has the advantage of being easier to clean and reuse.

2.4 CONCLUSIONS In general, the results of the study revealed that the quality of hyperspectral analysis can be substantially improved by optimizing certain operating conditions that can affect the

32

Cecilia Riccioli, et al.

porcelanalogxC.spf (190,224) (190,224 ) 0.6 0.55 0.5

Absorbance

0.45 0.4 0.35 0.3 0.25 0.2

Porcelain

0.15 0.1 1100

1150

1200

1250

1300

1350

1400

1450

1500

1550

1600

Wavelength (nm)

Sandpaper

lijalogxC.spf (130,14) 1.2

Absorbance

1

0.8

0.6

0.4

0.2

1100

1150

1200

1250

1300

1350

1400

1450

1500

1550

1600

Wavelength (nm)

espejologxC.spf (210,232)

1.2

Absorbance

1

0.8

0.6

0.4

Stainless Steel

0.2 1100

1150

1200

1250

1300

1350

1400

1450

1500

1550

1600

Wavelength (nm) Wavelength (nm)

teflonlogxC.spf (104,14)

Teflon

0.45

0.4

Absorbance

0.35

0.3

0.25

0.2

0.15

0.1

0.05 1100

1150

1200

1250

1300

1350

1400

1450

1500

1550

1600

Wavelength (nm)

FIGURE 2.6  From left: absorbance images of sample and sample holder (band at

1,520 nm). Spectra corresponding to the sample (line), to the sample holder (dashed), and to the edges (dotted). Intensity histograms representing absorbance values of the pixels ­corresponding to the images shown. images acquired. In particular, tests showed that a cap covering the lens was the most suitable dark reference method. Tests also suggested that frequent scanning of white and dark reference materials improves image quality. However, the results obtained when scanning reference material for every sample scan were very similar to those achieved when scanning the reference for every ten sample scans; for operational purposes, therefore, the latter option is recommended. Tests to optimize external lighting conditions showed that analysis in dark ambient conditions gave more stability to the images acquired. Finally, stainless steel was found to be the most suitable material for the sample holder. These optimized conditions should thus be used for NIR HCI analysis of large sets of MBM samples.

REFERENCES 1. Qin, J., Kim, M.S., Chao, K., Chan, D.E., Delwiche, S.R., Cho, B.K. (2017). Linescan hyperspectral imaging techniques for food safety and quality applications. Applied Sciences, 7, p. 125.

Optimization of Hyperspectral Image Cube Acquisition

33

2. Riccioli, C., Garrido Varo, A., Perez Marin, D. (2018). Identifying animal species in NIR hyperspectral images of processed animal proteins (PAPs): Comparison of multivariate techniques. Chemometrics and Intelligent Laboratory Systems, 172, pp. 139–149. 3. Gowen, A., Gaston, E., Burger, J. (2014). Hyperspectral imaging. In: O’Donnell, C., Fagan, C., Cullen, P. (eds.) Process Analytical Technology for the Food Industry. Food Engineering Series. Springer, New York. 4. Nansen, C., Geremias, L.D., Xue, Y., Huang, F., Parra, J.R. (2013). Agricultural case studies of classification accuracy, spectral resolution, and model over-fitting. Applied Spectroscopy, 67, pp. 1332–1338. 5. Anderson, G.L., Peleg, K. (2007). Quantification and reduction of erroneous differences between images in remote sensing. Environmental and Ecological Statistics, 14 (2), pp. 113–127. 6. Elmasry, G., Barbin, D.F., Sun, D.-W., Allen, P. (2012). Meat quality evaluation by hyperspectral imaging technique: An overview. Critical Review in Food Science and Nutrition, 52 (8), pp. 689–711. 7. Kamruzzaman, M., Elmasry, G., Sun, D.-W., Allen, P. (2012). Non-destructive prediction and visualization of chemical composition in lamb meat using NIR hyperspectral imaging and multivariate regression. Innovative Food Science & Emerging Technologies, 16, pp. 218–226. 8. Rogalski, A., Chrzanowski, K. (2014). Infrared devices and techniques (revision). Metrology and Measurement Systems, XXI (4), pp. 565–618. 9. Malvern Instrument. MatrixNIR Operations Manual (2005). Onley, MD. 10. Dixit, Y., Casado-Gavalda, M.P., Cama-Moncunill, R., Cama-Moncunill, X., Markiewicz-Keszycka, M., Cullen, P.J., Sullivan, C. (2017). Developments and challenges in online NIR spectroscopy for meat processing. Comprehensive Reviews in Food Science and Food Safety, 16, pp. 1172–1187. 11. Windham, W.R., Smith, D.P., Berrang, M.E., Lawrence, K.C., Feldner, P.W. (2005). Effectiveness of hyperspectral imaging system for detecting cecal contaminated broiler carcasses. International Journal of Poultry Science, 4 (9), pp. 657–662. 12. Gowen, A.A., O’Donnell, C.P., Taghizadeh, M., Gaston, E., O’Gorman, A., Cullen, P.J., Frias, J.M., Esquerre, C., Downey, G. (2008). Hyperspectral imaging for the investigation of quality deterioration in sliced mushrooms (Agaricus bisporus) during storage. Sensing and Instrumentation for Food Quality and Safety, 2, pp. 133–143. 13. Williams, P., Geladi, P., Fox, G., Manley, M. (2009). Maize kernel hardness classification by near infrared (NIR) hyperspectral imaging and multivariate data analysis. Analytica Chimica Acta, 653 (2), pp. 121–130. 14. Mark, H. (2008). In Donald, A.B., Emil, W.C. (eds) Handbook of Near-Infrared Analysis, Third Edition. CRC Press, Boca Raton, FL, pp. 307–331. 15. Burger, J., Geladi, P. (2006). Hyperspectral NIR image regression part II: Dataset preprocessing diagnostics. Journal of Chemometrics, 20, pp. 106–119. 16. Nouri, D., Lucas, Y., Treuillet, S. (2013). Calibration and test of a hyperspectral imaging prototype for intra-operative surgical assistance. Proc. SPIE 8676, Medical Imaging 2013: Digital Pathology, Feb 2013, Lake Buena Vista (Orlando Area), Florida, United States. 17. Geladi, P., Burger, P., Lestander, T. (2004). Hyperspectral imaging: Calibration problems and solutions. Chemometrics and Intelligent Laboratory Systems, 72 (2), pp. 209–217.

34

Cecilia Riccioli, et al.

18. ElMasry, G., Wang, N., Vigneault, C., Qiao, J., ElSayed, A. (2008). Early detection of apple bruises on different background colors using hyperspectral imaging. LWT Food Science and Technology, 41 (2), pp. 337–345. 19. Riccioli, C., Perez Marin, D., Guerrero Ginel, J.E., Fearn, T., Garrido Varo, A., (2012). Detection and quantification of ruminant meal in processed animal proteins: A comparative study of near infrared spectroscopy and near infrared chemical imaging. Journal of Near Infrared Spectroscopy, 20, pp. 623–633. 20. Blaschke, T. (2010). Object based image analysis for remote sensing. ISPRS Journal of Photogrammetry and Remote Sensing, 65, pp. 2–16. 21. Lu, G., Fei, B. (2014). Medical hyperspectral imaging: A review. Journal of Biomedical Optics, 19 (1), pp. 1–23. 22. Seul, M., O’Gorman, L., Sammon, M.J. (2000). Practical Algorithms for Image Analysis: Description, Examples, and Code. Cambridge University Press, New York, NY, pp. 21–37.

Chapter

3

Image Segmentation Sylvio Barbon Jr. and Ana Paula Ayub da Costa Barbon State University of Londrina

N.A. Valous and D.F. Barbin University of Campinas

CONTENTS 3.1 Introduction 35 3.2 Pre-processing 36 3.3 Segmentation 37 3.3.1 S upervised Segmentation 37 3.3.1.1 H ierarchical Segmentation 38 3.3.1.2 Bayesian Framework Segmentation 38 3.3.1.3 Evolutionary Cellular Automata-Based Segmentation 39 3.3.1.4 M inimum Spanning Forest 39 3.3.1.5 k-Nearest Neighbor Segmentation 39 3.3.2 U  nsupervised Segmentation 39 3.3.2.1 End Member Threshold Selection 40 3.3.2.2 Watershed 40 3.3.2.3 Spatial–Spectral Graph 41 3.3.2.4 k-Means Clustering 41 3.3.2.5 Superpixel Segmentation 41 3.4 Performance Metrics 41 3.5 Conclusion 42 References 42

3.1 INTRODUCTION Traditionally, food quality monitoring is focused on consumer safety and nutritional aspects through physicochemical methods or human involvement. Unfortunately, these techniques are subjective, destructive, laborious, and time-consuming, often requiring chemical reagents and producing residues. Indeed, fast and precise analytical methods are essential to ensure food quality and safety. In recent years, several imaging technologies have been applied for food quality ­assessment. However, standard image analysis in Red–Green–Blue (RGB) color space does not provide information about the composition of foods, since it can analyze only surface characteristics in the visible spectrum. A common alternative is based on near-infrared spectroscopy (NIRS), which provides spectral information in the near infrared-range across the

35

36

Sylvio Barbon Jr. et al.

sample surface. Nevertheless, the trade-off of spatial and spectral information arises, as NIRS is not able to report the spatial information of a pattern identified from a wavelength. In this context, hyperspectral imaging (HSI) emerges as an alternative solution capable of combining these two concepts: spatial and spectral information. HSI tackles the significant drawbacks for food quality evaluation from the perspective of shape, size, and position of the observed phenomena. Furthermore, HSI has been proved to be a powerful tool for evaluating the quality of foodstuffs, because it is robust, fast, and nondestructive (Ravikanth et al., 2017). Several research papers have already successfully reported the use of HSI for food quality evaluation by combining the advantages from computer vision and spectroscopy. For example, this methodology was applied to evaluate the quality of pork, beef, chicken, salmon, lamb, ham, fruits, and vegetables. A hyperspectral image is represented as a hypercube, that is, a block of threedimensional data: two dimensions are spatial (rows and columns) and one dimension is spectral (wavelength). A hypercube is formed by several continuous wavebands for each spatial position of the sample (pixel). In this way, a given pixel could be represented by a range of wavelengths providing the spectral signature. Generally, there are three approaches for acquiring hypercubes: point scan, line scan, and area scan methods (Raychaudhuri, 2016; ElMasry & Nakauchi, 2016). These acquisition methods are discussed in Chapter 2. Typically, for a hyperspectral image to be acquired, an objective lens, camera, spectrograph, acquisition system, lighting sources, translation stage, and computer are required. This system forms the hyperspectral image from light that is reflected from the sample through the lens, where it is separated into components according to the wavelength by the diffraction grating contained in the spectrograph. Finally, the formed image is saved in the computer for further processing. The pre-processing of the spectra is an important step when classifying or predicting some property from a sample. Usually, this stage includes thresholding, filtering, contrast enhancement, and masking to remove irrelevant information from the hypercube. These processes improve the linear relationship and minimize the effects of variability between sample concentration and spectral signals. Some defects presented as noise or blurring could be treated or even eliminated. Furthermore, it is possible to reduce the sample size, thus decreasing computation time and memory requirements. After hypercube preprocessing, the next step is image segmentation. This stage is one of the most challenging steps of the workflow, where the relevant objects are extracted (segmented) (Bong & Rajeswari, 2011). Currently, there are several techniques and algorithms to perform image segmentation, thus requiring experience to select the solution that best fits the problem under specific requirements, such as time and computational complexity. In many cases, it is necessary to make use of computational intelligence to find and segment the region of interest (ROI). Thus, the objective of this chapter is to present in a clear and precise way the concept and importance of the segmentation stage within the hyperspectral image processing workflow. Mainly, the focus is going to be on elucidating the steps that can be applied to hyperspectral images that are related to food quality evaluation.

3.2 PRE-PROCESSING The data generated by the HSI system are large and require solutions to decrease dimensionality without losing the original information. In many cases, noise is also generated

Image Segmentation

37

from the hyperspectral equipment and these noisy images should be evaluated, corrected, or eliminated for optimal system performance. Several challenges are posed by HSI, particularly related to noise treatment, dead pixels correction, light scattering, and image compression. These can be mitigated by pre-processing techniques. The first three degrade image quality through the absence of real information since the acquired bad pixels are related to defects in the former process. The lack of real information leads to costly segmentation algorithms, and perhaps to inaccurate segmentation. On the other hand, image compression techniques contribute to rapid processing, making HSI applications competitive (time wise) in comparison to other non-destructive approaches. In turn, the compression algorithm requires caution, as it could affect the image quality, and its application could be computationally costly. In this context, pre-processing methods for hyperspectral images such as smoothing, multiplicative scatter correction (MSC), standard normal variate (SNV), Fourier transform (FT), and Wavelet transform (WT) are utilized to calibrate the original spectra and improve the next steps, such as segmentation.

3.3 SEGMENTATION Although the greater dimensionality of HSI compared with multispectral images improves data information content considerably, it does introduce new challenges to image analysis techniques that have been specifically designed for multispectral data (Ghamisi et al., 2017). The segmentation process consists of partitioning an image into ROIs. In g­ eneral, objects need to be separated from their background to extract the necessary information. For this process, there are several techniques and algorithms that are ­implemented either manually or automatically. However, the difference from traditional image segmentation that is focused on a two- or three-dimensional space is that the methods for segmenting hyperspectral images must deal with many spectral channels. Furthermore, this high data dimensionality poses challenges to commonly used methods designed for grayscale, color, or multispectral image processing. Therefore, to get valuable information from the whole hypercube, different methods should be considered. It is important to highlight that some pixel-wise techniques process each pixel independently, focusing on the spectral dimension without information about spatial structure. The hyperspectral image segmentation methods are typically categorized as supervised and unsupervised. Supervised methods are dependent on known pixel attributes for segmenting a region. On the other hand, unsupervised methods do not require a priori information. Some novel categories such as semi-supervised (Li et al., 2010) are out of the scope and not covered in this chapter. 3.3.1 Supervised Segmentation In supervised segmentation, also called pixel-based segmentation, the user plays an important role since a seed is required related to foreground and background regions. A seed is a pixel that represents information for each ROI toward identifying its similar pixel neighbors. Thus, these methods use spatial and intensity information from the seed as a manual reference to segment a region. Region growing and machine learning are widely applied techniques, since the user reference or training sample is used to denote a

38

Sylvio Barbon Jr. et al.

TABLE 3.1  Supervised Segmentation Methods Method

Technique

Description/Comments

Hierarchical Region growing Spatial/spectral segmentation based on segmentation structural morphological operations Bayesian segmentation Machine learning Spatial and spectral information framework modeled with limited training samples having high dimensionality Evolved cellular Machine learning Application of cellular automata to automata reach a final state classified by a machine learning approach Minimum spanning Region growing Efficient implementation of the spatial forest and the spectral segmentation k-NN Machine learning Fast computation and good generalization performance

Reference Plaza et al. (2009) Li, BioucasDias and Plaza (2012) Priego et al. (2013) Pike et al. (2014) Ivorra et al. (2016)

pattern in the image or to induce a model, respectively. Table 3.1 shows a compilation of recent research work based on this approach. 3.3.1.1 Hierarchical Segmentation Image segmentation when based on mathematical morphology deals with fundamental operations on two sets: one set is used to satisfy the other set having a finely determined shape and size, known as structuring element (SE) (Serra, 1982). The structural matching of elements uses the mathematical morphological operations of erosion and dilation, which can be graphically illustrated by viewing the image data as an imaginary topographic relief where higher elevations are represented by brighter tones. Thus, the SE defines the new (dilated or eroded) scene based on its spatial properties such as height or width as a result of sliding the SE over the topographical relief. The application of morphological operators to hyperspectral images, which are based on multilayered data with hundreds of spectral channels, is not straightforward. A straightforward approach starts with grayscale morphology techniques for each channel separately (marginal approach). It is often unacceptable, because it may result in new spectral constituents that are not present in the original hyperspectral image as a result of processing the channels separately (Comer and Delp, 1999). An alternative (and perhaps more appropriate) way to approach the problem of multichannel morphology is to treat the data at each pixel as a vector (Soille, 2003). It is important to define an appropriate combination of vectors in vector space, which could be either spectral-domain partitioning or spatial-domain partitioning (Plaza et al., 2009). 3.3.1.2 Bayesian Framework Segmentation Proposed by Li, Bioucas-Dias and Plaza (2012), this segmentation technique is based on a Bayesian classifier. The spectral classification of a ROI or background uses multinomial logistic regression coupled with a subspace projection method. Subspace projection contributes to characterize noise and highly mixed pixels in an optimal way. The spatial information is modeled by Markov random fields using the maximum posteriori probability approach to image segmentation that is computed via an efficient min-cut based integer optimization technique. The identification of a pixel as a part of a ROI or background is performed using the Bayesian framework. The proposed Bayesian framework achieves good partitional capability when dealing with limited training samples and the

Image Segmentation

39

high dimensionality of the input data. The main advantage of this method is the combination of the spatial-contextual information capable of providing good performance in both the spectral and the spatial domain. 3.3.1.3 Evolutionary Cellular Automata-Based Segmentation Cellular automata, proposed by Von Neumann and Aw (1966), are a spatially extended system composed of a number of cells fashioned along a portion of space, which communicate only with their neighbors. Each cell (i.e., pixel) has a state that depends on its neighbors and a previous state. In this way, an iterative procedure is carried out until reaching the final target. The final state of each pixel reveals the ROI. More precisely, in this method, a machine learning algorithm is required to induce a model based on the final state in order to provide labels for these regions and thus classifying them; the most used algorithm for this purpose is the support vector machine (SVM). However, the most challenging task when applying the method is defining the rules to be used by the automata. Several evolutionary techniques have been applied to tackle this challenge. In Priego et al. (2013), evolutionary methods were applied to produce the CA rule sets that result in the best possible segmentation. The projection was avoided due to dimensionality ­reduction and possible loss of spectral information. 3.3.1.4 Minimum Spanning Forest This method uses dissimilarity measures between a pixel and its eight nearest neighbors. Many dissimilarity measures can be used for this purpose. Pike et al. (2014) used the vector norm, spectral angle mapper, and spectral information divergence as measures. A minimum spanning forest is constructed from a connected graph that consists of a set of separated unconnected trees, where the vertex represents pixels within the image and the edges represent the associated dissimilarity measure between these pixels. The forest obtained is a spanning forest of unconnected trees. 3.3.1.5  k-Nearest Neighbor Segmentation The k-nearest neighbor (k-NN) is a supervised segmentation method with fast computation times and good generalization performance. The “k” represents the number of neighbors selected considering the most similar pixels in the spatial and spectral dimensions. This similarity could be computed using different types of distances, e.g. Euclidean distance. Hence, the complete hypercube has pixels assigned to corresponding clusters according to the k-NN classification results. The k-NN predicts the classification label of a given query feature vector based on the k-closest training vectors. The majority vote of its k classifies this query pixel. This means that a region of pixels is classified into the training class that is most common among its k-NNs. The k number of neighbors is commonly an odd number. In the study by Ivorra et al. (2016), k-NN segmentation was explored to identify salmon tissues in a project of shelf-life prediction of expired vacuumpacked chilled smoked salmon. 3.3.2 Unsupervised Segmentation In contrast to supervised segmentation, the unsupervised methods do not require human intervention or explicit patterns to be tracked. Thereby, the use of unsupervised methods in HSI has been increasing and the techniques employed to produce an accurate segmentation follow diverse concepts, as summarized in Table 3.2. In brief, unsupervised

40

Sylvio Barbon Jr. et al.

TABLE 3.2  Unsupervised Segmentation Methods Method End member threshold selection Watershed

Technique

Description/Comments

Reference

Region-based Nonorthogonal pixels are linearly related focusing on defining a region

Graña et al. (2009)

Region-based Extension of traditional two-dimensional image watershed based on mathematical morphology operations Spatial–spectral Graph-based Straightforward graph computation graphs incorporating both spatial and spectral information k-means Clustering Identifies objects based on k groups of clustering similar pixels in the image Superpixel Clustering These techniques exploit spatial segmentation dependencies in the image

Tarabalka et al. (2010); Li, Chen and Huang (2018) Gillis and Bowles (2012) Piqueras et al. (2015) Fana et al. (2017)

segmentation is related to a pixel classifier built by an unsupervised procedure, often some kind of clustering. Graña et al. (2009) highlighted that this kind of approach suffers from the identification ambiguity problem: the region is detected; however, it is not possible to give it meaning, that is a mapping from a pixel region to an object. 3.3.2.1 End Member Threshold Selection This method is based on the end members, which involves the search for nonorthogonal planes that surround the data defining a minimum volume simplex. The method is computationally costly and requires the parametrization of the number of end members. Image pixels are modeled as the linear mixture of end members and some shade end members could be used to map information as variation in illumination. One or more non-shade end members could denote different materials within the image. End members can be computed from image pixels and the meaning of a region depends on the interpretation of a domain specialist. 3.3.2.2 Watershed Watershed is a widely used image segmentation algorithm, mainly implemented in computer vision images. Its HSI version integrates the spatial information from the hyperspectral image and provides classification maps with more homogeneous regions throughout the spectral range, compared with pixel-wise classification and previously proposed spectral–spatial classification techniques. The watershed is an interesting method, but morphological operators in multispectral images are not straightforward, since there is no natural way for ordering multivariate pixels (Tarabalka et al., 2010). Watershed transformation is commonly applied to the gradient, which is related to a scalar function to form the complete structure. In some cases, watershed segmentation results in severe over-segmentation, where every single local minimum of the gradient leads to one region. This type of problem requires post-processing through filtering or merging neighbor techniques. Li, Chen and Huang (2018) proposed an improved watershed segmentation method to deal with the segmentation of a damaged region. Among some modifications, the morphological gradient reconstruction and marker extraction were capable to aid the method to overcome traditional hyperspectral segmentation methods.

Image Segmentation

41

3.3.2.3 Spatial–Spectral Graph A graph is a structured collection of objects (vertex) and connections between them (edges) are assigned with numerical values (weights). This structure could be fashioned as an adjacency matrix where the pixels are the vertices and the weighted edges are the connections computed with spatial and spectral distances. The advantages of this technique are related to the graph structure that naturally incorporates both spatial and spectral information present in HSI. Indeed, the adjacency matrix is highly sparse, thus it is possible to apply the method to larger images than previous techniques, using sparse linear algebra routines. Gillis and Bowles (2012) highlighted the spatial–spectral graph as an important method capable of handling high-dimensional images without compromising performance or requiring image compression pre-processing. 3.3.2.4  k-Means Clustering The k-means algorithm is a widely applied method in unsupervised learning. The algorithm is less computationally intensive for segmentation purposes when compared with hierarchical approaches. Typically, it is applied on the whole dataset using a given k value to define the number of groups (or objects) that need to be segmented. The outcome is a solution composed of k clusters with the lowest value for the sum of squares for the within-cluster pixel distances to the centroids (a representative pixel signature of a cluster) (Piqueras et al., 2015). However, the definition of k could be a challenging task, even for simple problems, since noise could affect performance. Moreover, the centroids are initially randomly selected and for better results the whole dataset needs to be iteratively analyzed to define reliable centroids. The distance computations in a hyperspectral scenario consider the signature of a pixel that wraps the spatial and spectral information. 3.3.2.5 Superpixel Segmentation Superpixel segmentation algorithms have been broadly applied in HSI, mostly to exploit spatial dependencies in the image. Segmentation is often an exhaustive process to partition the image into homogeneous regions, and different techniques have been proposed for the segmentation of hyperspectral images, such as watershed and hierarchical segmentation. The superpixel can be defined as a perceptually uniform region and can be seen as a nonoverlapping region with an adaptive shape. This technique tends to divide images on visual boundaries containing single objects. Thus, principal component analysis and other methods need to be carried out as post-processing techniques.

3.4  PERFORMANCE METRICS Performance metrics provide support for interpreting and evaluating the accuracy of the method used in the segmentation (Tarabalka et al., 2010; Priego et al., 2013). There are several metrics that can be used to quantify and provide support to a given method: • Kappa coefficient: percentage of agreement (correctly classified pixels) corrected by the number of agreements that would be expected purely by chance. • Overall accuracy (OA): percentage of correctly classified pixels. • Average accuracy (AA): mean of class-specific accuracies, that is the mean of the percentage of correctly classified pixels for each class. • Class-specific accuracies: percentage of agreement of each class (correctly c­ lassified pixels).

42

Sylvio Barbon Jr. et al.

3.5 CONCLUSION Although the area of hyperspectral images segmentation is well established, there are more possibilities for further investigation. Most of the work in hyperspectral images segmentation algorithms is an extension from computer vision methods in two or three dimensions. These works mainly focus on the reduction of computational complexities, since the addition of spectral information increases the difficulty of processing images using traditional approaches. A necessity for successfully implementing hyperspectral images segmentation methods is the capacity to extract information from the large amount of data structured as a hypercube. Through the development of new or the optimization of existing techniques, there are more possibilities for researchers to further investigate the subject by developing fast, accurate, and automatic methodologies suitable for food quality evaluation.

REFERENCES Bong CW, Rajeswari M. Multi-objective nature-inspired clustering and classification techniques for image segmentation. Applied Soft Computing. 2011 Jun 30; 11(4):3271–3282. Comer M, Delp E. Morphological operations for color image processing. Journal of Electronic Imaging. 1999; 8:279–289. ElMasry GM, Nakauchi S. Image analysis operations applied to hyperspectral images for non-invasive sensing of food quality–A comprehensive review. Biosystems Engineering. 2016 Feb 29; 142:53–82. Fan F, Ma Y, Li C, Mei X, Huang J, Ma J. Hyperspectral image denoising with superpixel segmentation and low-rank representation. Information Sciences. 2017; 397: 48–68. Ghamisi P, Yokoya N, Li J, Liao W, Liu S, Plaza J, Rasti B, Plaza AJ. Advances in hyperspectral image and signal processing: A comprehensive overview of the state of the art. IEEE Geoscience and Remote Sensing Magazine. 2017; 5(4):37–78. Gillis DB, Bowles JH. “Hyperspectral image segmentation using spatial-spectral graphs.” Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XVIII. Vol. 8390. International Society for Optics and Photonics, 2012. Graña M, Villaverde I, Maldonado JO, Hernandez C. Two lattice computing approaches for the unsupervised segmentation of hyperspectral images. Neurocomputing. 2009; 72(10–12):2111–2120. Ivorra Martínez E, Sánchez Salmerón AJ, Verdú Amat S, Barat Baviera JM, Grau Meló R. Shelf life prediction of expired vacuum-packed chilled smoked salmon based on a KNN tissue segmentation method using hyperspectral images. Journal of Food Engineering. 2016; 178:110–116. Li J, Bioucas-Dias JM, Plaza A. Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning. IEEE Transactions on Geoscience and Remote Sensing. 2010; 48(11):4085–4098. Li J, Bioucas-Dias JM, Plaza A. Spectral–spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random fields. IEEE Transactions on Geoscience and Remote Sensing. 2012; 50(3):809–823. doi:10.1109/TGRS.2011.2162649.

Image Segmentation

43

Li J, Chen L, Huang W. Detection of early bruises on peaches (Amygdalus persica L.) using hyperspectral imaging coupled with improved watershed segmentation ­algorithm. Postharvest Biology and Technology. 2018; 135:104–113. Pike R, Patton SK, Lu G, Halig LV, Wang D, Chen ZG, Fe B. A minimum spanning forest based hyperspectral image classification method for cancerous tissue detection. In Medical Imaging 2014: Image Processing (Vol. 9034, p. 90341W). International Society for Optics and Photonics. Piqueras S, Krafft C, Beleites C, Egodage K, Von Eggeling F, Guntinas-Lichius O, Popp J, Tauler R, De Juan A. Combining multiset resolution and segmentation for hyperspectral image analysis of biological tissues. Analytica Chimica Acta. 2015; 881:24–36. Plaza A, Benediktsson JA, Boardman JW, Brazile J, Bruzzone L, Camps-Valls G, Chanussot J, Fauvel M, Gamba P, Gualtieri A, Marconcini M, Tilton JC, Trianni G. Recent advances in techniques for hyperspectral image processing. Remote Sensing of Environment. 2009; 113:S110–S122. Priego B, Souto D, Bellas F, Duro RJ. Hyperspectral image segmentation through evolved cellular automata. Pattern Recognition Letters. 2013 Oct 15; 34(14):1648–1658. Ravikanth L, Jayas DS, White ND, Fields PG, Sun DW. Extraction of spectral information from hyperspectral data and application of hyperspectral imaging for food and agricultural products. Food and Bioprocess Technology. 2017 Jan 1; 10(1):1–33. Raychaudhuri B. Imaging spectroscopy: Origin and future trends. Applied Spectroscopy Reviews. 2016 Jan 2; 51(1):23–35. Serra J. Image Analysis and Mathematical Morphology. Academic, New York (1982). Soille P. Morphological Image Analysis: Principles and Applications. Springer, Heidelberg (2003). Tarabalka Y, Chanussot J, Benediktsson JA. Segmentation and classification of hyperspectral images using watershed transformation. Pattern Recognition. 2010; 43(7):2367–2379. Von Neumann J, Aw B. 1966. Theory of Self-Reproducing Automata. University of Illinois Press: Champaign, IL.

Chapter

4

Data Extraction and Treatment Yao-Ze Feng and Hai-Tao Zhao Huazhong Agricultural University

CONTENTS 4.1 Introduction 45 4.2 Spectral Extraction and Treatment 46 4.2.1 S pectral Extraction 46 4.2.2 S pectral Treatment 48 4.2.2.1 Spectral Transforms 48 4.2.2.2 Spectral Pretreatment 48 4.3 Image Feature Extraction 50 4.3.1 R  un Length Matrices 50 4.3.2 GLCM 51 4.3.2.1 Gabor Filter 55 4.4 Conclusions 56 References 56

4.1 INTRODUCTION Hyperspectral imaging has shown its great power in wide applications particularly in the detection of food quality and safety (Feng and Sun 2012; Gowen et al. 2007; Manley 2014; Sun 2010). Such versatility within hyperspectral imaging is attributed to its unique characteristic that it allows simultaneous acquisition of comprehensive optical attributes of objects, including both spectral and image information (Park and Lu 2015). The data acquired using hyperspectral imaging systems are three dimensional with two dimensions for spatial coordinates and one dimension for wavelength scale. Figure 4.1 shows the structure of a hyperspectral image. The ways for interpreting hyperspectral data (or hypercube) are manifold. From Figure 4.1, it could be understood as an organized combination of spectra where each pixel is represented by a spectrum. Alternatively, the hypercube can be considered simply as a stack of grayscale images in different wavelengths. The first approach offers a good way to investigate spectral variations of the sample while the latter is most beneficial in exploring image features of the hypercube. Depending on the different ways of understanding as well as the aim of the analyses, both spectral and image information can be extracted and further processed to be correlated with the attributes of objects of interest by employing different chemometric algorithms. This chapter focuses on the extraction of hyperspectral information and the treatment of these data.

45

46

Yao-Ze Feng and Hai-Tao Zhao

HYPERCUBE

Spectral extraction

Image feature extraction

Wavelengths

FIGURE 4.1  Structure of hyperspectral images and data extraction.

4.2  SPECTRAL EXTRACTION AND TREATMENT 4.2.1 Spectral Extraction Spectra are produced when the intensity of light source is altered at different wavelengths after interaction with the objects under detection. Such variations in light intensity could be due to the absorption of light energy by the molecules in the matter and the scatter effect arisen by the inhomogeneity feature of the measured objects. Therefore, the spectra of samples record important chemical and physical information on the detected objects and this forms the basis for further modeling analyses. The extraction of spectra from the hypercube relies on the identification of regions of interest (ROIs). As mentioned earlier, the hypercube could be interpreted as stack combination of two-dimensional images at different wavelengths. This is helpful in visualizing the shape of objects and therefore offers a good way for selecting the desired ROIs. One way to identify ROIs is to take advantage of commercial software available. The utilization of commercial software such as ENVI is quite straightforward, where one only needs to manually define an area of any shape, and the software will help conduct statistical analysis on the selected region for different wavelengths and finally provide a mean spectrum to represent that particular sample. Although simple, this method requires huge workload since the statistical analysis of the large dataset is time-consuming and this tedious procedure has to be repeated many times. The other way to identify ROIs is to implement image processing algorithms, where segmentation of a two-dimensional image of the object is required. This image can be produced in many different ways depending on the characteristics of samples and image quality. During spectral extraction, it is greatly encouraged to extract as many spectra as possible from the sample. Particularly for images containing background, it is preferred to exclude the background while keeping all the information from the samples. To achieve this, different morphological operations should be employed in image segmentation to create good image masks. An example for the general procedure for producing a mask is shown in Figure 4.2. The first step is to produce a band math image by conducting band math where difference or sometimes ratio of two images is calculated. The aim of using a band math image is to enhance the contrast between foreground and background. However, a hyperspectral image consists of many two-dimensional images at different wavelengths, it is therefore important to allocate appropriate bands for band math operation from the

Data Extraction and Treatment

Band math

47

Thresholding

Reshape

Postprocessing

HYPERCUBE

Mask application Reshape

Mask

Reshape

Compute average

Spectra

FIGURE 4.2  Flow chart for spectral extraction.

massive data space. In practice, the raw spectra of background and samples are first selected and compared. Two bands showing the maximum difference and minimum variation between the spectral parameters (i.e., reflectance, transmittance, or absorbance) of background and sample are identified, respectively (Feng et al. 2013). In such a way, the difference between background and foreground is maximized and therefore further segmentation could be much easier. The second step for building a mask is to segment the band math image. It is n ­ ormally achieved by setting a threshold across the obtained band math images. This thresholding value could be set manually or obtained by employing Otsu’s method (Ariana and Lu 2010). Both methods are based on the investigation of the histogram of the grayscale images. A histogram is a figure showing the distribution of grayscale values of image pixels. For the band math image, a histogram is mainly dominated by two peaks, one stands for sample components and the other for background due to the intrinsic advantage of band math image, which enhances chemical difference between samples and background. The manual method arbitrarily selects the threshold, whereas the Otsu’s method is an automatic method based on clustering and it aims to separate objects by minimizing in-class variance (Otsu 1979). Despite the method used, a binary image consisting of 0s and 1s is then produced after segmentation. This image is called mask, while 0s and 1s are used to represent background and objects, respectively. The last step for establishing the final mask is to carry out post-processing. Due to image variations, the initial segmentation may not produce perfect masks and therefore further post-processing is required to eliminate background portions that are mistakenly recognized as samples. Image morphological operators, including dilation, erosion, open, and close, are useful tools for such tasks. Erosion and opening operations can lead to the elimination of the boundaries of regions of foreground or small meaningless regions. Dilation and closing can result in smaller holes in the objects. Details about these operations can be found elsewhere (Heijmans 1994; Soille 2013). When the final mask is developed, it could then be applied to the original hyperspectral image to suppress the background into uniform 0s while keeping the original spectral profiles for samples. In practice, a mask is reshaped into a column (n-by-one) constituting 1s and 0s and the hyperspectral image is also reshaped but into a two-dimensional matrix with n rows and lambda (number of wavelengths) columns. The rows in the hyperspectral

48

Yao-Ze Feng and Hai-Tao Zhao

image are then mandatorily set to 0s if the corresponding rows in the unfolded mask vector are also 0s. On the other hand, to obtain the representative spectrum for each sample, rows in the hyperspectral image corresponding to sample rows in the reshaped column are averaged. Alternatively, the median value for all sample pixels at each wavelength is calculated. However, it was found that the average and median spectra tended to produce similar model performance since the number of pixels for each hyperspectral image is normally very large (Feng and Sun 2013b). 4.2.2 Spectral Treatment The raw spectra extracted from the calibrated hyperspectral images record the whole picture of the sample, not only targeted chemical and physical information but also a variety of data that are irrelevant to analysis or simply noise. Therefore, it is sometimes important to implement necessary spectral transforms or spectral preprocessing methods to modify the data or eliminate noise levels. 4.2.2.1 Spectral Transforms Depending on the imaging modalities, the acquired hyperspectral images could be recorded in reflectance, absorbance or transmittance. Spectral response in different units may indicate different contributions from chemical and physical attributes of the samples. Therefore, in order to account for the particular components or certain characteristics of samples of interest, it is necessary to investigate different spectral transforms. Reflectance is the most common mode for hyperspectral image acquisition and it is deemed to reflect the information of the top thin layer of the sample. It can be transformed into absorbance and Kubelka–Munk (K–M) units for easier interpretation of absorbance and scattering interaction between samples and light following the equations given below:

A = − log R (4.1) K–M =

(1 − R)2 . (4.2) 2R

In terms of spectral transformation, both A and K–M are nonlinear functions of R, which means A and K–M may provide possible alternatives to establish linear relationship between spectral variables and reference values of samples if a direct linear model based on reflectance is not satisfactory. On the other hand, absorbance and K–M unit, to some extent, can reflect the chemical information, due to absorbance phenomenon, and physical information, due to scattering effects. Therefore, they are useful tools in both potentially optimizing model performance and better interpretation of calibration ­models (Feng and Sun 2013a; Kamruzzaman et al. 2016). 4.2.2.2 Spectral Pretreatment The spectra attained not only contain complicated information of the sample under study but also interference from imaging device as well as environmental variations. Since the signal-to-noise ratio of sample spectra is very important in further modeling analysis, it is necessary to investigate various spectral preprocessing methods as introduced below to eliminate useless information.

Data Extraction and Treatment

49

4.2.2.2.1 Standard Normal Variate (SNV)

SNV is a reference-independent method that is frequently utilized to suppress scatter effect in the spectra. The SNV-corrected spectra can be computed using the following equation (Barnes et al. 1989):

XSNV =

Xorig − Xmean (4.3) SD

where Xorig, X mean and X SNV are original spectrum, mean value of spectrum and the SNV corrected spectrum, respectively, and SD is the standard deviation of the individual spectrum. 4.2.2.2.2 Multiplicative Scatter Correction (MSC)

Multiplicative scatter correction was first developed to eliminate scattering interference of the samples, to achieve the same function as SNV. It was then found useful in eradicating varying background spectra caused by nonscattering reasons. Hence, it is also called multiplicative signal correction (Geladi et al. 1985). Nevertheless, the basic principle of MSC is as follows. The basic idea of MSC is to make all spectra in the calibration set presented at the same scattering level so that the scattering effect is eliminated among samples. In mathematics, this is achieved by comparing the original spectra with a reference spectrum (usually mean spectrum for the calibration set). The specific calculation involves two steps; that is, coefficients estimation and spectral correction. In the coefficients estimation step, each individual original spectrum is regressed onto the reference spectrum to achieve linear regression coefficients, as shown below:

Xorig = b0 + b1Xref + e, (4.4)

where X ref is the reference spectrum; b 0 and b1 are correction coefficients; and e is the residual error. Knowing the values of correction coefficients, the original spectra could be modified as MSC-corrected spectra X MSC using the following equation:

XMSC =

X orig − b0 . (4.5) b1

Nevertheless, it should be pointed that there are two types of mean spectra that could be selected as reference spectra, with one being the average spectrum for each image (local spectrum for one sample) and the other being the average spectrum for all hypercubes (global spectra for all samples). Different reference spectra will lead to different levels of scattering reduction. If a local spectrum is used, only scattering effects within the sample is leveled up; if a global spectrum is involved, scatter correction among samples is achieved (Feng and Sun 2013b). The advantage of using a local spectrum is that this method necessitates no prior knowledge of reference for new samples since the reference could be calculated directly for the hypercube. The global spectrum method is more beneficial in eliminating scattering effect among all samples. One should choose the ­reference according to different modeling purposes. 4.2.2.2.3 Spectral Difference/Derivative

It is known that with the application of the first and second derivative methods, the offset and slope of functions could be removed, respectively. This also applies in spectral

50

Yao-Ze Feng and Hai-Tao Zhao

analyses. However, more practically, spectral difference is utilized instead of calculating derivatives. In derivative analyses, the key concern is enhancement of noise since the derivative method magnifies everything about the difference at neighboring wavelengths. To avoid the influence from random noise, Savitzky–Golay (SG) routine (Savitzky and Golay 1964) is usually implemented. The SG method tries to smooth off the random noise before conducting difference operations. The specific procedure involves the introduction of a sliding window. The width of the window is user-defined and normally covers odd number of data points. When the window is attached to each spectrum and sliding along the wavelength scale, the spectral segment in the window is fit with a polynomial function of second through fourth order, based on which derivative is carried out. For simple computation, convolution operation can be applied by using published fitting coefficients proposed by Savitzky and Golay (1964) and corrected by Steinier et al. (1972). Generally, utilization of derivative methods can exterminate baseline shift and slow background variations. Moreover, derivative methods could also help in enhancing spectral resolution and uncovering more detailed information that is not prominent in the original spectra. However, appropriate parameters, such as window width and order of polynomials should be optimized.

4.3  IMAGE FEATURE EXTRACTION Texture is the main image feature that could be extracted from hyperspectral images for further analyses, although color information can also be presented by hyperspectral imaging by establishing calibration models (Qiao et al. 2007). Image texture, as defined by IEEE (1990), illustrates the spatial distribution of pixel intensities. Extraction of texture features can help uncover more interesting information associated with objects of interest and it has been directly utilized or combined with spectral analysis to characterize measured objects. In image analysis, texture is essentially the description of relationship between pixels with the same or different gray levels. Current popular texture measures for application in hyperspectral imaging mainly involve run length matrices, gray level co-occurrence matrix (GLCM) and Gabor filter. 4.3.1 Run Length Matrices Run Length Matrices characterize texture features by the gray-level run, which is a set of consecutive pixels with the same gray level (Galloway 1974; Tang 1998). For an image, a matrix R can be attained to show the distribution of run length in a given direction. The horizontal and vertical coordinates of the matrix R(Ng, M) represent the number of gray level intensities N (256 for pixels coded with 8 bits) and the maximum number M of run length for a given direction (e.g., in the horizontal direction), respectively. A value R(g, l) of the matrix R counts the number of run lengths that is equal to l for the gray-level value g in a given direction of the image. Furthermore, five descriptors, such as short run emphasis (SRE), long run emphasis (LRE), gray-level non-uniformity (GLNU), run length nonuniformity (RLNU) and run length percentage (RLP), are used to stand for texture features (Galloway 1974). These features of run length matrices are calculated as follows:

SRE =

1 nr

Ng

M

g

l

∑∑

R ( g, l ) (4.6) l2

Data Extraction and Treatment







1 LRE = nr

Ng

M

∑ ∑ R ( g, l ) ⋅ l (4.7)

1 GLNU = nr 1 RLNU = nr

2

g

l

Ng

2

  

M

∑∑

 R ( g , l ) (4.8) 

  

N

 R ( g , l ) (4.9) 

g

M

l

∑∑ l

RLP =



51

g

2

nr , (4.10) nP

where nr is the total number of runs and n P is the number of pixels in the image, M is the maximum of run length and Ng is the number of gray levels. 4.3.2 GLCM GLCM is a very effective method to extract textural features (Haralick and Shanmugam 1973). To characterize image features, a set of GLCMs carrying texture information is calculated for four kinds of angular and the distance between adjacent pixel pairs are determined on the image. Afterward, 14 statistics of textural feature can be extracted from these GLCMs. The specific procedures are as follows. Define an image analyzed with Nx columns, Ny rows and Ng gray scales. The range of columns domain is denoted as Lx = {1, 2,  ,  N x } and the range of rows domain as  Ly = {1, 2,  ,  N y }. Therefore, there will be a total of Ly × L x pixels in the image at each particular band in the hypercube. Every pixel excluding those on the periphery of the image has eight nearest-neighbor pixels. As shown in Figure 4.3, with four angular, such as, 0°, 45°, 90° and 135° defined and every square unit standing for a pixel, the numbers (1–8) are the index of nearest-­ neighbor pixels at different angles for the center pixel as indicated by the dot. Specifically, among the neighbors, Pixels 1 and 5 are 0° (horizontal) nearest neighbors; pixels 4 and 8 are 45° nearest neighbors; pixels 3 and 7 are 90° (vertical) nearest neighbors; and pixels 2 and 6 are 135° nearest neighbors to the center pixel, respectively. Based on the neighborhood defined above, a matrix indicating occurring frequencies P for two neighboring pixels separated by a certain distance in the image can then be 90o

135o

6

7

8

11

5 4

45o

3

0o

2

FIGURE 4.3  Four defined angular (0°, 45°, 90° and 135°) and eight nearest-neighbor pixels (index 1 to 8) of the center pixel (occupied by the dot).

52

Yao-Ze Feng and Hai-Tao Zhao

calculated. Denote i and j for the gray scale of two pixels and d for the distance between the two pixels, P at different angles can be attained following equations 4.11–4.14 by traversing all pixels in the image.





P ( i, j, d ,0°) = #

y

× Lx ) k − m = 0 , l − n

= d , I ( k, l ) = i, I ( m, n ) = j P ( i, j, d , 45° ) = #

{((k, l ) , ( m, n)) ∈( L

y

}

(4.11)

× Lx ) ( k − m = d , l − n = −d ) OR

(k − m = −d , l −   n = d ) , I (k, l ) = i, I ( m, n ) = j} P ( i, j, d , 90° ) = #





{((k, l ) , ( m, n)) ∈( L

{((k, l ) , ( m, n)) ∈( L

y

× Lx ) k − m  

= d , l − n = 0, I ( k, l ) = i, I ( m, n ) = j P ( i, j, d ,135° ) = #

}

(4.12)

(4.13)

{((k, l ) , ( m, n)) ∈( L

× Lx ) ( k − m = d , l − n = d ) OR (4.14) (k − m = −d , l − n =   −d ) , I (k, l ) = i, I ( m, n ) = j y

}

where # denotes the number of elements in the set. An example is shown in Figure 4.4, in which (a) represents a 4×4 image with four gray scales (0–3); (b) is the general form of any relative frequencies matrix P. Taking the value at (2,1) in Figure 4.4(b) as an example, the element in the position (2,1) of the table is the total number of times two gray tones of value 2 and 1 occurred horizontally adjacent to each other when the distance d is set as 1. Equations 4.15–4.18 show all four relative frequency matrices with the distance equals to 1 pixel.   P ( i, j,1, 0° )   =   



(a)

4 2 1 0

2 4 0 0

(b)

1 0 6 1



0 0 1 2

     (4.15)  

















   











   











   











   

FIGURE 4.4  General rule for calculating the GLCM. (a) 4 × 4 image with four gray levels 0–3. (b) General form of any GLCM for image with gray level values of 0–3. # (i,j) stands for the number of times the gray tones i and j are neighbors.

Data Extraction and Treatment

53



  P ( i, j,1, 45° )  =   

4 2 0 0

1 2 2 0

0 2 4 1

0 0 1 0

   (4.16)  



  P ( i, j,1, 90° )  =   

6 0 2 0

0 4 2 0

2 2 2 2

0 0 2 0

   (4.17)  



  P ( i, j,1,135° ) =   

2 1 3 0

1 2 1 0

3 1 0 2

0 0 2 0

   . (4.18)  

To obtain gray-tone spatial dependence matrix, appropriate frequency normalization for the matrices should be performed as follows (4.19): pi , j =



Pi , j , (4.19) R

where R is the number of neighboring pixel pairs. When the relationship is nearest horizontal neighbor (d = 1, a = 0°), there will be  2 ( N x − 1) neighboring pixel pairs on each row. Given Ny rows, there will then be a total of 2N y ( N x − 1) nearest horizontal neighboring pairs. When the relationship is nearest right-diagonal neighbor (d = 1, a = 45°) there will be 2 ( N x − 1) neighboring cell pairs for each row except the first, for which there are none, and there are Ny rows. This provides a total of 2 ( N y − 1) ( N x − 1) nearest right-diagonal neighboring pairs. By symmetry, there will be 2 ( N x − 1) ( N y − 1) nearest vertical neighbor pairs and 2 ( N y − 1) ( N x − 1) nearest left-diagonal neighbor pairs. Finally, 14 descriptors as proposed by Haralick (1979) could be extracted from these GLCMs using the following equations to represent textural features. 1. Angular second moment f1 =



Ng

Ng

i =1

j =1

∑ ∑ {P ( i, j )} (4.20) 2

2. Contrast N g −1



f2 =

∑ n=0

   n2   

 p ( i, j )   (4.21) i =1 j =1  i− j = n 

Ng

Ng

∑∑

54

Yao-Ze Feng and Hai-Tao Zhao

3. Correlation Ng

Ng

i =1

j =1

∑ ∑ {p ( i, j )} − µ µ x

f3 =



σ xσ y

y

(4.22)

where μx, μy, σx and σy are the means and standard deviations of px and py, respectively. 4. Sum of squares: Variance Ng

f4 =



Ng

∑ ∑ ( i − µ ) P ( i, j ) (4.23) 2

i =1

j =1

5. Inverse difference moment

∑ ∑ 1 + ((i − j)) (4.24)

f5 =



Ng

Ng

i =1

j =1

P i, j

2

6. Sum average 2N g

f6 =



∑ i p

x+ y

( i ) (4.25)

i=2



7. Sum variance 2N g



f7 =

∑ (i − f )  p 2

8

x+ y

( i ) (4.26)

i=2

8. Sum entropy 2N g



f8 = −

∑P

x+ y 

( i ) log {Px + y ( i )} (4.27)

i=2

9. Entropy

f9 = −

Ng

Ng

i =1

j =1

∑ ∑ P ( i, j ) log {P ( i, j )} (4.28)

10. Difference variance N g −1



f10 =

∑ (i − µ ) p 2

i =0

x− y

( i ) (4.29)

Data Extraction and Treatment

55

11. Difference entropy N g −1

f11 = −



∑p

x− y

( i ) log {px − y ( i )} (4.30)

i =0

  12, 13.  Two different information measure of correlation HXY − HXY1 (4.31) max {HX, HY}

f12 =



(

f13 = 1 − exp  −2.0 ( HXY2 − HXY ) 



HXY = −



Ng

Ng

i =1

j =1

)

1/ 2

(4.32)

∑ ∑ p ( i, j ) log ( p ( i, j )) (4.33)

where HX and HY are entropies of px and py, and HXY1 = −



HXY2 = −



Ng

Ng

i =1

j =1

∑ ∑ p ( i, j ) log {p ( i ) p ( j )} (4.34) x

Ng

Ng

i =1

j =1

y

∑ ∑ p ( i ) p ( j ) log {p ( i ) p ( j )} (4.35) x

y

x

y

14. Maximal correlation coefficient f14 = ( Second largest eigenvalue of Q) (4.36) 12

where

Q ( i, j ) =



∑ k

p ( i, j ) p ( j, k) . (4.37) px ( i ) py ( k)

4.3.2.1 Gabor Filter Gabor filter evaluates the frequency components of an image in a specific region around the ROIs at certain directions (Grigorescu et al. 2002). In hyperspectral imaging analysis, for each band image, 2D Gabor Filters is popularly applied to extract image textural features (Liu et al. 2010). To explain the major differences among textures using frequency information, a circular symmetric Gabor filter G1 can be used which is a Gaussian function modulated by a circularly symmetric sinusoidal function with the following form (Ma et al. 2003):

G1 ( x, y; u, σ ) =

 x2 + y 2  1 exp − cos 2πu 2 2   σ 2πσ 2  

(

)

x2 + y 2  (4.38) 

56

Yao-Ze Feng and Hai-Tao Zhao

where (x, y) is the coordinate of a point in 2D space, u is the frequency of the sinusoidal wave and σ is the standard deviation of the Gaussian envelope. In addition to the spatial frequency information, the directional information of image texture also represents major differences in many applications (Bau et al. 2010; Liu et al. 2010). An oriented Gabor filter G0, which is a Gaussian function modulated by an oriented harmonic function has been defined below.

G0 ( x, y; u, σ , θ ) =

 x2 + y 2  1 exp − cos 2πu ( x cos θ + y sin θ )  (4.39) 2 2  2πσ  2σ 

where θ determines the orientation of the Gabor filter. To make Gabor filters more robust against brightness difference, discrete Gabor filters Gk(k = 0, 1) were tuned to zero direct current (DC) with the application of the following formula (Zhang et al. 2003): n

n

∑ ∑ G [ i, j ] k



k = Gk − G

i=−n j=−n

(2n  +  1)2

(4.40)

where n is the filter’s radius and ( 2n   +  1) is the size of the filter. The adjusted Gabor 0   and G 1 , were used to convolute the ROI of each sample image, respectively. ­filters, G 2

4.4  CONCLUSIONS The superb merit for hyperspectral imaging is that it embraces both spectral and image features in one single system thus providing voluminous opportunities for better characterization of objects of interest. To facilitate spectroscopic analysis, spectra of samples should be extracted from the hypercube and both spectral transforms and spectral preprocessing should be employed to prepare the spectra for further model calibration when correlated with reference values. On the other hand, image features, or particularly image texture features were also introduced, which included run length matrices, GLCM and Gabor filter. However, it should be noted that despite the variety of methods available, there is still a lack of rule of thumb for the determination of the optimal method for building a calibration model from the perspective of either spectral analysis or image processing. Considering current spectral analysis methods and image features are only tested for their validity by try and error method, future work should be focused on the introduction and development of more methods or features that can potentially become good descriptors for certain applications.

REFERENCES Ariana DP, Lu R (2010) Evaluation of internal defect and surface color of whole pickles using hyperspectral imaging Journal of Food Engineering 96:583–590. Barnes R, Dhanoa MS, Lister SJ (1989) Standard normal variate transformation and de-trending of near-infrared diffuse reflectance spectra Applied Spectroscopy 43:772–777.

Data Extraction and Treatment

57

Bau TC, Sarkar S, Healey G (2010) Hyperspectral region classification using a threedimensional Gabor filterbank IEEE Transactions on Geoscience and Remote Sensing 48:3457–3464. Feng Y-Z, ElMasry G, Sun D-W, Scannell AG, Walsh D, Morcy N (2013) Nearinfrared hyperspectral imaging and partial least squares regression for rapid and reagentless determination of Enterobacteriaceae on chicken fillets Food Chemistry 138:1829–1836. Feng Y-Z, Sun D-W (2012) Application of hyperspectral imaging in food safety inspection and control: A review Critical Reviews in Food Science and Nutrition 52:1039–1058. Feng Y-Z, Sun D-W (2013a) Determination of total viable count (TVC) in chicken breast fillets by near-infrared hyperspectral imaging and spectroscopic transforms Talanta 105:244–249. Feng Y-Z, Sun D-W (2013b) Near-infrared hyperspectral imaging in tandem with ­partial least squares regression and genetic algorithm for non-destructive determination and visualization of Pseudomonas loads in chicken fillets Talanta 109:74–83. Galloway MM (1974) Texture analysis using grey level run lengths NASA STI/Recon. Technical Report N 75. Geladi P, MacDougall D, Martens H (1985) Linearization and scatter-correction for near-infrared reflectance spectra of meat Applied Spectroscopy 39:491–500. Gowen A, O’Donnell C, Cullen P, Downey G, Frias J (2007) Hyperspectral imaging–An emerging process analytical tool for food quality and safety control Trends in Food Science & Technology 18:590–598. Grigorescu SE, Petkov N, Kruizinga P (2002) Comparison of texture features based on Gabor filters IEEE Transactions on Image Processing 11:1160–1167. Haralick RM (1979) Statistical and structural approaches to texture Proceedings of the IEEE 67:786–804. Haralick RM, Shanmugam K (1973) Textural features for image classification IEEE Transactions on Systems, Man, and Cybernetics 3(6):610–621. Heijmans HJ (1994) Morphological Image Operators. Advances in Electronics and Electron Physics Suppl. Boston, MA: Academic Press. IEEE (1990) IEEE Standards Glossary of Image Processing and Pattern Recognition Terminology. IEEE. Kamruzzaman M, Makino Y, Oshita S (2016) Rapid and non-destructive detection of chicken adulteration in minced beef using visible near-infrared hyperspectral imaging and machine learning Journal of Food Engineering 170:8–15. Liu L, Ngadi M, Prasher S, Gariépy C (2010) Categorization of pork quality using Gabor filter-based hyperspectral imaging technology Journal of Food Engineering 99:284–293. Ma L, Tan T, Wang Y, Zhang D (2003) Personal identification based on iris texture analysis IEEE Transactions on Pattern Analysis and Machine Intelligence 25:1519–1533. Manley M (2014) Near-infrared spectroscopy and hyperspectral imaging: Non-destructive analysis of biological materials Chemical Society Reviews 43:8200–8214. Otsu N (1979) A threshold selection method from gray-level histograms IEEE Transactions on Systems, Man, and Cybernetics 9:62–66. Park B, Lu R (2015) Hyperspectral Imaging Technology in Food and Agriculture. Springer: Berlin, Germany.

58

Yao-Ze Feng and Hai-Tao Zhao

Qiao J, Wang N, Ngadi M, Gunenc A, Monroy M, Gariepy C, Prasher S (2007) Prediction of drip-loss, pH, and color for pork using a hyperspectral imaging technique Meat Science 76:1–8. Savitzky A, Golay MJ (1964) Smoothing and differentiation of data by simplified least squares procedures Analytical Chemistry 36:1627–1639. Soille P (2013) Morphological Image Analysis: Principles and Applications. Springer Science & Business Media: Berlin, Germany. Steinier J, Termonia Y, Deltour J (1972) Smoothing and differentiation of data by simplified least square procedure Analytical Chemistry 44:1906–1909. Sun D-W (2010) Hyperspectral Imaging for Food Quality Analysis and Control. Elsevier: Amsterdam, The Netherland. Tang X (1998) Texture information in run-length matrices IEEE Transactions on Image Processing 7:1602–1609. Zhang DD, Kong WK, You J, Wong M (2003) Online palmprint identification IEEE Transactions on Pattern Analysis & Machine Intelligence 25:1041–1050.

Section

II

Chemometrics

Chapter

5

Multivariate Analysis and Techniques Mohammed Kamruzzaman Bangladesh Agricultural University

CONTENTS 5.1 Introduction 61 5.2 Multivariate Analysis of Hyperspectral Data 63 5.2.1 Classification Methods 65 5.2.2 Multivariate Regression 66 5.3 Validation of Multivariate Calibration Model 67 5.4 Evaluation of the Multivariate Classification Models 68 5.5 Evaluation of the Multivariate Regression Models 68 5.6 Multivariate Image Processing 69 5.7 Development of Classification Map 70 5.8 Prediction Map 71 5.9 Software for Multivariate Analysis and Image Processing 74 5.10 Application of Multivariate Analysis in Some Selected Applications of Meat 75 5.11 Conclusion 79 References 79

5.1 INTRODUCTION Recently, hyperspectral imaging has emerged as a novel, non-destructive and non-contact tool for quality and safety inspection of food and agriproducts. Hyperspectral imaging, also referred to literature as imaging spectroscopy, spectroscopic imaging, imaging spectroscopy or chemical imaging (Goetz et al., 1985), is a relatively new technique that integrates conventional imaging (also called computer vision) and spectroscopic techniques in one system, enabling acquisition of spatial and spectral information simultaneously from an object. This information then forms a three-dimensional hypercube. A hyperspectral image can thus be interpreted as a group of images taken at a number of different specific wavelengths where each pixel in the image represents a spectrum for this specific point of the object or a set of spectra on a two-dimensional area as shown in Figure  5.1. This technique has several advantages in that it is rapid, precise, non-­ destructive, and multi-analytical; hence, several constituents can be predicted simultaneously from the same spectrum (Kamruzzaman et al., 2015b; Pu et al., 2015b). This technique has proved its potential for food and agriproducts including fruits, vegetable,

61

62

Meat Onion

80

Cheese

Image at 950 nm

y Bread

Image at 1280 nm (a)

20

x

Meat (c) Image at 1550 nm (b)

40

0 900

1100

1300

1500

1700

Wavelength (nm) (d)

FIGURE 5.1  The conceptual view of a hyperspectral image. (a) A three-dimensional (3D) block of a hyperspectral image of a pizza comprising two spatial dimensions (x rows × y columns) and one spectral dimension (λ wavelengths), (b) Image I(x, y) at different single wavelength. (c) Single pixel of lean meat, bread, onion and cheese. (d) Spectral signatures of lean meat, bread, onion and cheese. In the hypercube, images at two adjacent wavelengths are very similar, while images at distant wavelengths can be much less similar.

Mohammed Kamruzzaman

Onion

60 Reflectance (%)

λ

Cheese Bread

Multivariate Analysis and Techniques

63

red meat such as beef, pork, and lamb, white meat such as chicken, turkey, and fish (Ariana & Lu, 2010; Barbin et al., 2012a;ElMasry et al., 2011a, b; Feng et al., 2013; Gowen et al., 2009; He et al., 2013; Iqbal et al., 2013a; Kamruzzaman et al., 2011; Kavdir & Guyer, 2008; Naganathan et al., 2008; Nakariyakul and Casasent, 2009; Park et al., 2011; Pu et al., 2014, 2015a; Tao et al., 2012; Sone et al., 2012; Wu et al., 2012). In addition, a number of reviews and book chapters have been published on hyperspectral imaging for evaluating agrifood products quality, safety, authenticity and adulteration detection (ElMasry et al., 2012a; Feng & Sun, 2012; Iqbal et al., 2013b; Gowen et al., 2015; Kamruzzaman & Sun, 2016; Kamruzzaman, 2016; Kamruzzaman et al., 2015a; Pu et al., 2015b; Sun, 2010; Wu & Sun, 2013). Indeed, hyperspectral imaging techniques have received a point of curiosity throughout the scientific community worldwide and its application is increasing day by day. One of the advantages in developing hyperspectral imaging systems is the wealth of the spectral data that resides in the images. This data is multivariate in nature due to a large number of data variables, one at each wavelength, for each sample. Multivariate data analysis is thus an indispensable part of this novel analytical technique. It is required to appropriately extract meaningful information from the spectra to correlate with the measured attribute under investigation. In general, multivariate data analysis methods comprise a group of statistical and mathematical techniques that analyze multiple variables simultaneously (Artyushkova & Fulghum, 2001). With the fusion of appropriate multivariate methods, hyperspectral imaging answers the questions about the sample such as what chemical species are in the sample, how much of each is present, and most importantly, where they are located. That is why multivariate analysis is the cornerstone of hyperspectral analysis. In the last few decades, multivariate analysis has emerged as an important analytical tool in many applications. The reason for the large interest in multivariate analysis (more specifically multivariate calibration) is that the technique is fast and cheap, not very accurate but accurate enough for many real analytical applications. Basically, the calibration model is simply a regression model that will allow the prediction of quality/safety traits based on spectral data. In recent years, the rapid progress has made all kinds of multivariate methods to deal with hyperspectral data, although it cannot meet the practical needs very well. Although there are many challenges for implementing hyperspectral imaging for real-time implementation, it is expected to become one of the most promising analytical tools with the fusion of multivariate data analysis for food quality, safety and other issues.

5.2  MULTIVARIATE ANALYSIS OF HYPERSPECTRAL DATA Hyperspectral images typically contain tens of thousands of spectra and hundreds of channels, wavelengths, or bands (Figure 5.2). This massive quantity of data within a hyperspectral image provides both challenges and opportunities for analysis. Multivariate data mining tools are thus required to efficiently extract meaningful information from this massive amount of data, and they must be used correctly. Multivariate methods decompose complex multivariable data into simple and easily interpretable structures to better understand the chemical and biological information of the tested samples (Bro et al., 2002; Geladi, 2003). The purpose of multivariate methods in spectral data is to determine the properties of materials through multiple measurements on the sample and then manipulating and interpreting its chemical data by applying mathematical, statistical and other methods (Wang & Paliwal, 2007). However, appropriate selection

64

Mohammed Kamruzzaman

λ

y x Gray scale image (single channel)

Color image (RGB) (3 discrete bands)

Multispectral image (2-10 discrete bands)

Hyperspectral image (10-200+ continuous bands)

FIGURE 5.2  Numbers of bands used in different types of images. The concept of multi-

spectral and hyperspectral imaging is similar but differs in the number of spectral bands used during image acquisition and data generation. of samples to be used in multivariate analysis is very important in establishing more precise multivariate calibration model. In reality, the calibration model must be made on a large number of samples representing a wide variation in quality traits that one might find during routine sampling. However, before or during building the calibration models through multivariate analysis, outliers (both X and y) should be identified and removed, as these can adversely affect the robustness and predictivity of the resulting model (Møller et al., 2005). Basically, an outlier is an observation that shows some kind of departure from the majority of recorded observations. There are usually three different types of outliers encountered during developing calibration model: (i) X-sample outliers; that is, samples, for which the spectra depart markedly from the others, (ii) X-variable outliers; that is, spectral variables that behave markedly differently from the others, and (iii) y-samples outliers; that is, anomalous in the value of the response (Houmøller et al., 2007; Valderrama et al., 2007). The Y outliers can be detected from a large residual value (yresidual = ypredicted − ymeasured), whereas X outliers can be detected by PCA of the predictors, the leverage, the scores on the latent variables (LVs) of partial least square PLS regression, the residual variance; that is, the fraction of the information in the predictors not used by the regression model (Forina et al., 2007). Once the spectral data is available for multivariate modeling, it is necessary to mitigate the noise from the data by applying pre-processing to obtain a good and robust prediction model. The purpose of spectral pre-processing is to identify and remove undesired effects such as light scattering, path length variations, and random noise resulting from variable physical sample properties or instrumental effects that interfere with the desired prediction. Generally, the selection of the optimum pre-processing tool often requires iteration between the calibration model and the pre-processing method. The best pre-processing method should produce a robust model with the best predicting ability (Rinnan et al., 2009). After pre-processing, data is now ready for final multivariate calibration. The final goal of a multivariate calibration is to predict the characteristics of new samples accurately from the spectra. Before prediction, one should make sure that the new samples are similar to the calibration samples. That is why, sample set incorporating all chemical and physical variations normally encountered during routine sampling should be used to generate and optimize a calibration model. If the calibration samples are not representative of the unknown samples to predict, then the prediction

Multivariate Analysis and Techniques

65

obtained from the calibration model will be unreliable (Geladi, 2003). Multivariate techniques typically applied in spectral data can be divided into multivariate classification for qualitative analysis and multivariate regression for quantitative analysis as shown in Figure 5.3. The detailed description of each multivariate technique is beyond the scope of this chapter, as some of the important multivariate techniques will be discussed in Chapters 6-11. Common definitions of some chemometric terms are given in Table 5.1. 5.2.1 Classification Methods Multivariate classification, also called pattern recognition, can be unsupervised or supervised. Supervised pattern recognition aims to establish a classification model in order to classify new unknown samples to previously defined known classes on the basis of its pattern of measurements. On the other hand, unsupervised classifications do not require a prior knowledge about the group structure in the data and the data are classified according to their natural groups. Hence samples are grouped according to a similarity metric, which can be distance, correlation or some combination of both. This type of analysis is often very useful for preliminary evaluation of the information contents in the spectral dataset. PCA is the most frequently used unsupervised technique for qualitative classification. Commonly used supervised methods are soft independent modeling of class analogy (SIMCA), linear discriminant analysis (LDA), K-means and fuzzy clustering, partial least squares-discriminant analysis (PLS-DA), Fisher discriminant analysis (FDA) and ­artificial neural network (ANN).

FIGURE 5.3  Multivariate analysis techniques for hyperspectral data analysis. MLR: Multiple Linear Regression; PCA: Principal Component Analysis; FA: Factor Analysis; LDA: Linear Discriminant Analysis; CDA: Canonical Discriminant Analysis; PLS-DA: Partial Least Squares–Discriminant Analysis; ANN: Artificial Neural Network; SVM: Support Vector Machine; PCR: Principle Component Regression; PLSR: Partial Least Square Regression; SIMCA: Soft Independent and Modelling Class Analogy; MCRALS: Multivariate Curve Resolution-Alternating Least Squares; PARAFAC: Parallel Factor Analysis; U-PLS/RBL: Unfolded Partial Least Squares Coupled to Residual Bilinearization; HCA: Hierarchical Cluster Analysis; GMM: Gaussian Mixture Model; SOM: Self-Organizing Map.

66

Mohammed Kamruzzaman

TABLE 5.1  Some Common Terms in Multivariate Chemometric with Definition Multivariate data Reference data Pre-processing (spectral)

Pre-processing (spatial)

Pre-treatment

Qualitative modeling

Quantitative modeling

Supervised learning

Unsupervised learning

Data containing many variables each at different wavelengths. Also called X-data or spectral matrix Often referred as the y data and measured by traditional wet-chemical methods to build the reference chemical matrix Mathematically treat the raw spectral data from hyperspectral images to correct undesired effects such as light scattering, path length variations and random noise resulting from variable physical sample properties or instrumental effects prior to spectral analysis. This may include derivatives, MSC, SNV, etc. Applied to the hyperspectral images to remove noise and to improve the quality of the spectral data. This may include histogram equalization, filtering, transformation and arithmetic operations, etc. Transforming the pre-processed data to make them suitable for analysis. This may include spectral transformation, normalization, scaling and outlier removal, etc. Qualitative models classify or discriminate the samples into certain groups based on their respective spectra without conducting chemical background determination. It compares the spectra and search for similarities or differences within the spectra The goal is to quantify the property of interest by building a relationship between a desired physical, chemical or biological attributes (y) of an object and its spectral matrix (X) Establish a classification model in order to classify new unknown samples to previously defined known classes on the basis of its pattern of measurements Unsupervised classifications do not require a prior knowledge about the group structure in the data and the data are classified according to their natural groups. Hence samples are grouped according to a similarity metric, which can be distance, correlation or some combination of both

5.2.2 Multivariate Regression The multivariate regression model consists of building a relationship between a desired physical, chemical or biological attribute of an object and its spectra. The most widely used multivariate regression methods in quantitative analysis are multiple linear regression (MLR), Principle Component Regression (PCR), and partial least squares regression (PLSR). MLR is used to build a mathematical function associating two or more independent variables with a dependent variable by fitting a linear equation to observed data. However, MLR fails if the number of variables is more than the number of samples. Generally, the number of variables in hyperspectral imaging experiment is larger than the number of samples. It is thus not possible to calculate MLR directly, and dimensionality reduction is required before its use. MLR models typically do not perform well in the spectral data because of the often high colinearity of the spectra and easily lead to overfitting and loss of robustness of the calibration models (Nicolai et al., 2007). PCR is a two-step multivariate method that involves PCA followed by MLR on the sample property using selected principal components (PCs) instead of the original variables as predictors. The disadvantage is that the PCs are ordered according to decreasing explained variance of the spectral matrix, and that the first PCs which are used for the

Multivariate Analysis and Techniques

67

regression model are not necessarily the most informative with respect to the response variable (Nicolai et al., 2007). PCR has the ability to eliminate the lesser PCs that allows some noise reduction in the regression. However, as mentioned before, PCR is a two-step procedure and thereby it has the risk of loss of useful information in the eliminated PCs, and some noise may remain in PCs used for regression. Moreover, the practical meaning of the PCs is unclear, and the relationship between the dependent variables and the PCs is indirect (Liu et al., 2013). In recent years, PLSR has become the de facto standard in multivariate spectral analysis (Barbin et al., 2012b,c; ElMasry et al., 2011b, 2012b,c; Feng et al., 2013; Kamruzzaman et al., 2012a,b; 2013a,b; 2018). The PLSR decomposes a large number of independent variables into a small number orthogonal factor called LVs. These LVs are designed in an appropriate way to capture most information in X as well as in y. These LVs are statistically independent (uncorrelated) and ideally carry all relevant information leading to more stable predictions. Subsequently, these LVs are correlated with reference values (i.e., true measured values) of the target parameter obtained by a laboratory reference method for the samples of the calibration set. In reality, PLSR performs better and requires fewer LVs than PCs for PCR (Kamruzzaman et al., 2015d). Therefore, PLSR is more parsimonious than PCR. It is not surprising because PCR estimates each PC of the spectral matrix (X) to maximize the amount of explained variance without using the response variable (y), so that there is no guarantee that the calculated PCs are important with respect to the response variable for prediction, while PLSR decomposes both X and y to calculate LVs that are really important for better prediction (Nicolai et al., 2007). When the spectral data and target attributes are not linearly related as a result of physical sample properties or instrumental effects, non-linear methods such as ANN and SVM regression are very suitable for analysis. The most widely used ANN is the multilayer feed forward neural network where the neurons are arranged in three layers: input layer, hidden layer and output layer. The spectral value at every wavelength is fed to the input layer, while the output layer delivers the prediction of the attribute. Feed forward neural network usually has one or more hidden layers, which enable the network to deal with nonlinear and complex correlation. In SVM regression, the input is first mapped in high-dimensional feature space using nonlinear mapping, and then a linear regression is constructed in this feature space. The solution of SVM will become more complex and the speed will decrease if the sample size is large. To solve these problems, an optimized version of SVM called the least squares support vector machine (LS-SVM) can be used (Suykens & Vandewalle, 1999).

5.3  VALIDATION OF MULTIVARIATE CALIBRATION MODEL Validation is an important step in all multivariate calibration methods. Once the calibration models have been developed, it is mandatory to check the reliability of the developed calibration models in predicting unknown samples to ensure that the derived model is representative and will work in the future for new, similar data. In multivariate analysis, the two most widely used validation methods are cross-validation (segmental crossvalidation or full cross-validation, leave-one-out cross-validation) and test-set (external) validation. The choice of validation method depends on the data, the problem, and the chemometric methods. However, there is no better validation than testing on an entirely independent set (external validation) and preferably they should consist of samples from different batches taken at different times.

68

Mohammed Kamruzzaman

5.4 EVALUATION OF THE MULTIVARIATE CLASSIFICATION MODELS The efficacy of the multivariate classification model can be determined by the individual and overall accuracy both in the calibration and validation sets. The individual classification accuracy can be determined by the number of correctly classified samples in each class divided by the total number of samples in each class. The overall classification accuracy of the model can be determined as the number of correct classifications in all classes divided by the total number of samples. On the other hand, model performance can also be statistically determined by using the sensitivity (i.e., how good the model is to correctly classify positive samples), specificity (i.e., how good the model is to correctly classify negative samples) and class error. These parameters can be expressed according to the following equations:

Sensitivity =

TP TP + FN



Specificity =

TN TN + FP



Class error = 1 −

sensitivity + specificity 2

where TP = true positive, TN = true negative, FP = false positive and FN = false negative. Another important statistic evaluation often used in multivariate classification is the receiver operator characteristics (ROC) curves. It is a technique for visualizing, organizing and selecting classifiers based on their performance. Spackman (1989) adopted ROC in evaluating and comparing machine learning algorithms. Nowadays, ROC is used widely in multivariate classification techniques due to the fact that simple classification accuracy is often a poor metric for measuring performance (Fawcett, 2006). In ROC curve, the TP rate (sensitivity) is plotted in a function of the FP rate ­(1−specificity) for different cutoff points. Therefore, each point on the ROC basically represents a sensitivity/specificity pair corresponding to a particular decision threshold. The best prediction model would generate a point in the upper left corner or coordinate (0, 1) of the ROC, representing 100% sensitivity (no FN) and 100% specificity (no FP). Therefore, the closer the ROC plot to the upper left corner, the higher the overall accuracy of the test.

5.5  EVALUATION OF THE MULTIVARIATE REGRESSION MODELS The quality of the multivariate calibration regression models are evaluated by several statistical criteria such as the standard error of calibration (SEC) or root mean squared errors for calibration (RMSEC), standard error of prediction (SEP) or root mean squared errors of prediction estimated by cross-validation (RMSECV), coef2 ficients of determination for calibration, cross-validation or prediction (Rc2 , Rcv and 2 Rp , respectively) or correlation coefficient for calibration, cross-validation or prediction (R c , R cv and R p, respectively). These parameters are calculated using the following equations:

Multivariate Analysis and Techniques N

∑ ( yˆ − y )

2

i

R2 = 1 −



69

i =1 N

∑

i

(5.1)

( yˆ i − yi )

2

i =1

N

∑ ( yˆ − y )

2

i







RMSEC or RMSECV =

SEC =

SEP =

1 Nc − 1

1 Np − 1

i

i =1

N

Nc

(5.2)

∑ ( yˆ − y ) (5.3) 2

i

i

i =1

Np

∑ ( yˆ − y − bias) . (5.4) 2

i

i

i =1

To obtain SEP, the Bias is computed as:



1 bias = Np

Np

∑ ( yˆ − y ) i

i

i =1

where yˆ i = predicted value of the ith sample, yi = measured value of the ith sample, N = number of samples, Nc = number of sample in the calibration set, Np = number of sample in the validation (testing) set and bias = average difference between predicted and measured values. Moreover, the ratio of prediction to deviation (RPD) can also be used in order to test the precision and accuracy of the models in terms of the standard deviation of the reference data. Mathematically, the RPD is defined as the ratio of standard deviation (SD) of the reference values over the standard error in cross-validation or prediction (RPD = SD/RMSECV or SECV or SEP). This means that if the RMSECV (or SECV or SEP) is smaller compared to SD, a relatively high RPD value results, and therefore the calibration model is considered robust. The higher the value of RPD, the greater the probability of the model to predict the composition precisely in samples outside the calibration set.

5.6  MULTIVARIATE IMAGE PROCESSING The main advantage of hyperspectral imaging over traditional spectroscopy is that it facilities the spatial distribution and the concentration gradients of different constituents in the tested sample. In addition to accurate determination of major constituents in the food samples, the advantage of hyperspectral imaging techniques is to display

70

Mohammed Kamruzzaman

the concentration gradients of different constituents in the food sample by more simple visualization. Indeed, the most attractive application of hyperspectral imaging experiments is to map the distribution of the main components on a sample with the aid of image processing to display the hidden information in the image that will qualitatively or quantitatively describe the properties of the tested samples (Ravn et al., 2008). The main aim of hyperspectral image processing was to display the hidden information in each pixel in the image to visualize and map quality attribute from sample to sample and even within the same sample. However, analyzing the substantial amount of data residing in hyperspectral images is a significant issue. A hyperspectral image with spatial dimensions of 320 × 256 pixels collecting 200 wavelengths will generate 81,920 spectra, each with 200 data points. Multivariate image analysis (MIA) techniques are thus required to handle such large amount of data. MIA can be applied for different purposes, such as classification, segmentation, defect detection and prediction. The input of MIA is usually a hypercube, but it is better to select some optimum wavelengths through suitable technique to speed up the MIA process. The success of MIA depends mostly on the quality of the developed model (Manley, 2014). If the multivariate model is not good then misleading classification maps or prediction maps might be obtained. Once the model is optimized and finalized, it can then be applied in a pixel-wise manner for all hyperspectral images to produce simple classification maps or prediction maps that need some post-processing routines to enhance the visualization display.

5.7  DEVELOPMENT OF CLASSIFICATION MAP The ultimate goal to develop classification map is to accurately identify and classify each pixel in the image to a particular class based on their spectral characteristics. By classifying every single pixel in the image, pixel-based classification could be extended to image-based classification to classify the tested sample. Classification enables the recognition of regions with similar spectral characteristics without conducting chemical background determination of these regions. For developing classification map, a qualitative multivariate analysis (e.g., PCA, LDA and PLSDA) is applied to the hyperspectral images. A step-by-step procedure for creating classification map via PCA is shown in Figure 5.4. In this classification map, hyperspectral images at the selected optimal wavelengths were used to speed up the classification process and to reduce the time required for image processing. The hyperspectral image at these fewer bands was unfolded into a two-dimensional (2-D) matrix so that each single-band image became a column vector. Each pixel in the image was then multiplied by the PC loading matrix (P) obtained from PCA of the extracted spectral data. After multiplication, each score vector was folded back to form a 2-D score image with the same dimensions of the single-band image. Since PCA performs a significant data compression, the first three PC score images were combined in one image to provide a pre-classification map, which was further explored with image post-processing to obtain classification map. In the classification map, pixels belonging to the same class (muscle) will appear in the same color as shown in Figure 5.5. To evaluate the results of classification algorithm, sample area can be calculated in terms of number of pixels within the sample. A sample is classified into a certain class if the number of pixels belonging to this class is much higher than the other classes.

Multivariate Analysis and Techniques

λ =237

71

λ6 λ2 λ1

Spectral analysis

Image at six key wavelengths (M×N× 6)

Masked image λ1.....................λ6

λ

Re-folded as images

*

(M×N)

PCs

Unfolding

3-D hypercube (M×N× λ)

Apply masking

PC loadings (6×6)

Scores in a column wise

Unfolded 2-D matrix Multiplication by PC loadings

Six score images (PC1 to PC6)

PC1

PC2

PC3

PC4

PC5

PC6

First three score images Post-processing

Pre-classified image

Classification map

FIGURE 5.4  An example of step-by-step procedures for building classification map through multivariate classification techniques. The procedure starts from 3-D hypercube of a meat steak and ending with its classification map (Kamruzzaman et al., 2011).

5.8  PREDICTION MAP As each pixel in a hyperspectral image has its own spectrum, the concentration of chemical composition/quality attribute can be calculated at each pixel in the sample to generate chemical maps, also called prediction map. However, it is practically impossible to

72

Mohammed Kamruzzaman

FIGURE 5.5  Pseudo RGB image and corresponding classification map of representative muscles with PM in red, ST in green, LD in blue, and fat in yellow, respectively. (a) RGB image and corresponding classification map of PM, (b) RGB image and corresponding classification map of ST and (c) RGB image and corresponding classification map of LD. Pseudo RGB image constructed from calibrated hyperspectral images by combining three wavelengths at 950, 1250 and 1350 nm as a replacement of red, green and blue channels to form a pseudo RGB image. Last column represents the misclassified images (Kamruzzaman et al., 2011).

Multivariate Analysis and Techniques

73

FIGURE 5.6  An example of step-by-step procedures for building prediction map through multivariate regression. The procedure starts from 3-D hypercube of a meat steak and ending with its prediction map (Kamruzzaman et al., 2013b).

measure the precise concentration of composition/quality attribute in every pixel of a sample. Therefore, multivariate quantitative models are used to interpolate these components in all spots of the sample. Multivariate quantitative models such as MLR, PCR and PLSR can be used for prediction maps. If nonlinear regression modeling needs to be addressed, ANN or LS-SVM can be considered. The main steps for creating these prediction maps are depicted in Figure 5.6. At first, 3-D hyperspectral image is reduced to a spectral image in the selected optimum wavelengths. The hyperspectral image at these fewer bands is unfolded into a 2-D matrix so that each single-band image became a column vector. This matrix is multiplied by the regression coefficient vector obtained from the calibration model applied to the selected wavelengths. Finally, the resultant matrix

74

Mohammed Kamruzzaman

FIGURE 5.7  Distribution maps of (a) L*, (b) fat content, (c) water content of lamb mus-

cles and (d) pork adulteration in minced lamb meat (Kamruzzaman et al., 2011, 2013b). is folded back to form a 2-D color image with the same dimensions of the single-band image. After multiplication, the resultant vector is folded back to form a 2-D image in which median filter can be employed to enhance the visual display. The resulting image is usually called chemical image or prediction map, in which the spatial distribution of the predicted attribute is easily interpretable (Figure 5.7).

5.9  SOFTWARE FOR MULTIVARIATE ANALYSIS AND IMAGE PROCESSING No hyperspectral imaging system would be complete without the use of software to gain high performance for acquisition, controlling and multivariate analysis. It is imperative to support the system with necessary software for multivariate data analysis and image processing. Indeed, multivariate analysis and image processing is the corner stone of hyperspectral imaging applications. Various software packages such as Environment for Visualizing Images (ENVI; ITT Visual Information Solutions, Boulder, CO, USA), Unscrambler (CAMO PROCESS AS, Oslo, Norway), and MATLAB® (The Math-Works Inc., Natick, MA, USA) are widely applied to perform multivariate analysis and image processing operations of hyperspectral data. ENVI is a powerful tool for hyperspectral image analysis and provides numerous image processing functions but lacks chemometric

Multivariate Analysis and Techniques

75

functions, meaning that ENVI cannot be used to analyze spectral data for multivariate calibration, which is the corner stone of hyperspectral data analysis. Unscrambler is a widely used chemometric software for multivariate data analyses; however, it cannot be used for hyperspectral image processing. MATLAB enables users to analyze hyperspectral images more flexibly than ENVI and Unscrambler. It has the capability of both image processing and multivariate analyses. MATLAB has the graphics features of 2-D and 3-D plotting functions, 3-D volume visualization functions and tools for interactively creating plots, which are required for the visualization of hyperspectral image data. In addition, MATLAB can be used to develop, test, verify and explore algorithms for online implementation and to create user interface. Besides MATLAB, hyperspectral image processing routines can also be developed using programming languages such as C/C++, IDL, LabVIEW and Visual Basic.

5.10  APPLICATION OF MULTIVARIATE ANALYSIS IN SOME SELECTED APPLICATIONS OF MEAT The applications of hyperspectral imaging are huge. A wide range of studies have been carried out on meat by different research groups throughout the world. However, most of these research studies on meat published by a leading research group headed by Professor Da-Wen Sun. A hyperspectral imaging system in the NIR range (900–1,700 nm) was installed in the Laboratory of Biosytems Engineering, University College Dublin, in 2009 for evaluating quality, safety and authenticity of muscles foods. The system consisted of a spectrograph, a12-bit CCD camera, an illumination unit of two 500  W tungsten halogen lamps, a translation stage operated by stepper motor and a computer installed with SpectralCube data acquisition software which controls the motor speed, exposure time, binning mode, wavelength range and image acquisition. The system scans the sample line by line at a constant speed of 2.8 cm/s. The movement of the translation stage is synchronized with the camera to obtain spectral images with a spatial resolution of 0.58 mm/pixel. Associated with multivariate analysis, this system was used for predicting quality, safety and authenticity of beef, lamb, pork, chicken and turkey (Barbin et al., 2012a,b; ElMasry et al., 2011a,b; Feng et al., 2013a; Iqbal et al., 2013a; Kamruzzaman et al., 2011). These applications are summarized in Table 5.2. Some of these applications are described below. This system was used by Kamruzzaman et al. (2012a,b, 2013a,b) for predicting quality, authenticity, and adulteration detection in lamb. At first, the potential of NIR hyperspectral imaging system (900–1,700 nm) coupled with multivariate analysis was examined for discriminating different muscle types of lamb (Kamruzzaman et al., 2011). Samples from semitendinosus (ST), M. longissimus dorsi (LD) and M. psoas major (PM) of Charollais breed were imaged, and principal components analysis (PCA) was used for dimensionality reduction, wavelength selection and visualizing hyperspectral data. Six optimal wavelengths were selected from the eigenvector plot of PCA and used for discrimination purposes. The authors then used LDA to discriminate lamb muscles and obtained an overall accuracy of 100%. A partial least squares (PLS) regression, as a multivariate calibration method, was used to correlate the NIR reflectance spectra with some selected quality parameters (pH, color values and drip loss) of lamb muscles (Kamruzzaman et al., 2012a). The models performed well for predicting pH, color and drip loss with the coefficient of determination (R 2) of 0.65, 0.91 and 0.77, respectively. In another study, the hyperspectral imaging technique was used for non-destructive prediction of chemical

76

Mohammed Kamruzzaman

TABLE 5.2  Application of Multivariate Analysis in NIR Hyperspectral (900–1,700 nm) Data for Some Selected Applications Species Lamb

Beef

Pork

Chicken

Turkey

Application Muscle discrimination in lamb meat L*a*b* values prediction in lamb meat Prediction of water, fat, and protein in lamb meat Authentication of lamb meat from beef and pork WBSF and sensory tenderness of lamb meat Adulteration detection in lamb meat Determination of WHC in beef Determination of chemical composition in beef Determination of color values, pH, and tenderness Quality classification in pork Determination of pH, color values, and drip loss in pork Determination of chemical composition in pork Determination of microbial spoilage in pork Determination of tenderness in pork Freshness determination in pork Detection of Enterobacteriaceae in chicken Detection of Pseudomonas in chicken Detection of TVC in chicken Quality classification in turkey Prediction of color, pH, and water content in turkey

Methods

References

PCA, LDA PLS

Kamruzzaman et al. (2011) Kamruzzaman et al. (2012a)

PLS PCA, PLS-DA

Kamruzzaman et al., (2012b) Kamruzzaman et al. (2012c)

PLS, PLS-DA

Kamruzzaman et al. (2013a)

PCA, PLS, MLR PCA, PLS PLS

Kamruzzaman et al. (2013b) ElMasry et al. (2011b) ElMasry et al. (2012a)

PLS

ElMasry et al. (2012b)

PCA PLS

Barbin et al. (2012a) Barbin et al. (2012b)

PLS

Barbin et al. (2012c)

PLS

Barbin et al. (2012d)

PLS PLS-DA PLS

Barbin et al. (2013b) Barbin et al. (2013a) Feng et al. (2013)

PLS PLS PCA, LDA PLS

Feng & Sun (2013a) Feng & Sun (2013b) ElMasry et al. (2011a) Iqbal et al. (2013a)

composition in lamb meat originated from different muscles. Multivariate calibration models were built by using PLS regression for predicting water, fat, and protein contents (Kamruzzaman et al., 2012b). The models had good prediction abilities for these chemical constituents with determination coefficient (Rp2) of 0.88, 0.88 and 0.63 with SEP of 0.51%, 0.40% and 0.34%, respectively. The feature wavelengths were identified using regression coefficients resulting from the PLSR analyses. New PLSR models were again created using the feature wavelengths, and finally chemical images were derived by applying the respective regression equations on the spectral image in a pixel-wise manner. The resulting prediction maps provided detailed information on compositional gradient in the tested muscles. The PLSR models were also developed for predicting instrumental tenderness measured as Warner–Bratzler shear force (WBSF) values and sensory tenderness from their corresponding spectral data (Kamruzzaman et al., 2013a). The results

Multivariate Analysis and Techniques

77

demonstrated that NIR hyperspectral imaging combined with PLSR could be utilized to determine lamb tenderness in a reasonable accuracy (Rcv = 0.84 for WBSF and 0.69 for sensory tenderness). Overall, the results confirmed that the spectral data collected from NIR hyperspectral imaging could become an interesting screening tool to quickly categorize lamb steaks as good (i.e., tender) or bad (i.e., tough) based on WBSF values and sensory scores with overall accuracy of about 94.51% and 91%, respectively. The reliability and accuracy of the hyperspectral imaging technique was investigated for identification and authentication of lamb meat from other red meat species. Hyperspectral images were acquired from LD muscle of pork, beef, and lamb and their spectral data were extracted and analyzed by PCA and PLS-DA for recognition and authentication of the tested meat. Six wavelengths were identified as important wavelengths from the second derivative spectra. The resulting wavelengths were used in a pattern recognition algorithms for classification of meat samples with PLS-DA yielding 98.67% overall classification accuracy in the validation sets. The developed classification algorithms were then successfully applied in the independent testing set for the authentication of minced meat. The results showed that the combination of hyperspectral imaging, multivariate analysis and image processing has a great potential as an objective and rapid method for identification and authentication of red meat species. Kamruzzaman et al. (2013b) used PCA to identify the most potential adulterate among pork, kidney, heart, and lung in minced lamb. The score plot of the first two PCs revealed the most potential adulterate, which was pork among others since lamb and lamb mixed with pork clustered very closely. Multivariate calibration model was then developed using PLSR to predict the level of pork adulteration in minced lamb. Good prediction model was obtained using the whole spectral range ­(910–1,700 nm) with a coefficient of determination (Rcv2) of 0.99 and RMSECV of 1.37%. Four important wavelengths were selected using weighted regression coefficients (Bw) and an MLR model was then established using these important wavelengths to predict adulteration. The MLR model resulted in a coefficient of determination (Rcv2) of 0.98 and RMSECV of 1.45%. The developed MLR model was then applied to each pixel in the image to obtain prediction maps to visualize the distribution of adulteration of the tested samples. Currently, hyperspectral imaging systems are used to cover mainly the visible nearinfrared (VNIR) and near-infrared (NIR) regions from 400–1,000 and 900–1,700 nm, respectively. Owing to their respective benefits and drawbacks, both wavelength ranges have been used for meat quality assessment (Barbin et al., 2012a; Qiao et al., 2007). The 400–1,000 nm range is industrially advantageous because of the wide availability and low-cost of CCD detectors compared to the more expensive InGaAs detectors used in the 900–1,700 nm region. Kamruzzaman et al. (2016a,b,c) used a VNIR system in the spectral range of 400–1,000 nm for designing a multispectral system for real-time prediction of red meat color, moisture, and water holding capacity WHC. The hyperspectral images of red meat samples (beef, lamb, and pork) were captured in reflectance mode in a dark room to avoid stray light from the surroundings at a controlled temperature (20°C) and humidity (65%). The main components of this instrument are a 12-bit charge-coupled device (CCD) camera, a line-scanning spectrograph coupled with a C-mount lens with fixed distance of 330 mm from sample surface, an illumination unit with one 50-W tungsten halogen, and one xenon lamp adjusted at a 45° angle to illuminate the camera’s field of view, a translation stage driven by a stepping motor with a user-defined speed, and a computer with data acquisition software. After the correction of hyperspectral images, the segmentation steps were carried out to separate the lean meat in a black homogenous background as illustrated in Figure 5.8. The goal of image segmentation was to isolate the

78

Mohammed Kamruzzaman

FIGURE 5.8  Main steps involved in image segmentation to identify ROI: (a) corrected hyperspectral image, (b) a high reflectance image at 625 nm, (c) a low reflectance image at 475 nm, (d) full mask containing the lean and fat parts of the sample, (e) fat mask, and (f) final mask which is used as the ROI resulted from subtraction of full mask and fat mask, i.e. (d, e) (Kamruzzaman et al., 2016a). TABLE 5.3  Application of Multivariate Analysis in VNIR Hyperspectral (400–1,000 nm) Data for Some Selected Applications (Kamruzzaman et al. 2016a,b,c). Calibration Application Moisture prediction in red meat WHC prediction in red meat Color (L*) Color (a*) Color (b*) Color (L*) Color (a*) Color (b*)

Prediction

Model

Rc2

RMSEC (%)

Rp2

PLSR LS-SVM PLSR LS-SVM PLS

0.97 0.99 0.94 0.97 0.97 0.95 0.90 0.99 0.97 0.90

1.57 1.00 0.52 0.36 1.41 1.10 1.01 0.85 0.79 0.99

0.96 0.97 0.92 0.94 0.94 0.91 0.83 0.96 0.95 0.85

LS-SVM

RMSEP (%) RPD 1.67 1.40 0.60 0.50 1.89 1.40 1.37 1.57 1.12 1.09

5.30 6.32 3.63 4.58 4.12 3.79 2.29 4.95 4.73 2.88

lean part from the fat and the background of the image. At first, a binary mask image was created by thresholding the image at 625 nm. This step produced a segmented image for the whole sample including the lean and fat portions of the sample. The main intention of segmentation was to isolate only the lean meat from the background and adjoining fat portion of the sample. Therefore, another image segmentation was performed at 475 nm for detecting fat portions of the sample. Finally, the lean portion was isolated by subtracting fat pixels from the first segmented image containing both lean and fat portions to produce the final mask as an ROI containing only the lean part in a black background. A mean spectrum was then obtained by averaging the spectra of all pixels within the ROI at each wavelength. After extracting spectral data, the authors used both linear (PLSR and MLR) and non-linear (LS-SVM) regression to predict these attributes from the spectral data. The performance of these multivariate models for different applications is summarized in Table 5.3. It is clearly seen that the model developed by LS-SVM had better performance than that of PLSR in predicting quality attributes in red meat in the spectral range of 400–1,000 nm. It is well known that variation in reference data will affect the prediction of quality parameters and also developed models. In those studies, a wide variation was taken into account by collecting samples from different batches, red meat species and

Multivariate Analysis and Techniques

79

geographical locations, meaning that the prediction accuracy was relatively insensitive to unknown changes of external factors (Nicolai et al., 2007). Consequently, the developed calibration models were sufficiently robust for practical application. The authors also used this VNIR hyperspectral imaging system and applied PLSR, PCR and MLR for predicting horse, pork and chicken adulteration in minced beef (Kamruzzaman et al., 2015c,d; 2016d). PLSR model developed using the raw spectra was very accurate with an Rp2 of 0.98 and SEP of 2.23% for predicting horse meat adulteration in minced beef (Kamruzzaman et al., 2015c). On the other hand, the authors obtained an Rp2 of 0.97 and RMSEP of 2.45 (w/w) for predicting chicken adulteration in minced beef (Kamruzzaman et al., 2016d). Similar results were reported for predicting pork adulteration in minced beef (Kamruzzaman et al., 2015d).

5.11 CONCLUSION Multivariate data analysis is an obligatory part of hyperspectral imaging technique. Nowadays, multivariate calibration has become an important analytical tool in many applications including meat and meat products. When the hyperspectral data are processed using appropriate multivariate techniques, it is possible to automatically identify the location of features that display specific spectral signatures and to map the chemical composition and ingredient of specific attributes in the form of concentration profiles. Many options are available for multivariate analysis of hyperspectral data. Different results may be obtained depending on the methods used for multivariate analyses. The choice of a particular method depends on the nature of the problem, the size of the dataset, ease of implementation and economic feasibility. Therefore, comparison and testing of different multivariate methods are necessary to select the best one to obtain a suitable method with high prediction accuracy for a particular application. Although hyperspectral imaging technology is currently suffering from some drawbacks, the future development of hyperspectral imaging instruments such as lower purchase costs and improved processing speed along with progress in robust multivariate data analysis techniques will lead this technology to more substantial and have widespread applications in the future.

REFERENCES Ariana, D. P., & Lu, R. (2010). Evaluation of internal defect and surface color of whole pickles using hyperspectral imaging. Journal of Food Engineering, 96, 583–590. Artyushkova, K., & Fulghum, J. E. (2001). Identification of chemical components in XPS spectra & images using multivariate statistical analysis methods. Journal of Electron Spectroscopy & Related Phenomena, 121, 33–55. Barbin, D., ElMasry, G., Sun, D.-W., & Allen, P. (2012a). Near-infrared hyperspectral imaging for grading and classification of pork. Meat Science, 90, 259–268. Barbin, D., ElMasry, G., Sun, D.-W., & Allen, P. (2012b). Predicting quality and sensory attributes of pork using near-infrared hyperspectral imaging. Analytica Chimica Acta, 719, 30–42. Barbin, D. F., ElMasry, G., Sun, D.-W., & Allen, P. (2012c). Non-destructive determination of chemical composition in intact and minced pork by near-infrared hyperspectral imaging. Food Chemistry, 138, 1162–1171.

80

Mohammed Kamruzzaman

Barbin, D. F., ElMasry, G., Sun, D.-W., Allen, P., & Noha, M. (2012d). Non-destructive assessment of microbial contamination in porcine meat using NIR hyperspectral imaging. Innovative Food Science & Emerging Technologies, 17, 180–191. Barbin, D. F., Sun, D.-W., & Su, C. (2013a). NIR hyperspectral imaging as non-destructive evaluation tool for the recognition of fresh and frozen-thawed porcine longissimus dorsi muscles. Innovative Food Science and Emerging Technologies, 18, 226–236. Barbin, D. F., Valous, N. A., & Sun, D.-W. (2013b). Tenderness prediction in porcine longissimus dorsi muscles using instrumental measurements along with NIR hyperspectral and computer vision imagery. Innovative Food Science and Emerging Technologies, 20, 335–342. Bro, R., van den Berg, F., Thybo, A., Anderseny, C. M., Jøgensenx, B. M., & Andersen, H. (2002). Multivariate data analysis as a tool in advanced quality monitoring in the food production chain. Trends in Food Science & Technology, 13, 235–244. ElMasry, G., Iqbal, A., Sun, D.-W., Allen, P., & Ward, P. (2011a). Quality classification of cooked, sliced turkey hams using NIR hyperspectral imaging system. Journal of Food Engineering, 103(3), 333–344. ElMasry, G., Kamruzzaman, M., Sun, D.-W., & Allen, P. (2012a). Principles and applications of hyperspectral imaging in quality evaluation of agro-food products, a review. Critical Reviews in Food Science & Nutrition, 52, 999–1023. ElMasry, G., Sun, D.-W., & Allen, P. (2011b). Non-destructive determination of waterholding capacity in fresh beef by using NIR hyperspectral imaging. Food Research International, 44, 2624–2633. ElMasry, G., Sun, D.-W., & Allen, P. (2012b). Near-infrared hyperspectral imaging for predicting colour, pH and tenderness of fresh beef. Journal of Food Engineering, 110, 127–140. ElMasry, G., Sun, D.-W., & Allen, P. (2012c). Chemical-free assessment and mapping of major constituents in beef using hyperspectral imaging. Journal of Food Engineering, 117, 235–246. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27, 861–874. Feng, Y.-Z., ElMasry, G., Sun, D.-W., Scannell, A. G. M., Walsh, D., & Morcy, N. (2013). Near-infrared hyperspectral imaging and partial least squares regression for rapid and reagentless determination of Enterobacteriaceae on chicken fillets. Food Chemistry, 138, 1829–1836. Feng, Y.-Z., & Sun, D.-W. (2012). Application of hyperspectral imaging in food safety inspection and control, a review. Critical Reviews in Food Science & Nutrition, 52(11), 1039–1058 (in press). Feng, Y.-Z., & Sun, D.-W. (2013a). Near-infrared hyperspectral imaging in tandem with partial least squares regression and genetic algorithm for non-destructive determination and visualization of Pseudomonas loads in chicken fillets. Talanta, 109, 74–83. Feng, Y.-Z., & Sun, D.-W. (2013b). Determination of total viable count (TVC) in chicken breast fillets by near-infrared hyperspectral imaging and spectroscopic transforms. Talanta, 105, 244–249. Forina, M., Lanteri, S., & Casale, M., (2007). Multivariate calibration-a review. Journal of Chromatography A, 1158, 61–93. Geladi, P. (2003). Chemometrics in spectroscopy. Part I. Classical chemometrics. Spectrochimica Acta Part B, 58, 767–782. Goetz, A. F. H., Vane, G., Solomon, J. E., & Rock, B. N. (1985). Imaging spectroscopy for earth remote sensing. Science, 228, 1147–1153.

Multivariate Analysis and Techniques

81

Gowen, A. A., Feng, Y.-Z., Gaston, E., & Valdramidis, V. (2015). Recent applications of hyperspectral imaging in microbiology. Talanta, 137, 43–54. Gowen, A. A., Taghizadeh, M., & O’Donnell, C. P. (2009). Identification of mushrooms subjected to freeze damage using hyperspectral imaging. Journal of Food Engineering, 93, 7–12. He, H.-J., Wu, D., & Sun, D.-W. (2013). Non-destructive and rapid analysis of moisture distribution in farmed Atlantic salmon (Salmo salar) fillets using visible and near-infrared hyperspectral imaging. Innovative Food Science and Emerging Technologies, 18, 237–245. Houmøller, L. P., Kristensen, D., & Rosager, H. (2007). Determination of SFC, FFA, and equivalent reaction time for enzymatically interestified oils using NIRS. Talanta, 71, 868–873. Iqbal, A., Sun, D-W., & Allen, P. (2013a). Prediction of moisture, color and pH in cooked, pre-sliced turkey hams by NIR hyperspectral imaging system. Journal of Food Engineering, 117, 42–51. Iqbal, A., Sun, D.-W., & Allen, P. (2013b). An overview on principle, techniques and application of hyperspectral imaging with special reference to ham quality evaluation and control. Food Control, 46, 242–254. Kavdir, I., & Guyer, D. E. (2008). Evaluation of different pattern recognition techniques for apple sorting. Postharvest Technology, 99, 211–219. Kamruzzaman, M. (2016). Food adulteration and authenticity. In: Food Safety-Basic Concepts, Recent Issues, and Future Challenges, (eds.) Jinap, S. & Iqbal, S. Z., Springer International Publishing: Germany, pp. 127–148. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2011). Application of NIR hyperspectral imaging for discrimination of lamb muscles. Journal of Food Engineering, 104, 332–340. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2012a). Prediction of some quality attributes of lamb meat using near infrared hyperspectral imaging and ­multivariate analysis. Analytica Chimica Acta, 714, 57–67. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2012b). Non-destructive prediction and visualization of chemical composition in lamb meat using NIR hyperspectral imaging and multivariate regression. Innovative Food Science and Emerging Technologies, 16, 218–226. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2012c). Potential of hyperspectral imaging and pattern recognition for categorization and authen­ tication of red meat. Innovative Food Science and Emerging Technologies, 16, 316–235. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2013a). Non-destructive assessment of instrumental and sensory tenderness of lamb meat by NIR hyperspectral imaging. Food Chemistry, 141, 389–396. Kamruzzaman, M., Makino, Y., & Oshita, S. (2015b). Non-invasive analytical technology for the detection of contamination, adulteration, and authenticity of meat, poultry, and fish: A review. Analytica Chimica Acta, 853, 19–29. Kamruzzaman, M., Makino, Y., & Oshita, S. (2015d). Hyperspectral imaging in tandem with multivariate analysis and image processing for non-invasive detection and visualization of pork adulteration in minced beef. Analytical Methods, 7, 7496–7502. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016a). Online monitoring of red meat color using hyperspectral imaging. Meat Science, 116, 110–117.

82

Mohammed Kamruzzaman

Kamruzzaman, M., Makino, Y., & Oshita, S. (2016b). Hyperspectral imaging for realtime monitoring of water holding capacity in red meat. LWT-Food Science and Technology, 66, 685–691. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016c). Parsimonious model development for real-time monitoring of moisture in red meat using hyperspectral imaging. Food Chemistry, 196, 1084–1091. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016d). Rapid and non-destructive detection of chicken adulteration in minced beef using visible near-infrared hyperspectral imaging and machine learning. Journal of Food Engineering, 170, 8–15. Kamruzzaman, M., Makino, Y., Oshita, S., & Liu, S. (2015c). Assessment of visible nearinfrared hyperspectral imaging as a tool for detection of horsemeat adulteration in minced. Food & Bioprocess Technology, 8, 1054–1062. Kamruzzaman, M., Nakauchi, S., & ElMasry, G. (2015a). On-line screening of meat and poultry products using hyperspectral imaging. In: High Throughput Screening for Food Safety Assessment—Biosensor Technologies, Hyperspectral Imaging and Practical Applications, (eds.) Bhunia, A. K., Kim, M. S., & Taitt, C. R., Woodhead Publishing Limited: Cambridge, UK, pp. 425–466. Kamruzzaman, M., & Sun, D.-W. (2016). Introduction to hyperspectral imaging. In: Computer Vision Technology for Food Quality Evaluation, (eds.), Sun, D.-W., Elsevier, Academic press: London, pp. 111–139. Kamruzzaman, M., Sun, D.-W., ElMasry, G., & Allen, P. (2013b). Fast detection and visualization of minced lamb meat adulteration using NIR hyperspectral imaging and multivariate image analysis. Talanta, 103, 130–136. Kamruzzaman, M., Takahama, S., & Dillner, A. M. (2018). Quantification of amine functional groups and their influence on OM/OC in the IMPROVE network. Atmospheric Environment, 172, 124–132. Liu, D., Zeng, X.-A., & Sun, D.-W. (2013). NIR spectroscopy and imaging techniques for evaluation of fish quality-A review. Applied Spectroscopy Reviews, 48, 609–628. Manley, M. (2014).  Near-infrared spectroscopy and hyperspectral imaging:  non-­ destructive analysis of biological materials. Chemical Society Review, 43, 8200–8214. Møller, S. F., Frese, J. V., & Bro, R. (2005). Robust methods for multivariate data a­ nalysis. Journal of Chemometrics, 19, 549–563. Naganathan, G. K., Grimes, l. M., Subbiah, J., Calkins, C. R., Samal, A., & Meyer, G. E. (2008). Visible/near-infrared hyperspectral imaging for beef tenderness prediction. Computers & Electronics in Agriculture, 64, 225–233. Nicolai, B. M., Beullens, K., Bobelyn, E., Peirs, A., Saeys, W., Theron, K. I., & Lammertyn, J. (2007).Nondestructive measurement of fruit and vegetable quality by means of NIR spectroscopy: A review. Postharvest Biology and Technology, 46, 99–118. Nakariyakul, S., & Casasent, D. (2009). Fast feature selection algorithm for poultry skin tumor detection in hyperspectral data. Journal of Food Engineering, 94, 358–365. Park, B., Yoon, S.-C., Windham, W., Lawrence, K., Kim, M., & Chao, K. (2011). Linescan hyperspectral imaging for real-time in-line poultry fecal detection. Sensing & Instrumentation for Food Quality & Safety, 5, 25–32. Pu, H., Sun, D.-W., Ma, J., Liu, D., & Kamruzzaman, M. (2014). Hierarchical variable selection for predicting chemical constituents in lamb meats using hyperspectral imaging. Journal of Food Engineering, 143, 44–52. Pu, H., Xie, A., Sun, D.-W., Kamruzzaman, M., & Ma, J. (2015a). Application of wavelet analysis to spectral data for categorization of lamb muscles. Food & Bioprocess Technology, 8, 1–16.

Multivariate Analysis and Techniques

83

Pu, H.-B., Kamruzzaman, M., Sun, D.-W. (2015b). Selection of feature wavelengths for developing multispectral imaging systems for quality, safety and authenticity of muscle foods—A review. Trends in Food Science and Technology, 45, 86–104. Qiao, J., Ngadi, M. O., Wang, N., Gariepy, C., & Prasher, S. O. (2007). Pork quality and marbling level assessment using a hyperspectral imaging system. Journal of Food Engineering, 83, 10–16. Ravn, C., Skibsted, E., & Bro, R. (2008). Near-infrared chemical imaging (NIR-CI) on pharmaceutical solid dosage forms-comparing common calibration approaches. Journal of Pharmaceutical & Biomedical Analysis, 48, 554–561. Rinnan, A., van den Berg, F., & Engelsen, S. B. (2009). Review of the most common preprocessing techniques for near-infrared spectra. Trends in Analytical Chemistry, 28, 1201–1222. Sone, I., Olsen, R. L., Sivertsen, A. H., Eilertsen, G., & Heia, K. (2012). Classification of fresh Atlantic salmon (Salmo salar L.) fillets stored under different atmospheres by hyperspectral imaging. Journal of Food Engineering, 109, 482–489. Spackman, K. A. (1989). Signal detection theory: Valuable tools for evaluating inductive learning. In: Proceedings of the Sixth International Workshop on Machine Learning, Morgan Kaufman: San Mateo, CA, pp. 160–163. Sun, D.-W. (2010). Hyperspectral Imaging for Food Quality Analysis and Control. Academic Press/Elsevier: San Diego, CA. Suykens, J. A. K., & Vandewalle, J. (1999). Least squares support vector machine classifiers. Neural Process Letter, 9, 293–300. Tao, F., Peng, Y., Li, Y., Chao, K., & Dhakal, S. (2012). Simultaneous determination of tenderness and Escherichia coli contamination of pork using hyperspectral scattering technique. Meat Science, 90, 851–857. Valderrama, P., Braga, J. W. B., & Poppi, R. J. (2007). Variable selection, outlier detection, and figures of merit estimation in a partial least-squares regression multivariate calibration model. A case study for the determination of quality parameters in the alcohol industry by near-infrared spectroscopy. Journal of Agricultural and Food Chemistry, 55, 8331–8338. Wang, W., & Paliwal, J. (2007). Near-infrared spectroscopy and imaging in food quality and safety. Sensing & Instrumentation Food Quality, 1, 193–207. Wu, D., & Sun, D.-W. (2013). Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review – Part I: Fundamentals. Innovative Food Science and Emerging Technologies, 19, 1–14. Wu, D., Sun, D.-W., & He., Y. (2012). Application of long-wave near infrared hyperspectral imaging for measurement of colour distribution in salmon fillet. Innovative Food Science and Emerging Technologies, 16, 361–372.

Chapter

6

Principal Component Analysis Cristina Malegori and Paolo Oliveri University of Genova

CONTENTS 6.1 Introduction: The Importance of Exploratory Methods 85 6.2 PCA as a Projection Method 86 6.3 How to Execute PCA 90 6.3.1 T  he Singular Value Decomposition (SVD) Method 90 6.3.2 T  he Nonlinear Iterative Partial Least Squares (NIPALS) Algorithm 90 6.3.3 A  dvantages and Disadvantages of the Two Approaches 92 6.4 PCA as a Compression Method 92 6.5 Unfolding and Refolding Hyperspectral Data 93 6.6 Approaches for PCA-Based Image Processing 94 6.6.1 I mage-Based Approach 94 6.6.2 O  bject-Based Approach 98 6.6.3 Pixel-Based Approach 98 6.7 PCA-Based Image Texture Analysis 102 6.8 Conclusions 103 Acknowledgement 104 References 104

6.1  INTRODUCTION: THE IMPORTANCE OF EXPLORATORY METHODS In a multivariate system, such as hyperspectral images, it is fundamental to deal with methods that allow exploration of the data structure in a visual way understanding typical characteristics, in terms of sample groupings and variable intercorrelations. Exploratory methods belong to the family of unsupervised strategies in which a priori information about sample identity is neither required nor used for building models (Oliveri & Forina, 2012). Several methods, including cluster analysis (Geladi, 2003), are available in this ­family but principal component analysis (PCA) is, for sure, the most widely employed method (Dorrepaal et al., 2016; Jolliffe, 2002). This method, which originated from the work of K. Pearson (1901), offers an overview of the case under study and allows drawing significant conclusions in the decision making process on the basis of the observed results. To achieve such an overview, PCA works in reducing the complexity of a multivariate system by summing up the principal information onto a reduced number of derived variables, called principal components (PCs). This method allows for visualizing of structures

85

86

Cristina Malegori and Paolo Oliveri

within data in an orthogonal space defined by a few PCs; without such an approach, a v-dimensional space would be needed for representing data described by v variables.

6.2  PCA AS A PROJECTION METHOD PCA is based on the assumption that a high variability is synonymous with a high amount of information (Forina, 2015); the mathematical parameter currently used to quantify variability is variance, defined, for one variable, as: n

∑(x − x)

2

i

s2 =



i =1

n −1

, (6.1)

where n is the total number of values measured for variable x. In a multivariate system, covariance also plays an important role in explaining the joint correlation between pairs of variables (e.g., x1 and x 2) and it is defined as: n

∑ (x

i ,1



s1,2 =

i =1

− x1 ) ⋅ ( xi ,2 − x2 ) n −1

. (6.2)

In a multivariate case (described by v variables), it is possible to define the variance and covariance matrix that summarizes all the information about variability and relations between variables of the dataset under study; it is a symmetric squared matrix that has as many rows and columns as the number of variables (v·v). Elements on the leading diagonal correspond to the variances of each variable while elements outside that diagonal are covariances between pairs of variables (Miller, 2005). PCs are built as linear combinations of the original variables, aimed at maximizing variance and minimizing covariance. As a result, the first PC (PC1) is geometrically found as the direction of maximum variance corresponding to the maximum information within data, and the second PC (PC2) is defined as the direction of maximum residual variance among all the possible directions orthogonal to PC1. Orthogonality means that covariance is null between PC1 and PC2. This procedure is followed until the whole information is explained, with the implication that higher-order PCs are associated with a lower amount of variability/information; all the pairs of PCs constructed in this way are orthogonal by definition and, therefore, zero covariant. To focus on the data structure, it is fundamental that the directions of maximum variance pass through the data centroid, defined as the barycenter of the measured points; to do that, data must be mean-centered with respect to variables, individually considered, by subtracting the mean of the corresponding variables from each measurement (Bro & Smilde, 2003). The maximum ­number of PCs that can be obtained is equal to the minimum value between the number of variables (v) and the number of measurements decreased by one (n−1) in case of the centered data. This number corresponds to the rank of the data matrix X, structured with as many rows as measurements (n) and as many columns as variables (v) (Jolliffe, 2002). A simulated dataset described by two variables (x1 and x 2) is presented in Figure 6.1 to visualize how PCs are built from a geometrical point of view. In more detail, Figure 6.1a shows measured values as scatter points in the orthogonal space defined by

Principal Component Analysis (b)

3 PC1 2.5 2 1.5 α 1 0.5 0 -0.5 -1 -1.5 -2 PC2 -2.5 -3 -3-2.5 -2 -1.5-1-0.5 0 0.5 1 1.5 2 2.5 3

2 1.5 1

Covariance

x2

(a)

0.5

-0.5 -1 -1.5 -2

(c)100

0

pi/4

0

pi/4

pi/2

3pi/4

pi/2

3pi/4

Rotation Angle

pi

(d) 100 α

90

Variance % on PC2

90

Variance % on PC1

α

0

x1

80 70 60 50 40 30 20 10 0

87

80 70 60 50 40 30 20 10

0

pi/4

pi/23

3pi/4

pi

0

α

pi

Rotation Angle

Rotation Angle

FIGURE 6.1  Simulated data points in a bivariate space x1 vs. x2 and new rotated axes (a): PC1 (blue line) and PC2 (green line). Covariance between the two rotating axes (b), variance on PC1 (c) and variance on PC2 (d) are represented as a function of the rotation angle α.

the two original variables (x1 and x 2), inversely correlated. The blue line, laying along the direction in which points are more scattered, corresponds to the lowest-order PC (PC1); this new axis individuates an angle with the original variable x1, indicated with α, which describes the position of PC1 in the original orthogonal space. α corresponds to the position in which the projections of data points onto the new axis have the maximum dispersion, quantified by the highest variance value, as illustrated in Figure 6.1b, where variance is represented as a function of the rotation angle. In the specific case in which the system is described by two original variables, the maximum number of PCs is limited to two. Consequently, taking into consideration the orthogonality constrains, PC2 (green line in Figure 6.1a) is fixed as the axis perpendicular to PC1 within the bi-dimensional space. Dispersion of data point projections onto PC2 is characterized by a minimum variance value (in a bivariate case), as indicated in Figure 6.1d; the covariance between PC1 and PC2 is null, as shown in Figure 6.1c. Having more than two variables allows building PC2 as the direction of maximum variance not already explained by PC1, among all the possible directions orthogonal to PC1; this approach is repeated iteratively, until the last component is fixed as explained before. In this way, it is possible to resume the informative variability within data in a reduced number of new variables/axes that can be used for simple and powerful graphical representations of data structures. In order to represent the data points in the new orthogonal space defined by a pair of PCs, point coordinates are obtained as orthogonal projections onto the new rotated axes (PCs). These coordinates, commonly referred to as scores, can be computed as a function of the rotation angles; in more detail, scores are calculated by multiplying the original coordinates (onto the original variables) by a rotation matrix containing the cosines of the rotation angles (directions cosines):

cos α

cos β

cos γ

cos δ

, (6.3)

88

Cristina Malegori and Paolo Oliveri

where β is the angle between x 2 and PC1, γ is the angle between x1 and PC2, and δ is the angle between x 2 and PC2, in the bivariate case, as explained in Figure 6.2. Considering the trigonometric relationships among the four angles, the rotation matrix can also be expressed as a function of α:

cos α

sin β

− sin α

cos α

. (6.4)

Therefore, considering three data points, as an example, their scores can be computed as:



S1,PC1

S1,PC2

S2,PC1

S2,PC2

S3,PC1

S3,PC2

=

x1,1

x1,2

x2,1

x2,2

x3,1

x3,2

⋅

cos α − sin α

sin α cos α

,

(6.5)

where the score matrix (S) is obtained by matrix multiplication between the original data matrix and the rotation matrix. The rotation matrix is referred to as loading matrix (L); the term “loading” indicates the importance of each original variable in defining the new ones, considering that all the original variables contribute in the construction of the PCs. The more a variable x is important in defining a given PC, the closer the two axis directions and, therefore, the lower the α angle and the highest the cosine value. Similar to cosine values, loadings are dimensionless and can assume values ranging from −1 to 1. The loading matrix is a square orthogonal matrix, meaning that its transposed and its inverse matrices are equal; pre-multiplying L by its transposed matrix, an identity matrix (i.e., a diagonal squared matrix with unitary values on the leading diagonal) is obtained. The multiplication applied to obtain scores can be expressed in the matrix notation form as: Sn,p = X n,v ⋅ L v ,p . (6.6)

3

PC2

x2

2.5 2

γ

1.5

β

1 0.5

α

δ

0

x1

-0.5 -1 -1.5 -2

PC1

-2.5 -3

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

FIGURE 6.2  Simulated data points in a bivariate space x1 vs. x 2 and new rotated axes: PC1 and PC2. β indicates the angle between x 2 and PC1, γ indicates the angle between x1 and PC2 and δ indicates the angle between x 2 and PC2.

Principal Component Analysis

89

In other words, given a data matrix X n,v (with n measurements and v variables), the PCs (Sn,p) can be calculated as linear combinations of the original variables. As an example, the score of a single data point (i) on PC1 can be expressed as:

si ,PC1 = xi ,1 ⋅ l1,PC1 + xi ,2 ⋅ l2,PC1 + xi ,3 ⋅ l3,PC1 + .... + xi ,v ⋅ lv ,PC1 = x i ,v ⋅ lv ,PC1. (6.7)

The construction of PC1 scores, generalized for all the n measurements, can be expressed in the matrix notation as:

sn,1 = x n,v ⋅ lv ,PC1. (6.8)

The computation is analogous for all the components. Notice that components are directions of maximum variance. Therefore, different algorithms can find the same directions but opposite orientations; in this case, scores and loadings axes may be overturned but the joint interpretation does not change. Scatter representation of the score values of each data points in the orthogonal space described by a couple of informative (low-order) PCs, called score plot, allows evaluating the distribution of data, highlighting similarities and differences among samples. In such a way, it is possible to identify outliers, trends and groupings or the occurrence of particular regularities and distributions among samples. Another fundamental output of PCA is the loading plot, in which loading values of each variables can be represented either as scatter point in the PC bi-dimensional space (Sciutto et al., 2012) or as a profile against original variables (Giovenzana et al., 2014); the two types of loading plots allow understanding the contribution of each original variable in constructing the components. The scatter plot allows to evaluate the type of correlation (positive or negative) between couples of variables: variables located in close positions of the orthogonal space are positively correlated; the closer the variables, the higher the correlation. Conversely, variables located in opposite quarters are negatively correlated, while the ones located orthogonally with respect to the axes origin are poorly correlated. Conclusions about correlation that can be drawn from a scatter loading plot are limited to the fraction of information represented in the plot, assessable by the percentage of cumulative explained variance (Kjeldahl & Bro, 2010). Inferences about correlation are more reliable if the explained variance of the component is relevant. The loading profile representation is particularly useful when original variables are continuous signals (e.g., in the case of spectroscopic data), because the loading profile can be directly compared with the original signals, allowing a straightforward interpretation of variables. Furthermore, the interpretation of a loading scatter plot for hundreds of ­original variables would be rather complicated. A further possibility to combine the information coming from the two ­representations (score plot and loading scatter plot) on the same chart is the so called biplot, given by the simultaneous representation of samples and variables in the same orthogonal space, with the aim of understanding relationships between them. In more detail, the position of the samples is ascribable to the variables that are located in the same area of the plot. It is important to notice that samples located in opposite areas with respect to certain variables are characterized by low values of these variables. For merging the two plots, characterized by axes that differ in terms of both meaning and scale, several possibilities are available including forcing the respective origins to coincide and range scaling of the axes before overlapping the two spaces (Kjeldahl & Bro, 2010). The same conclusion could be drawn by visually comparing the two separate plots; this is the preferred way

90

Cristina Malegori and Paolo Oliveri

in case of hyperspectral data, in which the combined visualization could be onerous and difficult to be interpreted (Pirro et al., 2012).

6.3  HOW TO EXECUTE PCA 6.3.1 The Singular Value Decomposition (SVD) Method Remembering that PCA is based on the assumption that a high variability is s­ ynonymous with a high amount of information and that covariance matrix resumes information regarding data variability of a given data matrix X (Section 6.2), one of the most important approaches for computing PCs starts from the diagonalization of the covariance matrix. Diagonalizing means transforming a square matrix into a diagonal matrix, which has non-zero elements only along its leading diagonal. When a covariance matrix is diagonalized, all the covariance terms become zeros while the leading diagonal ­contains variance values, referred to eigenvalues. The sum of these eigenvalues is the so-called trace of the diagonalized matrix and corresponds to the total variance of the data. SVD allows diagonalizing the covariance matrix as follows:

C = L ⋅ V ⋅ L′, (6.9)

where C is the covariance matrix of the data matrix X, L is the loading matrix (eigenvectors of the covariance matrix) and V is the diagonalized matrix that corresponds to the covariance matrix of PCs (Wall et al., 2003). In more detail, elements on the leading diagonal correspond to the variances of each new variable (PC) and decrease along the diagonal. The presence of zero elements outside the diagonal indicates that all the covariances between pairs of PCs are null and, therefore, that the new variables are uncorrelated/orthogonal. To obtain the coordinates of the data points in the new space, scores are computed by multiplying the original data matrix (X) with the loading matrix (L), as schematized in equation 6.6. 6.3.2 The Nonlinear Iterative Partial Least Squares (NIPALS) Algorithm Another important algorithm used to perform PCA is NIPALS, which computes PCs one by one, starting from the lowest-order ones, in an iterative way, constructing subsequent approximation of the components until reaching a predefined convergence criterion (Wold et al., 1987). The first step is to approximate the lowest order PC (PC1) as the direction that ­connects the axes origin to one of the data points (A) either randomly chosen (Figure 6.3a) or individuated as the point with the largest distance from the origin (data centroid for mean-centered data). According to the direction of this new variable, loadings and scores of the first approximation of PC1 are computed. In more detail, loadings are obtained by dividing coordinates of the chosen point (A) by their Euclidean norm (square root of the sum of square of the coordinates); in this way, the coordinates become the direction cosines of PC1 first approximation. After that, as many regression models (through the origin) as the number of variables are built; in each model, scores on PC1 are regarded as

Principal Component Analysis

91

the independent variables, while one variable at a time is the dependent one (Figures 6.3b and c). Angular coefficients of these models are then normalized by their Euclidean norm obtaining a new estimation of PC1 loadings that individuates a new direction in the orthogonal space; using such loadings, a second approximation of scores is computed (Figure 6.3d). This approach continues iteratively until reaching convergence: the comparison between subsequent approximations of scores is used as the convergent criterion. In more detail, to stop the iteration for the individuation of the component, the difference between the sums of squares of the two subsequent score series has to be lower than a predefined threshold (usually equal to 10 −7).

(a)

(b)

25

15

20 10

15 10

5

0

x1

x2

5

A

0

-5 -5

-10 -15

-10 -20 -25

(c)

-20

-10

10

0

x1

-15 -15

20

-10

-5

0

5

10

15

Score PC1 (d)

15

25 20

10

15 10

5

x2

x2

5 0

0

-5 -5

-10 -15

-10

-20 -15 -15

-10

-5

0

Score PC1

5

10

15

-25

-20

-10

0

x1

10

20

FIGURE 6.3  Simulated data points in a bivariate space x1 vs. x 2 . First NIPALS a­ pproximation of PC1 (full line) as the direction connecting the axes origin to point A (a) and projected scores (small dots along the full line); marginal regressions of both x1 and x 2 vs. PC1 scores, respectively (b, c); second (full line) and first (dashed line) NIPALS ­approximations of PC1 and projected scores (small dots along the full line).

92

Cristina Malegori and Paolo Oliveri

The individuation of the second PC (PC2) is achieved following an identical scheme starting from the residual matrix (instead of the original data matrix) obtained as follows:

E n,v = X n,v – s n,PC1 ⋅ l PC1,v , (6.10)

where E is the residual matrix, X is the original data matrix and s and l are the score and loading vectors of the PC1, respectively. To calculate the explained variance of a given PC as percentage of the whole ­variability, the sum of squares of the scores on that PC has to be divided by the sum of squares of the original data matrix X. 6.3.3 Advantages and Disadvantages of the Two Approaches SVD works on the covariance matrix while the NIPALS algorithm is executed directly on the original data matrix; consequently, with a huge amount of data points (as it usually occurs for hyperspectral images), NIPALS iterations can be onerous and time consuming. Conversely, the SVD algorithm itself is much more rapid; the limitation in this case is given by the preliminary computation of the covariance matrix, which can be computationally heavy when the number of data points is very big (more than 100,000). To overcome this limitation, it is possible to compute the covariance matrix in an alternative way (Barros & Rutledge, 2005; Burger & Gowen, 2011; Geladi et al., 1989): the whole data matrix X is divided into sub-matrices, along the row dimension. Then, on each data sub-matrix, a partial covariance matrix is computed obtaining a covariance sub-matrix; at the end, the global covariance matrix is computed by summing all the covariance sub-matrices. This approach allows to perform SVD also on hyperspectral data. An advantage of the NIPALS algorithm is the possibility to work with missing data and, consequently, to estimate such values; this property can be exploited to deal with the presence of dead pixels or spike points (after removal) within images (Nelson et al., 1996). A warning on such an approach must be underlined: results are reliable only if few missing data are present in the original data matrix and if solid correlation structures characterize variables.

6.4  PCA AS A COMPRESSION METHOD It is possible to rebuild the original data matrix by multiplying the score matrix by the loading one:

X n,v = Sn,p ⋅ LTp,v . (6.11)

Such a reconstruction is complete, obtaining a reconstructed matrix identical to the ­original one, when all the new variables are included in both the score and loading ­matrices (p = max PCs).

X*n,v = X n,v = Sn,p ⋅ LTp,v . (6.12)

Principal Component Analysis

93

Conversely, if a reduced number of components (p < max PCs) is retained, the r­ econstructed dataset differs from the original one for a certain amount of variability considered as error (E):

X*n,v = Sn,p ⋅ LTp,v (6.13)



X n,v = Sn,p ⋅ LTp,v + En,v . (6.14)

This approach is called abstract factor analysis (AFA) (Jolliffe, 2002) and allows filtering out of the non-informative part of the dataset, comprising only the relevant information. The reconstructed data matrix (X*) is therefore cleansed from spurious variability (such as noise) and can be submitted to further data processing. Additionally, PC scores can be used as the input matrix for a wide number of ­chemometric techniques. This is a very important possibility for algorithms that require the number of data points to be larger than the number of variables, such as linear ­discriminant analysis (LDA) (Kamruzzaman et al., 2011) and quadratic discriminant analysis (QDA) (Wu et al., 1996). Furthermore, using PC scores as the input matrix, with the help of orthogonality between PCs, is an advantage for methods that are ­negatively influenced by the presence of strong correlation structures within variables. For example, application of ordinary least squares regression (OLS) when original input variables are highly inter-correlated, as in the case for spectral data, would lead to highly unstable models, with unreliable predictions. To overcome this limitation, least squares ­regression models can be built starting from PC scores: the method is referred as principal ­component regression (PCR) (Næs & Martens, 1988). PCA constitutes the first step also for qualitative supervised algorithms, such as the soft independent modeling of class analogy (SIMCA) method, aimed at performing oneclass classification (Oliveri, 2017). Another interesting approach involves computation of scores on a reduced number of PCs, to summarize and evaluate the information obtained from different analytical data sources globally (Casale et al., 2010). To do this, PC scores are concatenated after a suitable scaling and merged in a unique data matrix that is submitted to further chemometric data processing. Such an approach is commonly referred as mid-level data fusion (Borràs et al., 2015).

6.5  UNFOLDING AND REFOLDING HYPERSPECTRAL DATA Hyperspectral data are usually stored in 3D data matrices, often referred as ­hypercubes, in  which rows and columns represent pixel position in the image, while the third ­dimension describes the spectral variables. Since PCA is applicable only on 2D data matrices, ­hypercubes have to be reorganized in a bi-dimensional structure prior to their processing; this procedure, called unfolding (Figure 6.4), can be performed following different schemes all aimed at having pixels as rows of the new matrix and variables as columns (Amigo et al., 2015). A key point of this approach is the possibility of coming back to the original data organization (refolding) by applying the inverse procedure (Figure 6.4). In such a way, the spatial information can be exploited for visualization of PCA outcomes in a simple and incisive way.

94

Cristina Malegori and Paolo Oliveri V variables

UNFOLDING

X·Y pixels

X pixels

REFOLDING

Y pixels

FIGURE 6.4  General scheme of an unfolding and refolding procedure for a 3D data

cube.

6.6  APPROACHES FOR PCA-BASED IMAGE PROCESSING When a hyperspectral image is submitted to multivariate data processing, including PCA, several approaches can be applied, depending not only on the type of data but also on the specific task that has to be achieved. In more detail, it has to be considered if the interesting information has to be looked for at an image level, at an object level or at a pixel level. In the first approach (image based), the focus is on whole images, which are being studied and compared one with respect to the other; narrowing the field of view, with the object-based approach, the focus is on single objects inside images; further narrowing the view, with the pixel-based approach, the focus is on individual pixels studied as single data points. In the following paragraphs, each approach will be described in detail, following the flow sheet reported in Figure 6.5. 6.6.1 Image-Based Approach An approach is defined as image based (or at the image level) when the whole image is taken into consideration as a single sample in the data processing; the processing can be divided into different steps, ending with a final PCA performed on a data matrix in which rows correspond to the single images under study. In this way, the information coming from an image is considered as a whole to characterize global properties; this approach is mainly applied when the focus is on structures among samples, without deeply ­investigating variability within pixels in single images.

Principal Component Analysis

95

Image acquision Background removal Unfolding

X · Y Pixels

Spectral Variables

IMAGE-BASED

PCA

Object idenficaon

PCA

PCA

Score matrix

X · Y Pixels

PC Scores

X · Y Pixels

PC Scores

Concatenaon

PC Scores X · Y Pixels

PIXEL-BASED

OBJECT-BASED

Concatenaon

Feature calculaon

Original data matrix

Score matrix

Brushing

Score matrix

Feature calculaon

Feature matrix

Final PCA

Features Objects

Iamges

Features

Feature matrix

Final PCA

FIGURE 6.5  Flowsheet for the three main approaches for performing PCA on hyper-

spectral data. To follow this approach, each image has to be described by features derived from the spectra recorded. The simplest and most intuitive way to characterize globally a single image is to calculate the average spectrum from all the pixels; this strategy, despite being applied in several studies, significantly underutilizes information embodied in images, both in terms of spatial distribution and spectral variability (Lu, 2007). To keep into consideration this important part of the HSI information, several ­procedures can be deployed starting either from the original data matrix or from the output of a previous data processing, such as PCA. From both these approaches, it is

96

Cristina Malegori and Paolo Oliveri

possible to extract feature profiles and derived indices that can be evaluated again with a multivariate data processing. Such a methodology also allows the incorporation of spatial and spectral variability in the final outcomes, exploiting the biggest advantage of HSI. Some examples of feature profiles can be the frequency distributions of spectral intensities at interesting wavelengths or of scores on informative PCs. These profiles can be used as a continuous signal as an input for further multivariate analysis or for the extraction of derived indices. This last option allows to describe frequency distributions in a synthetic way by computing a reduced number of descriptive statistical parameters: among location indices, means, medians and modes can be calculated, while standard deviations, kurtoses and percentile intervals allow describing dispersion of the frequency distribution. When the calculation of features and/or indices starts from PCs, it is fundamental to define the number of components to be considered. Several methods are available for evaluating the number of significant components (Todeschini, 1997), and also examination of patterns within score images can help in this direction (as explained in more detail in Section 6.7.3). The components rejected by this selection define the residuals from the PC model (E) that allow highlighting, for instance, of anomalous behaviors of a single image from the whole structure (e.g., in the case of outlier). Also on residuals, profiles and indices can be computed. A strategy typically definable as image based is the one referred to as ­hyperspectrograms (Ferrari et al., 2013); following this method, each image is preliminarily submitted to PCA and, afterwards, scores, loadings and residual are concatenated creating one profile for each image. As an example of the image-based approach, a dataset of HSI images for ­monitoring cheese ripening is described. A total number of 20 samples of a short-ripened slicing cheese (Formaggetta, Caseificio Val d’Aveto, Italy) were acquired at ten different ripening stages (two samples for each level) by an SWIR3 hyperspectral camera (Specim Ltd, Finland) working in the 1,000–2,500 nm spectral range. Each image corresponds to the central area (120 × 120 pixels) of a cross-sectioned cheese wheel, including possible holes typically encountered in this type of cheese. After removal of pixels corresponding to the holes, 20 images were merged and PCA was performed on the whole data matrix; afterwards, score images were visually evaluated, revealing that the information associated with modification along time (ripening) is contained in PC1 (Figure 6.6). Therefore, only the lowest-order component was retained and features were ­calculated on PC1 score distribution of each single image (Figure 6.7). In this case study, one location parameter (mean) and several dispersion ­parameters (standard deviation, kurtosis, skewness and percentiles) were computed and used to Time 1

Time 2

Time 3

Time 4

Time 5

Time 6

Time 7

Time 8

Time 9

Time 10

R1

R2

FIGURE 6.6  PC1 score images of 20 Formaggetta cheese specimens: two samples

(R1 and R2) at ten ripening stages (from Time 1 to Time 10).

Principal Component Analysis Time 9 - R1

Frequency

Frequency

Frequency

Time 8 - R1

Time 2 - R2

Time 3 - R2

Time 4 - R2

Time 5 - R2

Time 6 - R2

Time 7 - R2

Time 8 - R2

Time 9 - R2

-2 2 0 PC1 Score

-2 2 0 PC1 Score

-2 2 0 PC1 Score

-2 2 0 PC1 Score

-2 2 0 PC1 Score

-2 2 0 PC1 Score

-2 2 0 PC1 Score

-2 2 0 PC1 Score Time 10 - R2

Frequency

Time 1 - R2

Frequency

-2 2 0 PC1 Score

Frequency

-2 2 0 PC1 Score

Frequency

-2 2 0 PC1 Score

Frequency

-2 2 0 PC1 Score

Frequency

-2 2 0 PC1 Score

Frequency

-2 2 0 PC1 Score

Frequency

-2 2 0 PC1 Score

Frequency

-2 2 0 PC1 Score

Frequency

-2 2 0 PC1 Score

-2 2 0 PC1 Score

Time 10 - R1

Frequency

Time 7 - R1

Time 6 - R1

Frequency

Frequency

Time 5 - R1

Frequency

Time 4 - R1

Time 3 - R1

Frequency

Time 2 - R1

Frequency

Frequency

Time 1 - R1

97

-2 2 0 PC1 Score

-2 2 0 PC1 Score

FIGURE 6.7  Distributions of PC1 scores of each single image of the 20 Formaggetta

cheese specimens: two samples (R1 and R2) at ten ripening stages (from Time 1 to Time 10). characterize each image. Hence, the final data matrix is composed by 20 rows, one for each image/sample, and five columns, one for each parameter calculated. Such a data matrix, after column autoscaling, can be submitted to further data processing; in this example, an exploratory analysis was performed by a final PCA. In Figure 6.8a, PC1 vs. PC2 scores of the final PCA are shown, and it is possible to clearly follow sample evolution along time, especially lengthwise PC1; Figure 6.8b ­highlights the relation between the most informative component and time, revealing an interesting trend. As a conclusion, in this example, it was possible to follow cheese (a) 8

(b) 5

R2-t01 R2-t02 R1-t04 R1-t03 R2-t03 R2-t04

R1-t01

R2-t07

2

R2-t09 R2-t08 R1-t07 R2-t05 R2-t10

1

R1-t06 R2-t06 R1-t05

PC1 Score

Scores on PC2 (27.5%)

R1-t02

0 -2

R1-t10 R1-t09

R1-t08

R1-t08 R1-t07 R2-t05

R1-t06

R1-t05

0 R2-t03

-1 -2

-4

R1-t03 R2-t01

R2-t02

R1-t01

R1-t02

R2-t06

R2-t07

R1-t10

R2-t10 R2-t09

R2-t08

R2-t04 R1-t04

-3

-6 -8

R1-t09

R2

3

4 2

R1

4

6

-4 -5

-4

-3

-2 -1 1 0 Scores on PC1 (63.5%)

2

3

4

5

-5

1

2

3

4

5

6

7

8

9

10

Time

FIGURE 6.8  PC1 vs. PC2 scores of the final image-based PCA (a) and PC1 score

e­ volution vs. time (b). The 20 Formaggetta cheese specimens: two samples (R1 and R2) at ten ripening stages (from Time 1 to Time 10).

98

Cristina Malegori and Paolo Oliveri

ripening, resuming the outcomes in simple charts and confirming that the image-based approach is particularly suitable when samples to be studied correspond to entire images; in particular, this strategy allows to compare images/samples taking into consideration their own spatial variability. 6.6.2 Object-Based Approach An approach is defined as object based (or at the object level) when the focus is neither on the whole image nor on the single pixel but on individual objects of interest contained in the images. In this case, the final matrix submitted to data processing is structured with as many rows as objects. Therefore, the first and key step of the object-based approach is the identification of pixels belonging to a given object. Several procedures have been proposed (Gonzalez & Woods, 2002); all of them include background removal and morphological analysis, aimed at recognizing target shapes in the image. Once pixels constituting the same object have been identified, feature profiles and/or derived indices can be computed—separately for each single object—exactly in the same way described at the image level in Section 6.7.1 (Kucheryavski et al., 2010). As an example of the object-based approach, a total number of 42 beans of green coffee (23 non-defective, 9 black beans and 10 dry beans) were acquired by an SWIR3 hyperspectral camera working in the 1,000–2,500 nm spectral range. Images of coffee batches were first segmented—for background elimination—and, subsequently, submitted to morphological analysis, which allowed individuating clusters of contiguous pixels corresponding to individual coffee beans, as illustrated in Figure 6.9. Instead of considering each pixel separately, all the pixels defining a bean were ­considered as a single object. As in the previous approach, each object was ­characterized by computing relevant descriptive statistical parameters (mean, standard deviation, ­kurtosis, percentiles, residuals) describing the distributions of scores on a selected number of PC (five PCs). Such parameters were computed to describe the dispersion of pixels belonging to each single bean. Therefore, each row of the data matrix represents one of the beans evaluated, while the columns are the statistical parameters calculated for each bean, conveniently autoscaled. This matrix was then submitted to a final PCA whose results are shown in Figure 6.10; these scores allow distinguishing defective from non-defective beans, showing the way forward to a potential automatization of fault identification in green coffee batches. In particular, non-defective beans are grouped in the left-bottom corner (negative values of scores on both PC1 and PC2), while black beans and dry beans are found toward positive values lengthwise PC1 and PC2, respectively. 6.6.3 Pixel-Based Approach An approach is defined as pixel based (or at the pixel level) when the differences among pixels express the useful information; in this case, each pixel is described by its own spectral profile independent of the other pixels in the image. When this approach is followed, PCA is computed on a matrix structured with as many rows as pixels, obtained from the hypercube by unfolding. After PC computation, scores are obtained for each pixel, and can be used both for drawing scatter score plots (in a Cartesian space defined by a couple of PCs) and for reconstructing score images/maps. In these maps, scores of a

Principal Component Analysis

2

1

3

99

4

5

8 7

6

9 11

10

12

13

15

17

16

14 20

19

18

21

24

22

23

25 26

27

29

28

31

30 32 33

33 35 37

36

39

40

38

41

42

FIGURE 6.9  Output of morphological analysis: binary image of the 42 green coffee

samples (white areas). Red numbers identify non-defective beans, while blue and green numbers identify black beans and dry beans, respectively. single PC of interest are refolded in the position corresponding to the related pixel in the image, allowing the visualization of how the image areas are characterized by different intensity of scores. To do this, score values are coded by a given color scale, either gray scale or chromatic (e.g., from blue to red). Depending on the particular case under study, the useful information can be contained in higher order PCs because, in many cases, the lowest order PCs are associated with interfering information such as illumination differences and physical roughness of sample surfaces. To understand if a given PC carries

100

Cristina Malegori and Paolo Oliveri SCORE PLOT non-defective

12

black beans dry beans

10

Scores on PC2 (23.0%)

8 6 4 2 0 -2 -4 0

5

10

15

Scores on PC1 (41.8%)

FIGURE 6.10  PC1 vs. PC2 score scatter plot of the final object-based PCA. Red squares identify non-defective beans, while blue and green squares identify black beans and dry beans, respectively.

meaningfully arranged information, a recognizable pattern inside the score map has to be detected. Conversely, when a given score map shows randomly distributed patterns, this may indicate that the PC under examination is not associated with interesting information. When a score scatter plot is obtained from PCA at the pixel level, usually a huge number of points are represented inside the orthogonal space; in this case, the interest is rarely focused on a single pixel but clouds of pixels scattered along a component have to be identified and studied. To understand the relationships among groupings of points in the score scatter plot and spatial regions in the score map, the so-called brushing procedure can be applied. Typically, brushing is performed by selecting a cluster of points of interest within the scatter plot: the algorithm automatically recognizes the corresponding points (pixels) in the image and graphically highlights the matching region in the image. The procedure can be also performed in the opposite direction, by selecting spatial regions within the score image, and directly highlighting the corresponding points in the score scatter plot. The brushing approach is particularly useful from an exploratory point of view, since it allows not only to understand relationships between areas in the score image and clusters in the score scatter plot, but also to determine the contribution of the spectral variables in characterizing each region in the image. This last achievement is possible by jointly analyzing score maps, score scatter plots and loading scatter plots—in which the importance of the original spectral variables in the definition of two given PCs is shown, as in

Principal Component Analysis

101

any PCA application. Loading values, represented in a scatter way, also give information about inter-correlation among variables—the closer the loading values, the higher the correlation degree between a couple of variables. For a big amount of variables—a t­ ypical case for spectroscopic data—loadings can be represented as intensity profiles against the spectral variables, helping to easily visualize the contribution of each variable in constructing the new components. In fact, such a representation allows establishing a direct parallelism between spectral regions, which is revealed as important by loading values and original absorption bands in spectral profiles. As an example of the pixel-based approach, in a single image (presented as total intensity image in Figure 6.11), a total number of eight acerola fruits (a tropical fruit from Brazil; Malegori et al., 2016; Malegori et al., 2017) were evaluated by the Sisuchema ­system (Specim Ltd, Finland) working between 1,000 and 2,500 nm spectral range. The aim of this study was to reveal the presence of mold sporocarps; in this case, only specific areas within fruits can be contaminated. For this reason, it is important to characterize single pixels constituting each image/object. PCA was applied at the pixel level, after background removal, on an image containing four non-contaminated fruits— on the left—and four contaminated fruits—on the right. With the help of the brushing procedure (as shown in Figure 6.12), it is possible to highlight, in the orthogonal space PC1 vs PC2, that pixels ascribable to mold sporocarps are well grouped in the same area of the scatter plot.

FIGURE 6.11  RGB image of acerola fruits: four non-defective fruits (left) and five fruits

contaminated by molds (right).

102

Cristina Malegori and Paolo Oliveri

FIGURE 6.12  (a) PC1 vs. PC2 score scatter plot of the pixel-based PCA. (b) RGB image of

­acerola fruits: four non-defective fruits (left) and five fruits contaminated by molds (right). Combining such information with loading analysis allows understanding the ­spectroscopic bases of mold identification.

6.7  PCA-BASED IMAGE TEXTURE ANALYSIS Texture analysis refers to the characterization of regions in an image by their texture content and attempts to quantify intuitive qualities described by terms such as rough, smooth, silky or bumpy as a function of the spatial variation in pixel intensities. In fact, considering a given material, a smooth surface reflects the incident radiation in a regular way, obtaining a uniform distribution of pixel intensities; conversely, a rough surface scatters light in several directions resulting in a difference between the intensity of each pixel compared with its neighbor pixels (Figure 6.13).

242

64

217

128

89

0

89

64

191

191 128 128 128

89

128

128 128 128

FIGURE 6.13  Incidence radiation reflected by a rough surface (left) and by a smooth

surface (right), and corresponding simulated 3 × 3 ROIs of grey-scale pixel intensities.

Principal Component Analysis

103

Image texture algorithms are traditionally conceived for gray-scale images and their basic steps include the definition of a neighborhood around the pixel of interest and the calculation of statistics for that neighborhood. The most famous algorithms applied for image texture purposes are: Gray Level Co-occurrence Matrix (GLCM) (Haralick et al., 1973; Malegori et al., 2016) and Angle Measure Technique (AMT) (Andrle, 1994; Fongaro & Kvaal, 2013). Such methods could also be applied on hypercubes, either on a derived image at a single channel or on the total intensity image; in this way, a huge amount of information would be lost. Bharati and MacGregor (1998) proposed a method that overcomes this limitation and allows keeping all the data information into consideration, for simultaneously evaluating texture and spectral properties (Prats-Montalbán & Ferrer, 2014). This algorithm constructs a new unfolded matrix in which each pixel (on rows) is described by its entire spectrum concatenated with spectra of its surrounding pixels; if neighborhood is defined at one-pixel distance, the number of surrounding pixels is equal to 8. In this way, the number of variables of the new data matrix is increased by a factor eight (e.g., starting from 100 original variables, the new data matrix will be composed by 900 columns). In this way, each pixel is described by its own information besides the one coming from neighborhood, adding spatial information to the system. Such a complex matrix is submitted to PCA summarizing in the higher order PCs the variability associated with texture features. The resulting score maps visually reveal differences in pixel intensities ascribable to roughness/smoothness of the sample surface. To better understand this innovative approach and as an example of PCA-based image texture analysis, hyperspectral images—acquired by an SWIR3 hyperspectral camera working in the 1,000–2,500 nm spectral range—of five different flours were submitted to the PCA-based image texture analysis. These flours—durum wheat flour, strong flour (Manitoba), wholemeal, plain flour, soft flour—differ not only for wheat variety but also for granulometry, which give rise to differences in terms of image texture characteristics. When texture has to be studied it is fundamental that sample surface is prepared to avoid interferences due to rough physical irregularities that can mask the actual texture properties. After the selection of suitable regions of interest (ROI with dimensions equal to 50 × 50), a preliminary PCA was performed revealing that the four lowest order PCs group exclusively the durum wheat flour, the only sample from Triticum durum Desf. wheat. If this sample is removed from the dataset, differences among the other samples, all from Triticum aestivum L., were detectable lengthwise in PC3. This behavior is due to the fact that the standard PCA approach takes into consideration only the spectral information, which accounts for chemical differences between wheat varieties. PCA-based image texture analysis allows combining such an information with the spatial information related to neighborhood of each pixel; in the outcomes, lowest-order components are still associated with chemical differences, while for higher-order PCs, the pattern ­highlighted in score images is ascribable to textural feature, as shown in Figure 6.14.

6.8 CONCLUSIONS This chapter allows a deep understanding of the correct approaches for applying exploratory PCA on hyperspectral data, which is a fundamental step for every multivariate data processing protocol. Starting from the understanding of the basic principles of the method and of the most widespread algorithm, the suitable procedure for performing

104

Cristina Malegori and Paolo Oliveri Durum

Strong

Wholemeal Plain flour

So flour

PC 1

PC 2

PC 3

PC 4

PC 5

PC 6

FIGURE 6.14  PC1 to PC6 score images of five different typologies of flours after PCA-

based image texture analysis. PCA is presented. The core aspect is the need to choose the right strategy according to a specific aim, starting from the whole image consideration, passing through a focus on single objects, arriving a single pixel evaluation.

ACKNOWLEDGEMENT Financial support by the Italian Ministry of Education, Universities and Research (MIUR) is acknowledged—Research Project SIR 2014 “Advanced strategies in near infrared spectroscopy and multivariate data analysis for food safety and authentication,” RBSI14CJHJ (CUP: D32I15000150008).

REFERENCES Amigo, J. M., Babamoradi, H., & Elcoroaristizabal, S. (2015). Hyperspectral image a­ nalysis. A tutorial. Analytica Chimica Acta, 896, 34–51. doi:10.1016/j.aca.2015.09.030. Andrle, R. (1994). The angle measure technique: A new method for characterizing the complexity of geomorphic lines. Mathematical Geology, 26(1), 83–97. doi:10.1007/ BF02065877. Barros, A. S., & Rutledge, D. N. (2005). Segmented principal component transformprincipal component analysis. Chemometrics and Intelligent Laboratory Systems, 78(1), 125–137. doi:10.1016/j.chemolab.2005.01.003.

Principal Component Analysis

105

Bharati, M. H., & MacGregor, J. F. (1998). Multivariate image analysis for real-time ­process monitoring and control. Industrial & Engineering Chemistry Research, 37(12), 4715–4724. doi:10.1021/ie980334l. Borràs, E., Ferré, J., Boqué, R., Mestres, M., Aceña, L., & Busto, O. (2015). Data fusion methodologies for food and beverage authentication and quality assessment – A review. Analytica Chimica Acta, 891, 1–14. doi:10.1016/j.aca.2015.04.042. Bro, R., & Smilde, A. K. (2003). Centering and scaling in component analysis. Journal of Chemometrics, 17, 16–33. doi:10.1002/cem.773. Burger, J., & Gowen, A. (2011). Data handling in hyperspectral image analysis. Chemometrics and Intelligent Laboratory Systems, 108(1), 13–22. doi:10.1016/j. chemolab.2011.04.001. Casale, M., Casolino, C., Oliveri, P., & Forina, M. (2010). The potential of coupling information using three analytical techniques for identifying the geographical origin of Liguria extra virgin olive oil. Food Chemistry, 118(1), 163–170. Dorrepaal, R., Malegori, C., & Gowen, A. (2016). Tutorial: Time series hyperspectral image analysis. Journal of Near Infrared Spectroscopy, 24(2), 89. doi:10.1255/ jnirs.1208. Ferrari, C., Foca, G., & Ulrici, A. (2013). Handling large datasets of hyperspectral images: Reducing data size without loss of useful information. Analytica Chimica Acta, 802, 29–39. doi:10.1016/j.aca.2013.10.009. Fongaro, L., & Kvaal, K. (2013). Surface texture characterization of an Italian pasta by means of univariate and multivariate feature extraction from their texture images. Food Research International, 51(2), 693–705. doi:10.1016/j. foodres.2013.01.044. Forina, M. (2015). Fifty years of chemometrics, fifty years with chemometrics. ResearchGate. doi:10.13140/RG.2.1.2199.3445. Geladi, P. (2003, May 30). Chemometrics in spectroscopy. Part 1. Classical chemometrics. Spectrochimica Acta - Part B Atomic Spectroscopy, 58(5), 767–782. doi:10.1016/ S0584-8547(03)00037-5. Geladi, P., Isaksson, H., Lindqvist, L., Wold, S., & Esbensen, K. (1989). Principal ­component analysis of multivariate images. Chemometrics and Intelligent Laboratory Systems, 5(3), 209. doi:10.1016/0169–7439(89)80049-8. Giovenzana, V., Beghi, R., Malegori, C., Civelli, R., & Guidetti, R. (2014). Wavelength selection with a view to a simplified handheld optical system to estimate grape ripeness. American Journal of Enology and Viticulture, 65(1), 117–123. Gonzalez, R., & Woods, R. (2002). Digital Image Processing. Upper Saddle River, NJ: Prentice Hall. doi:10.1016/0734-189X(90)90171-Q. Haralick, R. M., Shanmugam, K., & Dinstein, I. (1973). Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-3(6), 610–621. doi:10.1109/TSMC.1973.4309314. Jolliffe, I. T. (2002). Principal Component Analysis. New York: Springer-Verlag. Kamruzzaman, M., Elmasry, G., Sun, D. W., & Allen, P. (2011). Application of NIR hyperspectral imaging for discrimination of lamb muscles. Journal of Food Engineering, 104(3), 332–340. doi:10.1016/j.jfoodeng.2010.12.024. Kjeldahl, K., & Bro, R. (2010). Some common misunderstandings in chemometrics. Journal of Chemometrics, 24(7–8), 558–564. doi:10.1002/cem.1346. Kucheryavski, S., Esbensen, K. H., & Bogomolov, A. (2010). Monitoring of pellet coating process with image analysis-a feasibility study. Journal of Chemometrics, 24(7–8), 472–480. doi:10.1002/cem.1292.

106

Cristina Malegori and Paolo Oliveri

Lu, R. (2007). Nondestructive measurement of firmness and soluble solids content for apple fruit using hyperspectral scattering images. Sensing and Instrumentation for Food Quality and Safety, 1(1), 19–27. doi:10.1007/s11694-006-9002–9. Malegori, C., Franzetti, L., Guidetti, R., Casiraghi, E., & Rossi, R. (2016). GLCM, an image analysis technique for early detection of biofilm. Journal of Food Engineering, 185, 48–55. doi:10.1016/j.jfoodeng.2016.04.001. Malegori, C., Grassi, S., Marques, E., de Freitas, S., & Casiraghi, E. (2016, Dec). Vitamin C distribution in acerola fruit by near infrared hyperspectral imaging. Journal of Spectral Imaging, 1(5/6), 1–4. doi:10.1255/jsi.2016.a6. Malegori, C., Nascimento Marques, E. J., de Freitas, S. T., Pimentel, M. F., Pasquini, C., & Casiraghi, E. (2017). Comparing the analytical performances of Micro-NIR and FT-NIR spectrometers in the evaluation of acerola fruit quality, using PLS and SVM regression algorithms. Talanta, 165, 112–116. doi:10.1016/j.talanta.2016.12.035 Miller, C. E. (2005). Chemometrics in process analytical chemistry. In Process Analytical Technology: Spectroscopic Tools and Implementation Strategies for the Chemical and Pharmaceutical Industries (pp. 226–328), K. A. Bakeev (ed.), Hoboken, NJ: John Wiley & Sons. doi:10.1002/9780470988459.ch8. Næs, T., & Martens, H. (1988). Principal component regression in NIR ­analysis: Viewpoints, background details and selection of components. Journal of Chemometrics, 2(2), 155–167. doi:10.1002/cem.1180020207. Nelson, P. R. C., Taylor, P. A., & MacGregor, J. F. (1996). Missing data ­methods in PCA and PLS: Score calculations with incomplete observations. Chemometrics and Intelligent Laboratory Systems, 35(1), 45–65. doi:10.1016/ S0169-7439(96)00007-X. Oliveri, P. (2017, Aug 22). Class-modelling in food analytical chemistry: Development, sampling, optimisation and validation issues—A tutorial. Analytica Chimica Acta, 982, 9–19. doi:10.1016/j.aca.2017.05.013. Oliveri, P., & Forina, M. (2012). Data analysis and chemometrics. Chemical Analysis of Food: Techniques and Applications, 25–57. doi:10.1016/ B978-0-12-384862-8.00002-9. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine Series 6, 2(11), 559–572. doi:10.1080/14786440109462720. Pirro, V., Eberlin, L. S., Oliveri, P., & Cooks, R. G. (2012). Interactive hyperspectral approach for exploring and interpreting DESI-MS images of cancerous and normal tissue sections. The Analyst, 137, 2374–2380. doi:10.1039/c2an35122f. Prats-Montalbán, J. M., & Ferrer, A. (2014). Statistical process control based on multivariate image analysis: A new proposal for monitoring and defect detection. Computers & Chemical Engineering, 71, 501–511. doi:10.1016/j.compchemeng.2014.09.014. Sciutto, G., Oliveri, P., Prati, S., Quaranta, M., Bersani, S., & Mazzeo, R. (2012). An advanced multivariate approach for processing X-ray fluorescence spectral and hyperspectral data from non-invasive in situ analyses on painted surfaces. Analytica Chimica Acta, 752, 30–38. doi:10.1016/j.aca.2012.09.035. Todeschini, R. (1997). Data correlation, number of significant principal components and shape of molecules. The K correlation index. Analytica Chimica Acta, 348, 419–430. doi:10.1016/S0003-2670(97)00290-0. Wall, M. E., Rechtsteiner, A., & Rocha, L. M. (2003). Singular value ­decomposition and principal component analysis. In A Practical Approach to Microarray Data Analysis (pp. 91–109). Boston, MA: Kluwer Academic Publishers. doi:10.1007/0-306-47815-3_5.

Principal Component Analysis

107

Wold, S., Esbensen, K., & Geladi, P. (1987). Principal component analysis. Chemometric Intelligent Laboratory Systems, 2, 37–52. doi:10.1002/0470013192.bsa501. Wu, W., Mallet, Y., Walczak, B., Penninckx, W., Massart, D. L., Heuerding, S., & Erni, F. (1996). Comparison of regularized discriminant analysis, linear discriminant analysis and quadratic discriminant analysis, applied to NIR data. Analytica Chimica Acta, 329(3), 257–265. doi:10.1016/0003-2670(96)00142-0.

Chapter

7

Partial Least Squares Regression Leo M.L. Nollet University College Ghent

CONTENTS 7.1 Meat 109 7.2 Bacteria 110 7.3 Fruits 111 7.4 Seeds 112 7.5 Melamine 112 References 112

Partial least squares regression (PLS regression—PLSR) is a statistical method, a multivariate calibration technique that bears some relationship with principal components regression. PLS finds a linear regression model by projecting the predicted variables and the observable variables to a new space. Because both the X and Y data are projected to new spaces, the PLS family of methods are known as bilinear factor models. PLS discriminant analysis (PLS-DA) is a variant used when the Y is categorical. PLS is used to find the fundamental relationship between two matrices (X and Y), that is a latent variable approach to model the covariant structures in these two spaces. A PLS model tries to find the multidimensional direction in the X space that explains the maximum multidimensional variance direction in the Y space. PLS regression is particularly suited when the matrix of predictors has more variables than observations, and when there is multicollinearity among X values. A few examples of hyperspectral imaging and PLSR are given in the next paragraphs. Other applications are given in Chapters 12–17.

7.1 MEAT M. Kamruzzaman et al. [1] investigated the potential of hyperspectral imaging in the near-infrared (NIR) range of 900–1,700 nm for non-destructive prediction of chemical composition of lamb meat. Hyperspectral images were acquired for lamb samples originated from different breeds and different muscles. The mean spectra of the samples were extracted from the hyperspectral images, and multivariate calibration models were built using PLSR for predicting water, fat, and protein contents. The feature wavelengths were identified using regression coefficients resulting from the PLSR analyses. New PLSR models were again created using the feature wavelengths and finally chemical images

109

110

Leo M.L. Nollet

were derived by applying the respective regression equations on the spectral image in a pixel-wise manner. The resulting prediction maps provided detailed information on compositional gradient in the tested muscles. The objective of the study of G.K. Naganathan et al. [2] was to predict beef tenderness by PLS analysis of NIR hyperspectral images. Tenderness is a primary determinant of consumer satisfaction of beefsteaks. A pushbroom hyperspectral imaging system (wavelength range: 900–1,700 nm) with a diffuse-flood lighting system was developed. After imaging, steaks were vacuum-pack-aged and aged until 14 days postmortem. After aging, the samples were cooked and slice shear force (SSF) values were collected as a tenderness reference. After reflectance calibration, a region of interest (ROI) of 150 × 300 pixels at the center of longissimus muscle was selected. PLSR, gray-level textural co-occurrence matrix analysis, and canonical discriminant model were carried out to predict beef tenderness. Hyperspectral imaging system operating in the NIR region (900–1,700 nm) was developed for non-contact measurement of surface color, pH, and tenderness of fresh beef [3]. Hyperspectral images were acquired for beef samples and their spectral signatures were extracted. The real color (expressed as L*a*b*), pH, and tenderness of the same samples were recorded using traditional contact methods and then modeled with their corresponding spectral data using PLSR. The weighted regression coefficients of the resulting PLSR models were used to identify the most important wavelengths and to reduce the high dimensionality of the hyperspectral data. G. ElMasy et al. [4] studied the postmortem nondestructive prediction of water holding capacity (WHC) in fresh beef using NIR hyperspectral imaging. These images were acquired for different beef samples originated from different breeds and different muscles. Both principal component analysis and PLSR models were used to optimize the prediction. The purpose of the studies by M. Kamruzzaman et al. [5, 6] was to develop and test a hyperspectral imaging system (900–1,700 nm) to predict some quality attributes and instrumental and sensory tenderness of lamb meat. WarnerBratzler shear force values and sensory scores by trained panelists were collected as the indicator of instrumental and sensory tenderness, respectively. PLSR models were developed for predicting instrumental and sensory tenderness with reasonable accuracy. The study by P. Talens et al. [7] was carried out to investigate the ability of hyperspectral imaging technique in the NIR spectral region (900–1,700 nm) for the prediction of water and protein contents in Spanish cooked hams. Multivariate analyses using PLSR and PLS-DA were applied to the data to develop statistical models.

7.2 BACTERIA Hyperspectral imaging was used in direct and fast determination of Pseudomonas loads in raw chicken breast fillets [8]. A line-scan hyperspectral imaging system (900–1,700 nm) was employed to obtain sample images. These were further corrected, modified, and processed. The prepared images were correlated with the true Pseudomonas counts of these samples using PLS regression. Different spectral extraction approaches, spectral preprocessing methods, and as wavelength selection schemes were tested. It was demonstrated that hyperspectral imaging is an effective tool for nondestructive measurement of Pseudomonas in raw chicken breast fillets. Y.-Z. Feng [9] applied NIR hyperspectral imaging and PLSR as a rapid and reagentless determination method of Enterobacteriaceae on chicken fillets. The constructed

Partial Least Squares

111

prediction map provided the distribution of Enterobacteriaceae bacteria on chicken fillets, which cannot be achieved by conventional methods. The feasibility of using hyperspectral imaging technology combined with wavelet transform and multiway PLS (N-PLS) algorithm to predict the total viable count (TVC) of spiced beef during storage was investigated [10]. The mean spectral data were extracted from the hyperspectral images and further decomposed in nine levels by daubechies8 (db8) wavelet function to obtain an approximation coefficient and nine detail coefficients. Wavelet coefficients were selected to compose different three-dimensional matrices, and N-PLS algorithm was further integrated to establish the predictive models for detecting TVC in spiced beef. The model using three-dimensional data array and N-PLS algorithm has great potential in the TVC value detection of spiced beef and other meat productions. Time series-hyperspectral imaging (TS-HSI) in visible and NIR region (400–1,700 nm) was performed for a rapid and non-invasive determination of surface TVC of salmon flesh during spoilage process [11]. Reference TVC values were measured using standard plate count method. Two calibration methods were applied: least-squares vector machines (LS-SVM) and PLSR.

7.3 FRUITS Hyperspectral imaging in the NIR region (900–1,700 nm) was used for non-intrusive quality measurements (sweetness and texture) in melons [12]. Sample sweetness and hardness values were recorded using traditional intrusive methods. PLSR, principal component analysis, support vector machine (SVM), and artificial neural network models were created to predict the sweetness and hardness values in melons from the hyperspectral data. Experimental results for the three types of melons show that PLSR produces the most accurate results. To reduce the high dimensionality of the hyperspectral data,  the weighted regression coefficients of the resulting PLSR models were used to identify the most important wavelengths. On the basis of these wavelengths, each image pixel was used to visualize the sweetness and hardness in all the portions of each sample. Hyperspectral imaging in the visible and NIR (400–1,000 nm) regions was tested for nondestructive determination of moisture content (MC), total soluble solids (TSS), and acidity (expressed as pH) in strawberry [13]. The spectral data were analyzed using PLS analysis. Optimal wavelengths were selected using from PLS models. Multiple linear regression (MLR) models were established using only the optimal wavelengths to predict the quality attributes. Moreover, for classifying strawberry based on ripeness stage, a texture analysis was conducted on the images based on grey-level co-occurrence matrix (GLCM). M. Huang and R. Lu [14] investigated the potential of hyperspectral scattering technique for detecting mealy apples. Spectral scattering profiles between 600 nm and 1,000 nm were acquired. The mealiness of the apples was determined by the hardness and juiciness measurements from destructive confined compressed tests. Prediction models for hardness and juiciness were developed using PLSR. The possibility of applying the hyperspectral imaging technique for prediction of some physicochemical and sensory indices of table grapes was checked [15]. Berries were analyzed by a hyperspectral imaging system and analyzed for their pH, total acidity, and soluble solid content (SSC) according to common methods. Quantitative descriptive sensory analysis was performed by a trained panel. A PLSR model was applied in order to find correlations between spectra information and physicochemical indices.

112

Leo M.L. Nollet

Hyperspectral images of intact grapes during ripening were recorded using an NIR hyperspectral imaging system (900–1,700 nm) [16]. Spectral data have been correlated with grape skin total phenolic concentration, sugar concentration, titratable acidity, and pH by modified PLSR (MPLS) using a number of spectral pretreatments and different sets of calibrations. Hyperspectral imaging technique was used for predicting the firmness and SSCs of blueberries [17]. A pushbroom hyperspectral imaging system was used to acquire hyperspectral reflectance images (500–1,000 nm). Prediction models were developed based on PLS method. The relationships of NIR spectral images (1,000–1,700 nm), starch/starch-free patterns visually assessed and RGB color images of apples were studied [18]. PLS-DA technique was used on hyperspectral NIR images to classify single pixels using its NIR reflectance spectrum. This work showed the feasibility of NIR imaging spectroscopy as a tool for apple fruit maturity determination. The quality and maturity stages of banana fruit were studied at three different temperatures (20°C, 25°C, and 30°C) by using hyperspectral imaging technique in the visible and NIR (400–1,000 nm) regions [19]. Quality parameters such as MC, firmness and TSS were determined and correlated with the spectral data. Spectral data were analyzed using PLS, PCA, and MLR.

7.4 SEEDS NIR hyperspectral imaging combined with multivariate data analysis was applied to identify rice seed cultivars [20]. Along with PLS-DA, soft independent modeling of class analogy (SIMCA), K-nearest neighbor algorithm (KNN), and SVM, a novel machine learning algorithm called random forest (RF) was applied. Spectra from 1,039 to 1,612 nm were used as full spectra to build classification models. PLS-DA and KNN models showed over 80% classification accuracy, and SIMCA, SVM, and RF models showed 100% classification accuracy in both the calibration and prediction sets. Twelve optimal wavelengths were selected by weighted regression coefficients of the PLS-DA model. Based on optimal wavelengths, PLS-DA, KNN, SVM, and RF models were built. All optimal wavelength-based models (except PLS-DA) produced classification rates over 80%.

7.5 MELAMINE In the study by J. Lim et al. [21], NIR hyperspectral imaging technique in the spectral range of 990–1,700 nm and combined regression coefficient of PLSR model was used to detect melamine particles in milk powders.

REFERENCES 1. M. Kamruzzaman, G. ElMasry, D.-W. Sun, P. Allen. Non-destructive prediction and visualization of chemical composition in lamb meat using NIR hyperspectral imaging and multivariate regression. Innovative Food Science & Emerging Technologies, 16, 2012, 218–226.

Partial Least Squares

113

2. G.K. Naganathan, L.M. Grimes, J. Subbiah, C.R. Calkins, A. Samal, G.E. Meyer. Visible/near-infrared hyperspectral imaging for beef tenderness prediction. Computers and Electronics in Agriculture, 64(2), 2008, 225–233. 3. G. ElMasry, D.-W. Sun, P. Allen. Near-infrared hyperspectral imaging for predicting colour, pH and tenderness of fresh beef. Journal of Food Engineering, 110(1), 2012, 127–140. 4. G. ElMasry, D.-W. Sun, P. Allen. Non-destructive determination of water-holding capacity in fresh beef by using NIR hyperspectral imaging. Food Research International, 44, 2011, 2624–2633. 5. M. Kamruzzaman, G. ElMasry, D.-W. Sun, P. Allen. Non-destructive assessment of instrumental and sensory tenderness of lamb meat using NIR hyperspectral imaging. Food Chemistry, 141, 2013, 389–396. 6. M. Kamruzzaman, G. ElMasry, D.-W. Sun, P. Allen. Prediction of some quality attributes of lamb meat using near-infrared hyperspectral imaging and multivariate analysis. Analytica Chimica Acta, 714, 2012, 57–67. 7. P. Talens, L. Mora, N. Morsy, D.F. Barbin, G. ElMasry, D.-W. Sun. Prediction of water and protein contents and quality classification of Spanish cooked ham using NIR hyperspectral imaging. Journal of Food Engineering, 117, 2013, 272–280. 8. Y.-Z. Feng, D.-W. Sun. Near-infrared hyperspectral imaging in tandem with partial least squares regression and genetic algorithm for non-destructive determination and visualization of Pseudomonas loads in chicken fillets. Talanta, 109, 2013, 74–83. 9. Y.-Z. Feng, G. ElMasry, D.-W. Sun, A.G.M. Scannell, D. Walsch, N. Morcy. Nearinfrared hyperspectral imaging and partial least squares regression for rapid and reagentless determination of Enterobacteriaceae on chicken fillets. Food Chemistry, 138, 2013, 1829–1836. 10. D. Yang, A. Lu, D. Ren, J. Wang. Detection of total viable count in spiced beef using hyperspectral imaging combined with wavelet transform and multiway partial least squares algorithm. Journal of Food Safety, 38(1), 2018, e12390. 11. D. Wu, D.-W. Sun. Potential of time series-hyperspectral imaging (TS-HIS) for noninvasive determination of microbial spoilage of salmon flesh. Talanta, 111, 2013, 39–46. 12. M. Sun, D. Zhang. How to predict the sugariness and hardness of melons: A nearinfrared hyperspectral imaging method. Food Chemistry, 218, 2017, 413–421. 13. G. ElMasry, N. Wang, A. ElSayed, M. Ngadi. Hyperspectral imaging for nondestructive determination of some quality attributes for strawberry. Journal of Food Engineering, 81(1), 2007, 98–107. 14. M. Huang, R. Lu. Apple mealiness detection using hyperspectral scattering technique. Postharvest Biology and Technology, 58, 2010, 168–175. 15. A. Baiano, C. Terracone, G. Peri, R. Romaniello. Application of hyperspectral imaging for prediction of physico-chemical and sensory characteristics of table grapes. Computers and Electronics in Agriculture, 87, 2012, 142–151. 16. J. Nogales-Bueno, J.M. Hernández-Hierro, F.J. Rodríguez-Pulido, F.J. Heredia. Determination of technological maturity of grapes and total phenolic compounds of grape skins in red and white cultivars during ripening by near infrared hyperspectral image: A preliminary approach. Food Chemistry, 152, 2014, 586–591. 17. G.A. Leiva-Valenzuela, R. Lu, J.M. Aguilera. Prediction of firmness and soluble solids content of blueberries using hyperspectral reflectance imaging. Journal of Food Engineering, 115, 2013, 91–98.

114

Leo M.L. Nollet

18. P. Menesatti, A. Zanella, S. D’Andrea, C. Costa, G. Paglia, F. Pallotino. Supervised multivariate analysis of hyper-spectral NIR images to evaluate the starch index op apples. Food Bioprocess Technology, 2, 2009, 308–314. 19. P. Rajkumar, N. Wang, G. ElMasry, G.S.V. Raghavan, Y. Gariepy. Studies on banan fruit quality and maturity stages using hyperspectral imaging. Journal of Food Engineering, 108, 2012, 194–200. 20. W. Kong, C. Zhang, F. Liu, P. Nie, Y. He. Rice seed cultivar identification using near-infrared hyperspectral imaging and multivariate data analysis. Sensors, 13, 2013, 8916–8927. 21. J. Lim, G. Kim, C. Mo, M.S. Kim, K. Chao, J. Qin, X. Fu, I. Baek, B.-K. Cho. Detection of melamine in milk powders using near-infrared hyperspectral imaging combined with regression of partial least square regression model. Talanta, 151, 2016, 183–191.

Chapter

8

Linear Discriminant Analysis Leo M.L. Nollet University College Ghent

CONTENTS 8.1 Wheat 115 8.2 Fruits and Vegetables 116 References 117 Linear discriminant analysis (LDA) is most commonly used as a dimensionality r­ eduction technique in the pre-processing step for pattern classification and machine learning ­applications. Dimensionality reduction or dimension reduction is the process of reducing the number of random variables under consideration. The goal is to project a dataset onto a lower-dimensional space with good class separability in order avoid overfitting. It tries to find a linear combination of features that characterizes or separates two or more classes of objects or events. In contrast to principal component analysis (PCA), LDA is “supervised” and computes the directions or linear discriminants that will represent the axes that maximize the separation between multiple classes. Often the goal of an LDA is to project a feature space (a dataset of n-dimensional samples) onto a smaller subspace k (where k ≤ n−1) while maintaining the class-­discriminatory information. The terms Fisher’s linear discriminant (FLD) and LDA are often used interchangeably, although they are not exactly the same.

8.1 WHEAT Even though some wheat classes may look similar, their chemical composition and consequently the end-product quality can vary significantly [1]. Visual differentiation of wheat classes suffers from disadvantages such as inconsistency, low throughput, and labor intensiveness. A near-infrared (NIR) hyperspectral imaging system was used to develop classification models to differentiate wheat classes. Wheat bulk samples were scanned in the wavelength region of 960–1,700 nm. The scanned images were used for the differentiation of wheat classes using a statistical classifier, an artificial neural network (ANN) classifier, LDA, and quadratic discriminant analysis (QDA). Wavelet texture analysis was used for classification of eight Western Canadian wheat classes using NIR hyperspectral imaging of bulk samples [2]. Hyperspectral images (HSI) (slices) at 10-nm interval were acquired in the wavelength range of 960–1,700 nm. Based on a stepwise LDA, top 100 features were selected and used for classification of wheat

115

116

Leo M.L. Nollet

classes. Linear and quadratic statistical classifiers and a standard back propagation neural network (BPNN) classifier were used for classification using top 10–100 features. In another approach, principal component score images obtained from hypercubes were used for wavelet analysis and classification. The highest average classification accuracy of eight classes was 99.1% when top 90 features were used for classification in a linear discriminant classifier. The BPNN had the highest average classification accuracy of 92.1% using the top 70 features. Using wavelet features from score images, the PC2 features gave the highest classification accuracy (79.9%). The wavelet texture features of HSI can be used effectively for classification of wheat classes of Western Canada.

8.2  FRUITS AND VEGETABLES J. Gómez-Sanchis et al. [3] analyzed a set of techniques to detect rotten citrus without the use of UV lighting. The techniques used included hyperspectral image acquisition, pre-processing and calibration, feature selection, and segmentation using linear and nonlinear methods for classification of fruits. Different methods such as correlation analysis, mutual information, stepwise, and genetic algorithms based on LDA were studied to select the most relevant bands. Image segmentation relies on the combination of efficient band selection techniques and also on pixel classification methods such as classification and regression trees (CART) and LDA. In the study by G. Polder et al. [4], HSI of several stages of ripeness of tomatoes were recorded and analyzed (450–850 nm). The preprocessed images were analyzed with PCA and LDA. In the study by A.A. Gowen et al. [5], the potential use of hyperspectral imaging for early detection of freeze damage in white button mushrooms was investigated. HSI of mushroom samples were obtained using a pushbroom line-scanning HSI instrument in the wavelength range of 400–1,000 nm. Weight loss and Hunter L-value of mushroom samples were also measured over the storage period. A procedure based on principal components analysis and LDA was developed for classification of both selected spectra and whole mushrooms into undamaged and freeze-damaged groups. Applying this procedure to an independent test set, 100% of whole undamaged mushrooms and 97.9% of freezedamaged samples were correctly classified. Bruising in fruits and its detection is a great concern for food safety to the consumers, and for incurrence of economic losses to industry. The research by M. Nagata et al. [6] was aimed at developing techniques for detection of compression bruises in strawberries (Fragaria × ananassa Duch.) using NIR hyperspectral imaging. Using stepwise LDA, optimal wavelengths of 825 nm and 980 nm were identified. The three judgment methods (LDA, normalized difference, and ANN) had performed equally well, while the normalized difference method was found to be more useful. Early detection of bruises in apples was studied using a system that included hyperspectral cameras equipped with sensors working in the visible and NIR (400– 1,000 nm), short wavelength infrared (1,000–2,500 nm), and thermal imaging camera in ­mid-wavelength infrared (3,500–5,000 nm) ranges [7]. Principal components analysis and minimum noise fraction (MNF) analyses of the images that were captured in particular ranges made it possible to distinguish between areas with defects in the tissues and the sound ones. The fast Fourier analysis of the image sequences after pulse heating of the fruit surface provided additional information not only about the position

Linear Discriminant Analysis

117

of the area of damaged tissue but also about its depth. The comparison of the results obtained with supervised classification methods, including soft independent modeling of class analogy, LDA, and support vector machines confirmed that broad spectrum range (400–5,000 nm) of fruit surface imaging can improve the detection of early bruises with varying depths. The potential of RGB (red, green, blue) digital imaging and hyperspectral imaging (900–1,700 nm) was evaluated for discriminating maturity level in apples under different storage conditions along the shelf life [8]. Segmentation, preprocessing, and partial least squares-discriminant analysis were used for hyperspectral data analysis, while illumination correction, dimensionality reduction, and LDA were used for RGB data analysis. The results showed that hyperspectral discrimination classified different storage regimes better than RGB, with an overall success rate of 95.83%. X. Cheng et al. [9] presented a novel method that combines PCA and FLD methods to show that the hybrid PCA-FLD method maximizes the representation and classification effects on the extracted new feature bands. This method is applied to the detection of chilling injury on cucumbers. Based on tests on different types of samples, results show that this new integrated PCA-FLD method outperforms the PCA and FLD methods when they are used separately for classifications. This method adds a new tool for the multivariate analysis of HSI and can be extended to other hyperspectral imaging applications for the safety and quality inspections in fruits and vegetables. The work by X. Wei et al. illustrated the use of HSI at the wavelengths between 400 and 1,000 nm to classify the ripeness of persimmon fruit [10]. Spectra and images of 192 samples were investigated, which were selected from four ripeness stages (unripe, midripe, ripe, and over-ripe). Three classification models—LDA, soft independence modeling of class analogy, and least squares support vector machines were compared. The best model was LDA, of which the correct classification rate was 95.3%.

REFERENCES 1. S. Mahesh, A. Manickavasagan, D.S. Jayas, J. Paliwal, N.D.G. White. Feasibility of near-infrared hyperspectral imaging to differentiate Canadian wheat classes. Biosystems Engineering, 101(1), 2008, 50–57. 2. R. Choudhary, S. Mahesh, L. Paliwal, D.S. Jayas. Identification of wheat classes using wavelet features from near infrared hyperspectral images of bulk samples. Biosystems Engineering, 102, 2009, 115–127. 3. J. Gómez-Sanchis, L. Gómez-Chova, N. Aleixos, G. Camps-Valls, C. MontesinosHerrero, E. Moltó, J. Blasco. Hyperspectral system for early detection of rottenness caused by Penicilliumdigitatum in mandarins. Journal of Food Engineering, 89(1), 2008, 80–86. 4. G. Polder, G.W.A.M. van der Heijden, I.T. Young. Hyperspectral image analysis for measuring ripeness of tomatoes. 2000ASAE International Meeting, Milwaukee, Wisconsin, 2000. 5. A.A. Gowen, M. Taghizadeh, C.P. O’Donnell. Identification of mushrooms subjected to freeze damage using hyperspectral imaging. Journal of Food Engineering, 93, 2009, 7–12. 6. M. Nagata, J.G. Tallada, T. Kobayashi. Bruise detection using NIR hyperspectral imaging for strawberry (Fragaria x ananassa Duch.). Environmental Control in Biology, 44(2), 2006, 133–142.

118

Leo M.L. Nollet

7. P. Baranowski, W. Mazurek, J. Wozniak, U. Majewska. Detection of early bruises in apples using hyperspectral data and thermal imaging. Journal of Food Engineering, 110, 2012, 345–355. 8. C. Garrido-Novell, D. Pérez-Marin, J.M. Amigo, J. Fernández-Novales.Grading and color evolution of aplles using RGB and hyperspectral imaging vision cameras. Journal of Food Engineering, 113, 2012, 281–288. 9. X. Cheng, Y.R. Cheng, Y. Tao, C.Y. Wang, M.S. Kim, A.M. Lefcourt. A novel integrated PCA and FLD method on hyperspectral image feature extraction for cucumber chilling damage inspection. Transactions of the ASAE, 47(4), 2004, 1313–1320. 10. X. Wei, F. Liu, Z. Qiu, Y. Shao, Y. He. Ripeness classification of astringent persimmon using hyperspectral imaging technique. Food Bioprocess Technology, 7, 2014, 1371–1380.

Chapter

9

Support Vector Machines Leo M.L. Nollet University College Ghent

CONTENTS 9.1 Seafood 119 9.2 Fruits 120 9.3 Seeds 120 References 121 In machine learning, support vector machines (SVMs) are supervised learning models associated with learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one or the other of two categories, an SVM training algorithm builds a model that assigns new examples to one category or the other, making it a non-probabilistic binary linear classifier. An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall.

9.1 SEAFOOD H.J. He et al. [1] studied a rapid and non-destructive prediction of tenderness in fresh farmed salmon fillets using visible and near infrared (VIS–NIR) hyperspectral imaging. Hyperspectral images of tested fillets with different tenderness levels were acquired and their spectral features were extracted in the region of 400–1,720 nm. Two calibration algorithms, namely partial least squares regression (PLSR) and least-square support vector machine (LS-SVM) analyses, were used to correlate the extracted spectra of salmon samples with the reference tenderness values estimated by Warner–Bratzler shear force (WBSF) method to quantitatively predict tenderness of salmon fillets with a good performance. D. Wu et al. [2] studied an online hyperspectral imaging system in the spectral region of 380–1,100 nm to determine the moisture content of prawns at different dehydrated levels. Hyperspectral images of prawns were acquired at different dehydration periods. Spectral data were analyzed using PLSR and LS-SVMs to establish the calibration models. Hyperspectral imaging technique in the spectral wavelength range of 400–1,000 nm was used to determine the total volatile basic nitrogen (TVB-N) contents of grass carp fillets during the frozen storage [3]. The quantitative calibration models were built between

119

120

Leo M.L. Nollet

the spectral data extracted from the hyperspectral images and the reference measured TVB-N values by using PLSR and LS-SVMs. The LS-SVM model using full spectral range had a better performance than the PLSR model for prediction of TVB-N value. The best SPA-LS-SVM model was used to achieve the visualization map of TVB-N content distribution of the tested fish fillet samples. VIS–NIR hyperspectral imaging was investigated as a rapid and nondestructive technique to determine whether fish has been frozen-thawed (F-T) [4]. A total of 108 halibut (Psetta maxima) fillets were studied, including 48 fresh and 60F-T samples. The hyperspectral images of fillets were captured using a pushbroom hyperspectral imaging system in the spectral region of 380 to 1,030 nm. A region of interest (ROI) at the image center was selected, and the average spectral data were generated from the ROI image. Dimension reduction was carried out on the ROI image by principal component analysis (PCA). The first three principal components (PCs) explained over 98% of variances of all spectral bands. Gray-level co-occurrence matrix (GLCM) analysis was implemented on the three PC images to extract 36 textural feature variables in total. LS-SVM classification models were developed to differentiate between fresh and F-T fish based on spectral variables, textural variables, combined spectral, and textural variables. The overall results indicate that VIS–NIR hyperspectral imaging technique is promising for the ­reliable differentiation between fresh and F-T fish.

9.2 FRUITS Bruises are not visible externally owing to the special physical properties of kiwifruit peel. Q. Lü [5] proposed an hyperspectral imaging technique to inspect the hidden bruises on kiwifruit. VIS–NIR (408–1,117 nm) hyperspectral imaging data were collected. Multiple optimal wavelength (682, 723, 744, 810, and 852 nm) images were obtained using PCA on the high-dimensional spectral image data (600–900 nm). The bruise regions were extracted from the component images of the five waveband images using radial basis function (RBF)-SVM classification. Early detection of bruises in apples was studied using a system that included hyperspectral cameras equipped with sensors working in the VIS–NIR (400–1,000 nm), short wavelength infrared (1,000–2,500 nm), and thermal imaging camera in mid-wavelength infrared (3,500–5,000 nm) ranges [6]. PCA and minimum noise fraction (MNF) analyses of the images, captured in particular ranges made it possible to distinguish between areas with defects in the tissues and the sound ones. Fast Fourier analysis of the image sequences after pulse heating of the fruit surface provided additional information not only about the position of the area of damaged tissue but also about its depth. The comparison of the results obtained with supervised classification methods, including soft independent modeling of class analogy, linear discriminant analysis and SVMs confirmed that broad spectrum range (400–5,000 nm) of fruit surface imaging can improve the detection of early bruises with varying depths.

9.3 SEEDS A classification algorithm was developed to differentiate individual fungal infected (Aspergillus niger, Aspergillus glaucus, and Penicillium spp.) and healthy wheat kernels [7]. An NIR reflectance hyperspectral imaging system captured hyperspectral images

Support Vector Machines

121

at 20 wavelengths spaced evenly between 1,000 and 1,600 nm. Four statistical features (mean, variance, skewness, and kurtosis) were extracted from the hyperspectral image data of single kernels at each wavelength. The statistical features at all wavelengths composed the pattern vector of a single kernel. The dimensionality of pattern vectors was reduced by PCA. A multiclass SVM with kernel of RBF was used for classification. Using the statistical features, the wheat kernels infected by A. niger, A. glaucus, and Penicillium spp. and healthy wheat kernels were classified with accuracies of 92.9%, 87.2%, 99.3%, and 100%, respectively. The NIR hyperspectral imaging combined with multivariate data analysis was applied to identify rice seed cultivars [8]. Spectral data were exacted from hyperspectral images. Along with partial least squares discriminant analysis (PLS-DA), soft ­independent modeling of class analogy (SIMCA), K-nearest neighbor (KNN) algorithm, and SVM, a novel machine learning algorithm called random forest (RF) was applied. Spectra from ­ odels. PLS-DA 1,039 nm to 1,612 nm were used as full spectra to build classification m and KNN models showed over 80% classification accuracy, and SIMCA, SVM, and ­ rediction RF models showed 100% classification accuracy in both the calibration and p sets. Twelve optimal wavelengths were selected by weighted regression coefficients of the PLS-DA model. Based on optimal wavelengths, PLS-DA, KNN, SVM, and RF models were built. All optimal wavelength-based models (except PLS-DA) produced ­classification rates over 80%. X. Zhang et al. [9] developed hyperspectral imaging in the VIS and NIR region to discriminate different varieties of commodity maize seeds. First, hyperspectral images of 330 samples of six varieties of maize seeds were acquired in the 380–1,030 nm wavelength range. Second, PCA and kernel principal component analysis (KPCA) were used to explore the internal structure of the spectral data. Third, three optimal wavelengths (523, 579, and 863 nm) were selected by implementing PCA directly on each image. Then, four textural variables including contrast, homogeneity, energy, and correlation were extracted from GLCM of each monochromatic image based on the optimal wavelengths. Finally, several models for maize seeds identification were established by LS-SVM and back propagation neural network (BPNN) using four different combinations of PCs, kernel PCs, and textural features as input variables, respectively. The recognition accuracy achieved in the PCA–GLCM–LS-SVM model (98.89%) was the most satisfactory one. In the study by Z.-Y Liu et al. [10], hyperspectral reflectance of rice panicles was measured at the VIS and NIR regions. The panicles were divided into three groups according to health conditions: healthy panicles, empty panicles caused by Nilaparvata lugens, and panicles infected with Ustilaginoidea virens. Low-order derivative spectra were obtained using different techniques. PCA was performed to obtain the principal component spectra (PCS) of the foregoing derivative and raw spectra to reduce the reflectance spectral dimension. Support vector classification was employed to discriminate the healthy, empty, and infected panicles, with the front three PCS as the independent variables. The results demonstrated that it is feasible to use VIS and NIR spectroscopy to discriminate health conditions of rice panicles.

REFERENCES 1. H.J. He, D. Wu, D.W. Sun. Potential of hyperspectral imaging combined with chemometric analysis for assessing and visualising tenderness distribution in raw farmed salmon fillets. Journal of Food Engineering, 126, 2014, 156–164.

122

Leo M.L. Nollet

2. D. Wu, H. Shi, S. Wang, Y. He, Y. Bao, K. Liu. Rapid prediction of moisture content of dehydrated prawns using online hyperspectral imaging system. Analytica Chimica Acta, 726, 2012, 57–66. 3. J.H. Cheng, D.W. Sun, X.A. Zeng, H.B. Pu. Non-destructive and rapid determination of TVB-N content for freshness evaluation of grass carp (Ctenopharyngodon idella) by hyperspectral imaging. Innovative Food Science & Emerging Technologies, 21, 2014, 179–187. 4. F. Zhu, D. Zhang, Y. He, F. Liu. Application of visible and near infrared hyperspectral imaging to differentiate between fresh and frozen-thawed fish fillets. Food Bioprocess Technology, 6, 2013, 2931–2937. 5. Q. Lü, M.J. Tang, J.R. Cai, J.W. Zhao, S. Vittayapadung. Vis/NIR hyperspectral imaging for detection of hidden bruises on kiwifruits. Czech Journal of Food Sciences, 29, 2011, 595–602. 6. P. Baranowski, W. Mazurek, J. Wozniak, U. Majewska. Detection of early bruises in apples using hyperspectral data and thermal imaging. Journal of Food Engineering, 110, 2012, 345–355. 7. H. Zhang, J. Paliwal, D.S. Jayas, N.D.G. White. Classification of fungal infected wheat kernels using near-infrared reflectance hyperspectral imaging and support vector machine. Transactions of the ASABE, 50(5), 2007, 1779–1785. 8. W. Kong, C. Zhang, F. Liu, P. Nie, Y. He. Rice seed cultivar identification using near-infrared hyperspectral imaging and multivariate data analysis. Sensors, 13, 2013, 8916–8927. 9. X. Zhang, F. Liu, Y. He, X. Li. Application of hyperspectral imaging and chemometric calibrations for variety discrimination of maize seeds. Sensors, 12, 2012, 17234–17246. 10. Z.-Y. Liu, J.-J. Shi, L.-W. Zhang, J.-F. Huang. Discrimination of rice panicles by hyperspectral reflectance data based on principal component analysis and support vector classification. Journal of Zhejiang University Science B, 11(1), 2010, 71–78.

Chapter

10

Decision Trees Leo M.L. Nollet University College Ghent

CONTENTS References 124

Decision tree (DT) is a nonparametric supervised learning method used for classification and regression. DT builds classification or regression models in the form of a tree. It breaks down a dataset into smaller subsets while at the same time an associated DT is incrementally developed. The final result is a tree with decision nodes and leaf nodes. A decision node has two or more branches, and a leaf node represents a classification or decision. The topmost decision node in a tree corresponds to the best predictor called root node. DTs can handle both categorical and numerical data. Tree-based learning algorithms are considered to be one of the best and mostly used supervised learning methods. Tree-based methods are predictive models with high accuracy, stability and ease of interpretation. Valero Valbuena [1] proposed the construction and processing of a new region-based, hierarchical, hyperspectral image representation known as the binary partition tree (BPT). This hierarchical region-based representation can be interpreted as a set of hierarchical regions stored in a tree structure. The BPT succeeds in presenting the following: (i) decomposition of the image in terms of coherent regions and (ii) inclusion of relations of the regions in the scene. Once the BPT is constructed, the fixed tree structure allows implementing efficient and advanced application-dependent techniques on it. Only a few examples of hyperspectral imaging technology and DTs related to food analysis are found in the literature. Apple bruising, as a mechanical damage, occurs due to impact, compression, vibration or abrasion during handling. For sorting and grading systems, the information about how long the bruise exists in the affected fruit can be valuable. Visible and nearinfrared (VNIR) and short wavelength infrared (SWIR) spectral characteristics of sound and bruised apple tissues were analyzed during a two-week period after bruising [2]. Supervised classification methods, including support vector machines, linear logistic regression, neural networks and DTs, were used and compared to check their effectiveness for distinguishing time after bruising with respect to five varieties of apples. The detection system included hyperspectral cameras equipped with sensors working in the VNIR (400–1,000 nm) and SWIR (1,000–2,500 nm) ranges. The results of supervised classification revealed good applicability of hyperspectral imaging in the VNIR and SWIR spectral ranges for detecting the number of days after bruising. The linear logistic

123

124

Leo M.L. Nollet

regression neural networks models were found to be the best classifiers in the majority of models developed. The study by Velásquez et al. [3] aimed to develop a system to classify the marbling of beef using the hyperspectral imaging technology. The Japanese standard classification of the degree of marbling of beef was used as a reference and 12 standards were digitized to obtain the parameters of shape and spatial distribution of marbling of each class. A total of 35 samples of M. longissmus dorsi muscle were scanned by the hyperspectral imaging system at the wavelength of 400–1,000 nm in reflectance mode. The wavelength of 528 nm was selected to segment the sample and the background, and the wavelength of 440 nm was used to classify the samples. Processing algorithms on image, based on DT method, were used in the region of interest obtaining a classification error of 0.08% in the building stage.

REFERENCES 1. S. Valero Valbuena. Arbre de partition binaire: un nouvel outil por le représentation hiérarchique et l’analyse de simages hyperspectrales. Thèse Université de Grenoble, 2011. 2. P. Baranowski, W. Mazurek, J. Pastuszka-Wozniak. Supervised classification of bruised apples with respect to the time after bruising on the basis of hyperspectral imaging data. Postharvest Biology and Technology, 86, 2013, 249–258. 3. L. Velásquez, J.P. Cruz-Tirado, R. Siche, R. Quevedo. An application based on the decision tree to classify the marbling of beef by hyperspectral imaging. Meat Science, 133, 2017, 43–50.

Chapter

11

Artificial Neural Networks and Hyperspectral Images for Quality Control in Foods Luis Condezo-Hoyos Scientific Analyst and Consultant

Wilson Castro Universidad Privada del Norte

CONTENTS 11.1 Introduction 126 11.2 Artificial Neural Networks 126 11.2.1 Neuron and Models 126 11.2.1.1 McCulloch–Pitts Model 127 11.2.1.2 Perceptron Model 127 11.2.2 Architectures or Topology 128 11.2.2.1 Multilayer Feed-Forward Neural Networks 128 11.2.2.2 Radial Basis Function Neural Networks 129 11.2.2.3 Self-Organizing Maps 130 11.3  Applications in Food Quality Control 131 11.3.1 Exploratory Analysis and Classification 132 11.3.1.1 Detection of Defects and Damages 132 11.3.1.2 Detection of Contaminant 132 11.3.2 Regression Analysis 132 11.3.2.1 Analytical Quantitation 132 11.4 An Example in Open Access Software—Octave 132 11.4.1 Texture Analysis 134 11.4.2 Image Acquisition 135 11.4.3 Preprocessing 136 11.4.4 Data Extraction 138 11.4.5 Dimensionality Reduction 139 11.4.6 Artificial Neural Network Modeling 140 Appendices 143 References 151

125

126

Luis Condezo-Hoyos and Wilson Castro

11.1 INTRODUCTION Artificial neural networks (ANNs) are a set of mathematical nonlinear tools that allow to model complex systems. Although ANNs in their beginning were developed to mimic the functioning of human brain, it is from inclusion of nonlinear functions that are ­successfully applied to solve real-problems in different areas such as food science including quality control as well as analytical issues. The development of high-throughput and noninvasive analytical techniques in food science has the challenges of the variable processing related to the huge amount, noise, correlation, and mainly to the fact that functional relation to model is heavily nonlinear. Hyperspectral imaging technique (HIT) has been regarded as a promising analytical tool for food quality control through exploratory, classification, and regression analysis with advantages on traditional methods, such as nondestructive nature, and fast time and intensive data acquisition. HIT is an instrumental tool that integrates spectroscopic and imaging techniques to enable direct identification of different components and their spatial distribution in the tested sample. Spectral spatial data generated by HIT require the application of chemometric techniques for its analysis in order to carry out food quality control. However, where traditional chemometric techniques are unsatisfactory, ANNs are versatile and adaptive mathematical techniques in nonlinear problems providing a better model. In this chapter, the application of ANNs in food quality using HIT data is discussed by presenting a general, simple, and schematic description of the technique including the principles, neuron model, and architecture, its main application in food quality control, and an example of the full procedure of applying these techniques in the hardness modeling of Swiss-type cheese during ripening process, including an open access software developed in Octave.

11.2  ARTIFICIAL NEURAL NETWORKS ANNs are computational networks which attempt to simulate, in a highly simplified model, the networks of nerve cells of the biological central nervous system [1–3]. Whereas biological neural network consists of neurons which are interconnected among them, artificial networks have processing units (neuron) that receive N input values which are pondered and processed to produce output values following a specific nonlinear mathematical function (activation function) and simple and defined neuron interconnection (architecture or topology) [1–3] (Figure 11.1). 11.2.1 Neuron and Models The neuron processing unit consists of a summing part of N input values and weights each value, which are used to compute a parameterized activation value [1–3]. The output part, related to the activity of neuron, produces a signal from the activation value. From a mathematical point of view, a neuron can be defined as a nonlinear, parameterized, and bounded function where variables are input values and its values are called the output (Figure 11.2). In the case of hyperspectral image data, the inputs values are continuous and deterministic, i.e., spectra from different sample regions are acquired.

ANNs and HSI for Quality Control in Foods

127

FIGURE 11.1  Biological and artificial neural networks.

FIGURE 11.2  General representation of an artificial neuron.

11.2.1.1 McCulloch–Pitts Model The parameterized activation value (Z) is given by the weighted sum of its input values (Xi) (Equation 11.1). In this model, weights (Wi) are fixed values. Output signal (s) is a typically nonlinear function f(Z) of the activation value Z. Activation value (Z): n



Z = 

∑W X (11.1) i

i

i =1

11.2.1.2 Perceptron Model The Rosenblatt’s perceptron model for an artificial neuron consists of inputs where weights (Wi) are adjustable providing to neuron a learning capacity. Output signal (s) is typically a nonlinear function f(Z) of the activation value Z. A desired or target output is compared with the actual output (Y), and the difference between above output values (error) is used to adjust the weights. This model is commonly used to

128

Luis Condezo-Hoyos and Wilson Castro

ANN modeling, and iterative procedures are performed with adjusted weight values, which minimize the error [1–3]. The parameterized activation values are calculated as in Equation 11.1. 11.2.2 Architectures or Topology How different neurons are interconnected among each other, what type of activation function and how these functions are combined (weights) to produce overall output is called network architecture or topology. ANN architecture is usually represented by a graph where units operating on the same input variables are organized in layers, while the weights that modulate the combination of the nonlinear functions are represented as lines connecting units in different layers. The main features of the most used neural network architecture are described in this chapter [1–4]. 11.2.2.1 Multilayer Feed-Forward Neural Networks In the multilayer feed-forward neural networks (MLF-NNs), information flows only in the forward direction operating sequentially so that the neurons in a layer do not receive the signals (inputs or outputs) until the units in the previous layer have produced them, i.e., the output vector of a layer is used as input for the successive layer (Figure 11.3). The input layer contains as many units as number of independent variables (Xi) and it allows to introduce the experimental data in the network. The second layer is composite for a variable number of additional neurons called hidden layer, which allows nonlinear data processing [1–4]. Output layer is used to provide answer or answers (Yi) for a given set of input values following Equation 11.2:

Ym  

 = f2  

j=h

 Wmj f 1   j =1

∑

i=n

∑ i =1

  Wji Xi + Wj 0  + Wk0  (11.2)  

where Ym is the mth component of the predicted output vector, Wmj and Wji are ­input-hidden and output-hidden weights, respectively, and f1 and f 2 are nonlinear activation functions of hidden and output neurons, respectively. Commonly, f1 can be tangent hyperbolic or sigmoid function, and f 2 can be the identity or tangent hyperbolic or sigmoid function. The optimization of MLF-NNs is to find the values of the parameters (weights) that allow to reach a desire output, i.e., those weights that allow to predict output values equal or nearly to experimental. Weight values are estimated from training experiments and after that they can be used to predict output in neural network modeling by backpropagation algorithm. This algorithm changes iteratively the weights following to the steepest descend method in a direction that minimizes the error (E), defined as the square distance between the desired and the actual output values:

∆Wij ( t ) = −η

∂E +  α∆Wij ( t − 1) (11.3) ∂Wij

In this equation, t is the number of interaction, η is a proportionality constant termed learning rate related to the partial derivative respect to that weight, and α is called momentum term that represents the variation of weight during the previous iteration, which consists of three types of layers, input, hidden, and output and the information flow in the forward direction (Figure 11.4).

ANNs and HSI for Quality Control in Foods

129

FIGURE 11.3  Graphical representation of a MLF-NNs.

FIGURE 11.4  Graphical representation of a RBF-NNs.

11.2.2.2 Radial Basis Function Neural Networks Like MLF-NNs, the architecture of radial basis function neural networks (RBF-NNs) also consists of three types of layers, input, hidden, and output and the information flow in the forward direction. The main difference is the kind of activation functions through hidden and output neurons that process the signals. In fact, the neurons of hidden layer

130

Luis Condezo-Hoyos and Wilson Castro

employ so-called radial basis function and output layer use a linear activation function. A radial function values depend only on the distance, usually Euclidean distance, between input vector (X) or group of input variables and a point called center (centroid, barycenter, and similar) [1–4]. Commonly, almost all the applications of RBF-NNs hidden neuron employ Gaussian basis function, and the predicted output vector is expressed as follows: j=h



Ym   =

∑W

mj

j =1

2 exp  − β j X − µ j    (11.4)  

where Wmj is hidden to output weights, βj represents the scale factor for the jth hidden neuron, and µj is the center. The minimization of error, usually least squares one, over a set of training input/­ output pair is the main aim of RBF-NNs through finding the best set of weights Wmj. In this case the center µj and scale factor should also be sequentially optimized fixing in the first step the center and scale factor to find the weights following an iterative steepest descent method similar to Equation 11.3. 11.2.2.3 Self-Organizing Maps Originally proposed by Kohonen, self-organizing maps (SOMs) were designed as a way to perform a nonlinear mapping from a high-dimensional variable space to target space preserving the distance and proximity between samples [4–6]. In the Kohonen mapping, target space is a two-dimensional (2D) array of neurons fully connected to the input layer onto which the samples are mapped. In fact, the main issue of SOMs is that similar input vectors excite neurons, which are very close in the 2D neuron array (Figure 11.5). The size of 2D neuron array should be at least three times the number of samples analyzed

FIGURE 11.5  Graphical representation of a SOM.

ANNs and HSI for Quality Control in Foods

131

FIGURE 11.6  Common SOM representation: (a) class map, (b) unified distance, and

(c) hit histogram. [4–6]. The preservation of the topology of the original space is performed specifying to each 2D neuron grids a number of nearest neighbor neurons, which can adopt a square, rectangular, or hexagonal configuration [4–6]. SOM-based algorithm for optimization includes competitive learning so that only one neuron (winning neuron [WN] or Best Matching Unit) in the 2D layer is selected after each array input (associated with the sample) is presented to the network. The WN is selected as the one having weight vector most similar to the input pattern. Commonly, Euclidean distance is used as an optimization criterion (Equation 11.5) [4–6]:

WN ← min

∑ ( X − W ) (11.5) 2

i

ji

i

where Xi is the ith coordinate of the input vector X and Wji is the ith weight level of neuron j. After the WN is selected, the weights of each Wji in the Kohonen layer are updated (∆Wji) through an iterative process (Equation 11.6). This computational process is reached when the size of neighborhood dmax is limited only to the WN at the end of the training phase and the learning rate (η) is reached at minimum value [4–6]:

 dr  ∆Wji =  η  1 − Xi −  Wjiold (11.6)  dmax + 1 

(

)

When the whole set of samples are inputted to the network, it will be mapped in the 2D neuron arrays and can be represented following different formats. These formats are useful tools to visualize how the network responds to the different samples, and they are used in the classification problems or sample identity. Three kinds of SOM representations are commonly used: class maps, hit histograms, and unified distance matrix (Figure 11.6) [4–6].

11.3  APPLICATIONS IN FOOD QUALITY CONTROL Hyperspectral imaging systems emerged in land-related research [7] and subsequently extended to other fields of science including food technology [8]. In relation to the l­atter, several works have been developed in a wide range of applications such as prediction of tenderness in beef [9–11], prediction of composition [12,13], detection of defects and damages [14–16], detection of contaminants in milk [17], and detection of fly fruit ­infestation [18] among others.

132

Luis Condezo-Hoyos and Wilson Castro

11.3.1 Exploratory Analysis and Classification 11.3.1.1 Detection of Defects and Damages The hyperspectral images require being coupled to chemometric tools to relate intensity values or spectral profiles and the values of parameters/characteristics on evaluation. The chemometric tools most commonly used are linear (Multiple Linear Regression—MLR, Partial Least Square regression—PLSR) and nonlinear types (K-Nearest Neighbor– KNN, ANN, Naybes Bayes Network—NBN, Support Vector Machine—SVM). 11.3.1.2 Detection of Contaminant ANNs in recent years have progressively increased their importance among the techniques routinely used in chemometrics mainly due to the increases in computing power which makes it possible to perform calculations on a computer that previously required a very powerful mainframe [13,19]. 11.3.2 Regression Analysis 11.3.2.1 Analytical Quantitation A bibliographic search of research focused on the applications of ANNs and hyperspectral images for food quality evaluations covering the last 10 years was made; these works are summarized in Table 11.1. Most of these researches tried to predict or classify products using its quality characteristics derived from the product nature or commercialization conditions using mainly multilayer perceptron (MLP), and only one work used a probabilistic Neuronal Network (PNN). Likewise, a wide range of values was obtained as accuracy and correlation coefficients which showed that its applications are possible but these need to be performed according to product and process. In Figure 11.7, the objectives of the previously mentioned studies are summarized and grouped, observing that they initially focused on the classification processes of superficial abnormalities due to storage (browning), manipulation (mechanical damage) or biological agents (microorganisms and insects damage), and classes (marbling) or ­varieties (wheat varieties). Later, works were conducted to determine the presence of desirable (anthocyanin content, volatile compounds) or undesirable compounds (total volatile nitrogen) and more recently to determine the changes in properties due to storage and maturation processes. In this scenario, it is understandable that the lines in which future work will be found will be related to the compositional changes due to maturation (sweetness–­ hardness) or production process (volatile compounds formation during baking).

11.4  AN EXAMPLE IN OPEN ACCESS SOFTWARE—OCTAVE Recent applications of HSI coupled to chemometric tools as partial least square ­regression (PLSR) in texture prediction of foods are reported by Dai et al. [38] for prawn, Wu et al. [39] for salmon fillets, and Leiva-Valenzuela et al. [45] for blueberries. About application of HSI to cheeses texture properties during ripening, only one work was found which uses HSI and linear regression for a reflectance at 1,387 nm, obtaining R 2 = 0.846 [46]. Then, in the next section, as an application example for ANN and hyperspectral images, is shown the hardness modeling of Swiss-type cheese during ripening process using

ANNs and HSI for Quality Control in Foods

133

TABLE 11.1  RNA and Multi- and Hyperspectral Images: Applications in Food Quality Control Material

ANN Typea

Applicationb

Detection of

Resultsc

Source

Apples Pork Wheat Wheat Apples Rice Wheat Grapes Chili Tangerine Apples Tangerine Fish

MLP MLP MLP MLP MLP MLP MLP MLP MLP MLP MLP MLP PNN

C C C C C C C P P C C C C

60–100 69–85 91–98 86–94 98–100 86–95 96–100 65 85–90 98–99 90 98 86–92

Ariana et al. [20] Qiao et al. [21] Choudhary et al. [22] Mahesh et al. [23] ElMasry et al. [14] [24] Singh et al. [25] Fernandes et al. [26] Atas¸ et al. [27] Gómez-Sanchis et al. [15] Baranowski et al. [28] Gómez-Sanchis et al. [29] Cheng et al. [30]

Prawn Grapes

MLP MLP

C P

93–95 89–92

Dai et al. [31] Fernandes et al. [19]

Pork Tomato

MLP MLP

P P

80–90 94–96

Huang et al. [32] Liu et al. [33]

Meat Lychee

MLP MLP

P P

86–98 89

Tao et al. [34] Yang et al. [35]

Prawn Chicken Peach Tuber

MLP MLP MLP MLP

P P C P

90–93 90–98 96 86–96

Dai et al. [36] Khulal et al. [37] Pan et al. [38] Su and Sun [39]

Pork Grapes

MLP MLP

P P

82–85 73–95

Cheng et al. [40] Gomes et al. [41]

Tomato Melon

MLP —

C P

Defects during storage Marbling Class Varieties Cold damage Fungi infection Insect damage Anthocyanin content Aflatoxin content Rottenness by fungi Mechanical damage Rottenness by fungi Type of conservation process TVNd content Anthocyanin content, pH and Brix TVN content Bioactive compounds content CFUe contents Browning and moisture contents TVN content TVN content Cold injury Volatile contents generation during baking TVN content Anthocyanin content and pH Insects infection Sweetness and hardness

88–95 80

Mireei et al. [42] Sun et al. [43]

a b c d e

MLP = multilayer perceptron; PNN = probabilistic neuronal network. C = classification; P = prediction. Accuracy or correlation coefficient. TV CFU = colony forming units. Multispectral image application.

functions and scripts implemented in GNU Octave, version 4.2.0. Octave is a free software of MATLAB® available for MacOs, BDS y Windows*; therefore, Octave functions and procedures are compatible with MATLAB. * The installation package and documentation are available at: https://www.gnu.org/software/octave/.

134

Luis Condezo-Hoyos and Wilson Castro • Insect damage or infection

2017

• Composition • Undesirable metabolites • Mechanical properties

2010

• Microorganisms damage or infection • Insect damage or infection

2009 • Cold damage

2007

• Marbling

2016 • Cold damage

2006

• Process condition

• Defects

• Undesirable metabolites

2011 • Composition

2012 • Microorganisms damage or infection • Composition

2015

2008

• Classes/Varieties

2013

• Process condition • Undesirable metabolites • Microorganisms damage or infection • Composition

• Mechanical damage • Microorganisms damage or infection Classification Prediction

FIGURE 11.7  Objectives in studies using HSI/MSI and ANN.

The methodology for this example of application is summarized in Figure 11.8 and detailed in following sections. 11.4.1 Texture Analysis During the evaluation stage, hardness in samples was measured at a weekly interval, t­ aking in each sample nine cubes of 10 mm in height distributed as shown in Figure 11.9a. These cubes were tempered at ambient temperature before texture profile analysis (TPA);

ANNs and HSI for Quality Control in Foods

135

Cheese samples

Texture analysis

Image acquision Preprocessing Data extracon Dimensionality reducon Modelling

ANN model

FIGURE 11.8  Methodology for hardness evaluation in cheese.

FIGURE 11.9  Conditions for TPA: (a) location in samples of cubes for TPA and (b) CT3

texture analyzer. TPA was performed with a CT3 texture analyzer (Brookfield Engineering Laboratories Inc, Ametek Brookfield, Middleborough, MA 02346, USA), Figure 11.9b. The conditions for TPA were number of cycles = 2, trigger = 6 g, deformation = 2 mm, constant velocity = 0.5 m/s, and relaxation time = 1 s. For each cycle, the hardness H1 and H 2 were obtained. Finally, the hardness for each week was calculated as the median value for the nine cubes. 11.4.2 Image Acquisition The cheese samples were placed on the displacement platform of the hyperspectral imaging system (Figure 11.10). The system operates in reflectance mode with scanning line by line (pushbroom), within the range of 400–1,000 nm, and obtains intensity images at intervals of 8 nm. The images obtained were arranged in a three-dimensional (3D) matrix

136

Luis Condezo-Hoyos and Wilson Castro

Cover Spectrometer

Light source Cheese sample

Sample plaorm

Computer

FIGURE 11.10  Positioning of cheese samples in a hyperspectral imaging system.

called hypercube [44], whose extension was bil (band interleaved by line); likewise, to read these images, another file, known as the header, whose extension is hdr was created. The systems’ parameters for HSI acquisition were speed of sample platform = 0.5 cm/s, distance to sample surface = 28.3 cm, and diaphragm aperture = 6.4 mm. This system is controlled through the software SpectrononPro 2.62. Which in addition to controlling the acquisition automatically performs the spatial correction of the images, according to works of Fukuda et al. [47], Lee et al. [48], and Ravikanth et al. [49], using Equation 11.7 and images of white (W) and dark (D) references which were acquired from a teflon pattern (reflectance value ~ 99.9%) and by blocking the lens (reflectance ~ 0.0%):

∀Iij ∈ Rijλ : Iij′ =

Iij − Dij (11.7) Wij − Dij

where Iij is the reflectance at position ij in the raw image R at all wavelengths λ, I′ij is the reflectance in the corrected image R′, Dij is reflectance of the pixel ij in the dark reference image, and Wij is reflectance of the pixel ij in the white reference. 11.4.3 Preprocessing The principal steps of the preprocessing stage performed in this example likewise the functions are explained here. The scripts have been developed exclusively for this example, but the reader must respect the format of images and the way in which the data are extracted can be used in other cases: a. Image loading. In this step, the hyperspectral images1 were loaded to computer memory, using the function readHSI, which code can be observed in Appendices A and B. It was developed on the base of the multibandread function of MATLAB but restricted to images in bil format. b. >> [ data, info]=readHSI (file _ Header, band); The syntax of this function is described below: >> [ data , info]=readHSI ( file _ Header , band ) ;

ANNs and HSI for Quality Control in Foods

137

Where: data = Information of reflectance of HSI at wavelength band as a 3D image (3D) 2D intensity image at a specific band info = Metadata of HSI filename = String with the name of file header to be readied band = Optional parameters used to extract a specific band. c. RGB format generation. In next steps representations in RGB format are necessary, so a function named imagenRGB was developed, see Appendix C. This function combines and scale intensity images of three bands, corresponding to 600, 616, and 528 nm, in a 3D matrix. The syntax is shown in the next lines: >> imRGB=image _ RGB (file _ Header ) ; Where: imRGB = Image in RGB format filename = String with the name of file header to be readied The following script showed the use of the previously commented functions to load hyperspectral images in memory and obtain the representation in RGB format. %===================================================== % Script for HSI reading and representation % two images are generated one in intensity format at % 80 th band and another in RGB format %===================================================== % Loading the hiperspectral image clc; clear [data, info]=readHSI('Sample1.bil.hdr',80);

% Creating the RGB image imRGB=image _ RGB('Sample1.bil.hdr');

% Plotting both images figure; imagesc(data), colorbar set(gca,'fontsize',16) % font size changing in axes

138

Luis Condezo-Hoyos and Wilson Castro

FIGURE 11.11  (a) Intensity image of 80th band and (b) RGB image.

h=colorbar;set(h,'fontsize',16); % font size changing in colorbar figure; imshow(imRGB), The results of this script are shown in Figure 11.11

11.4.4 Data Extraction For this step, the function cheeseROI was developed. This function allows to draw a quadrate over the surface of each sample defining so the area to be tested. Then inside this area, 10 regions of interest (ROI) with 15 pixels per side are selected randomly and of each ROI the median spectral profile is extracted. The syntax of this function is: [MROI, p_mean] = cheeseROI(image, info, data)

Where: image = image in RGB format info = Metadata of HSI data = Information of reflectance of HSI at wavelength band as a 3D image or 2D intensity image at a specific band MROI = Matriz with information of profiles for each pixels in the ROIs. Its dimensions are M × P × Q, M is number of bands, P is number of pixel, Q is number of ROIs P_mean = Matriz with a median spectral profile with dimensions M × Q. In Figure 11.12, the results of cheeseROI function application to the Sample1 image are shown; in Figure 11.12a, the square area and the ROIs selected are shown, and in Figure 11.12b, the median profile is shown.

ANNs and HSI for Quality Control in Foods

139

FIGURE 11.12  (a) Square area and ROIs selected and (b) median profiles.

FIGURE 11.13  Data arrangement in excel sheet for ANN training.

Then, the median profiles and TPA results for each sample were saved in an excel sheet; this sheet, which structure is shown in Figure 11.13, is provided with this book named Data.xlsx. From the data in this sheet, we proceeded to calculate and plot the average spectral profiles for each Hardness 1 value, see Figure 11.14. 11.4.5 Dimensionality Reduction According to Fernandes et al. [13], large dimensionality of spectrum profiles makes these inappropriate for neural network input. One reason is that the training time increases

140

Luis Condezo-Hoyos and Wilson Castro

FIGURE 11.14  Median spectrum profiles for Hardness 1 (kgf/cm 2).

with neural network size, and in addition, neural networks should be as small as possible to have good generalization ability. In response, the principal components analysis (PCA) produces linear combinations of the original spectra wavelengths and indicates directions of the largest variance in the spectra. This principal component is defined such that the first explains the largest amount of variance and the last component explains the least amount. Each component is orthogonal to the preceding one and there are as many components as there are wavelengths in the spectra; however, usually, a small percentage of the components accounts for most of the variability in the spectra. Consequently, it is possible to use only a few components as neural network input; the information contained in the remaining components can be discarded because it is not relevant for the application in hands. More details about PCA and other methods for wavelengths selection are presented in the review of Liu et al. [50]. To apply the PCA method, one script cheeseprincomp was created, see Appendix E. This script uses the data previously obtained, reading from the excel sheet, and produces, and saves in memory, a matrix with the principal components named specPC, defined in this case in variable ncomp as six. In the Figure 11.15a and b, the principal figures are shown. 11.4.6 Artificial Neural Network Modeling The artificial neural network was modeled (created, trained, and validated) using the Octave’s neural network package developed by Michel D. Schmid, version 0.1.9.1.* The principal functions in this package for creation, training, and simulation of ANN type MLP, which were used for script implementation, are commented in the following lines: a. Creation >>net = new ( Rx2 , [ S1 S2 . . . TFN} , BTF , BLF , PF ) ;

SN ] , { TF1 TF2 . . .

* Package and documentation are accessible from https://sourceforge.net/projects/octnnettb/.

ANNs and HSI for Quality Control in Foods

141

FIGURE 11.15  (a) Y explained by component and (b) score by two principal components.

PC1 PC2 PC3

H1 / H 2

PC4 PC5 PC6

FIGURE 11.16  Neural structure for multilayer perceptron used in the example.

Where: R×2 = Matrix of min and max values for R input elements Si = Size of ith layer, for N layers TFi = Transfer function of ith layer, default = “tansig” BTF = Backpropagation network training function, default = “trainlm” BLF = Backpropagation weight/bias learning function PF = Performance function, default = “mse.” The simplified structure used for this ANN (Figure 11.16) shows six input n ­ eurons corresponding to principal components obtained in the last stage, three neurons in hidden layer, defined using the pyramid principle summarized in Equation 11.8 according to Masters [51], and one output neurons for hardness values:

 NHL = NIL × NOL (11.8)

142

Luis Condezo-Hoyos and Wilson Castro

TABLE 11.2  Training Conditions for newff Parameter

Meaning

Value

Epochs Max failure Minimum gradient μ

Maximum number of epochs to train Maximum validation failures Minimum performance gradient Controls how much the weights are changed on each iteration

200 5 1.0e10

Initial value Increment Decrement Max Shown Time

Initial value of μ Increment of μ in each iteration Decrement of μ in each iteration Maximum value of μ Show training GUI Maximum time to train in seconds

0.0001 0.1 10 10e10 50 Inf

Where: NHL = Number of neurons in hidden layer NIL = Number of neurons in input layer NOL = Number of neurons in output layer. This function of creating neural networks requires establishing certain parameters to be used during training stage, see Table 11.2. b. Training >>nett =

train ( net , P , T , M , M , VV ) ;

Where: nett = Trained multilayer network net = Multilayer network, created with newff P = Input data for training, [n × m], n is input variables and m samples T = Target data for training, [p × m], n is output variables and m samples M = Not used right now, only for compatibility with MATLAB® VV = Validation data. Contains input and target values.

c. Simulation >>simout =

sim ( nett , P ) ;

Where: simout = Output values of the simulated network nett = Network, created and trained with new(...) and train(...) P = Input data. Using the contents of the functions in this package, a new script named ANN_modeling was developed, see Appendix F. This script allows creating and training neural networks type MLP using the data previously extracted and saved for our example in the excel sheet Data.xlsx. The results of this scripts are a structure named nett which could be used for predicting new cases and are automatically saved in memory. Likewise, using data from some ROIs, the model validation was obtained by using the results shown in Figure 11.17a and b.

ANNs and HSI for Quality Control in Foods

143

FIGURE 11.17  Real versus predicted (a) H1 (b) H 2 .

In both cases, the fit was higher than 95%, showing that it is possible to build a model for hardness during ripening using ANN and spectral profiles and it could be useful for hardness prediction in new cases.

APPENDICES A. SCRIPT FOR HSI READING function [data, info]=readHSI(header, band) %================================================= % This function requiere two files, one binary file (hypercube) in % .bil format and one header file (.hdr) with metadate % This function give us the options to extract a whole hypercube, scene or % profile; using the parameters: % header is the name of header file % ind_b is 'Direct' and bands is the band selected %=================================================== switch nargin case 1 Lect = 1; % whole hipercube case 2 Lect = 2; % scene or scenes endswitch %reading data from Header info = readHeader(header); switch info.data_type case 1 precision='int8'; case 2 precision='int16';

144

Luis Condezo-Hoyos and Wilson Castro

case 3 precision='int32'; case 4 precision='float'; case 12 precision='uint16'; endswitch offset=info.header_offset; interleave=info.interleave; switch info.byte_order case 0 byteorder='ieee-le'; case 1 byteorder='ieee-be'; endswitch filename=strcat(info.label,'.',info.interleave); size=[info.lines,info.samples,info.bands]; % whole hipercube if (Lect == 1); data = HSIreader(filename, size, precision, offset,... interleave, byteorder); elseif (Lect ==2); data = HSIreader(filename, size, precision, offset, ... interleave,byteorder,{'Band','direct', band}); endif endfunction

B. AUXILIARY SCRIPT FOR HSI READING function im = HSIreader(filename,dims,precision, offset,interleave,byte Order,varargin); %================================================================== Function to read a hyperspectral image; the paramteres are provided %  for the funtion % read HSI and only for *.bil files % this function read the header and define the specific parameters to be used below in fo = info_inputs(filename, dims, precision, offset, byteOrder, varargin{:}); %================================================================== %reading info info = get_indices(info); ndx = {info.rowIndex info.colIndex info.bandIndex}; im = readDiskFile(filename, info, ndx);

ANNs and HSI for Quality Control in Foods

145

im = permute(im, [3 1 2]); endfunction %====================================================================== function info = info_inputs(filename, dims, precision, offset, byteOrder, varargin) % Open the file. Determine pixel width and input/output classes. fid = fopen(filename,'r',byteOrder); info = getPixelInfo(fid,precision); info.filename = fopen(fid); fclose(fid); % Assign sizes validateattributes(offset,{'numeric'},{'nonnegative'}); info.offset = offset; validateattributes(dims,{'numeric'},{'numel',3}); info.dims = dims; info.byteOrder = byteOrder; % Calculate the size of the data if info.bitPrecision info.dataSize = prod(info.dims) * (info.eltsize/8); else info.dataSize = prod(info.dims) * info.eltsize; endif info.rowIndex = ':'; info.colIndex = ':'; info.bandIndex = ':'; info=get_indices(info); info.subset = ''; %Determine the method to cut bands if numel(varargin)>0 method='direct'; dimensions = {'row', 'column', 'band'}; param = varargin{1}{2}; info.bandIndex=varargin{1}{3}; info.subset='b'; endif endfunction %====================================================================== function im = readDiskFile(filename, info, srcNdx) % Function that read a file from the disk . info.fid = fopen(filename, 'r', info.byteOrder); lastReadPos = 0;

146

Luis Condezo-Hoyos and Wilson Castro

skip(info,info.offset,lastReadPos); % Changin indices and sizes srcNdx = srcNdx([2 3 1]); dim = info.dims([2 3 1]); % Setting size of output image outputSize = [length(srcNdx{1}), length(srcNdx{2}), length(srcNdx{3})]; im = zeros(outputSize(1), outputSize(2), outputSize(3), info. outputClass); % Setting the start and end of reading kStart=srcNdx{1}(1); kEnd=srcNdx{1}(end); destNdx(3) = 1; for i=srcNdx{3} pos(1) = (i-1)*dim(1)*dim(2); destNdx(2) = 1; for j=srcNdx{2} pos(2) = (j-1)*dim(1); % Determine what to read posStart = pos(1) + pos(2) + kStart; posEnd = pos(1) + pos(2) + kEnd; readAmt = posEnd - posStart + 1; % Read the entire dimension skipNum = (posStart-1)-lastReadPos; if skipNum if info.bitPrecision fread(info.fid, skipNum, info.precision); else fseek(info.fid, skipNum*info.eltsize, 'cof'); endif endif [data, count] = fread(info.fid, readAmt, info.precision); lastReadPos = posEnd;  Assign the specified subset of what was read to the output % matrix im(:,destNdx(2),destNdx(3)) = data(srcNdx{1}-kStart+1); destNdx(2) = destNdx(2) + 1; endfor destNdx(3) = destNdx(3) + 1; endfor fclose(info.fid); endfunction %====================================================================== function skip(info,offset,skipSize) %Skip into a specific position into the file

ANNs and HSI for Quality Control in Foods

147

if info.bitPrecision fseek(info.fid,offset,'bof'); fread(info.fid,skipSize,info.precision); else fseek(info.fid,offset+skipSize*info.eltsize,'bof'); endif endfunction %====================================================================== function info = getPixelInfo(fid, precision) % Returns size of each pixel. Size is in bytes unless precision is % ubitN or bitN, in which case width is in bits. % Reformat the precision string info.precision = precision(~isspace(precision)); if strncmp(info.precision(1),'*',1) info.precision(1) = []; info.precision = [info.precision '=>' info.precision]; endif % Determine the input and output types (classes) lastInputChar = strfind(info.precision,'=>')-1; if isempty(lastInputChar) lastInputChar=length(info.precision); endif info.inputClass = precision(1:lastInputChar); p = ftell(fid); tmp = fread(fid, 1, info.precision); info.eltsize = ftell(fid)-p; info.outputClass = class(tmp); f it is a bit precision, parse the precision string to determine % I eltsize. if isempty(strfind(info.precision,'bit')) info.bitPrecision = false; else info.bitPrecision = true; info.eltsize = sscanf(info.inputClass(~isletter(info.inputClass)),  '%d'); endif endfunction %%===================================================================== function info = get_indices(info) if ischar(info.rowIndex) && info.rowIndex == ':' info.rowIndex = 1:info.dims(1); end if ischar(info.colIndex) && info.colIndex == ':' info.colIndex = 1:info.dims(2); end if ischar(info.bandIndex) && info.bandIndex == ':' info.bandIndex = 1:info.dims(3); end endfunction

148

Luis Condezo-Hoyos and Wilson Castro

C. SCRIPT FOR RGB IMAGE GENERATION function imRGB=image_RGB(file_header) [data info]=readHSI(file_header); imR=uint8(data(:,:,76).*(255/info.reflectance_scale_factor)); imG=uint8(data(:,:,78).*(255/info.reflectance_scale_factor)); imB=uint8(data(:,:,67).*(255/info.reflectance_scale_factor)); imRGB=cat(3,imR,imG,imB); end

D. SCRIPT FOR PROFILES EXTRACTION function [MROI, p_mean] = cheeseROI(image, info, data) %%============================================= % Function for ROIs selection in cheese sample and % spectral profile selection from the hypercube %%============================================= %representing image figure; imshow(image) %selection of ROI limits msgbox("Select ROI limits","Cheese_ROI","warn") [x,y]=ginput(2); x=round(x); y=round(y); %ploting rectangle hold on line([x(1),x(2),x(2),x(1),x(1)],[y(1),y(1),y(2),y(2),y(1)]) hold off %Random ROIs number and dimension nROI = 10; % ROIs number dx = 7; % Dimention x_rnd=round(abs(x(1)-x(2))*rand(1,nROI) + min(x)); y_rnd=round(abs(y(1)-y(2))*rand(1,nROI) + min(y)); %Ploting random ROIs hold on plot(x_rnd,y_rnd,'*r'); hold off %ROIs data extraction nBandas=info.bands; MROI =zeros(nBandas,(2*dx+1)^2,nROI); for i=1:nROI c=(x_rnd(i)-dx:1:x_rnd(i)+dx); f=(y_rnd(i)-dx:1:y_rnd(i)+dx); %Loading MROI cont=0; for n=1:length(f) %row for m=1:length(c) %column cont=cont+1;

ANNs and HSI for Quality Control in Foods

149

perf=squeeze(data(f(n),c(m),:)); MROI(:,cont,i)=sgolayfilt(perf,2,21); endfor endfor endfor %Mediam spectral profile p_p=zeros(150,nROI); for i=1:nROI p=MROI(:,:,i);p=p'; p=median(p);p_mean(:,i)=p; end figure; waves=0:8:1192; hold on for i=1:10 plot(waves,p_mean(:,i)); endfor hold off grid on; box off xlabel('Wavelength (nm)'); ylabel('Intensity') endfunction

E. SCRIPT FOR PCA APPLICATION %==================================================================== % script to obtaing the principal components %==================================================================== %loading data from excel sheet clear;clc spect=xlsread('Data.xls','Data'); [r c]=size(spect); %principal components analysis using package statsitics-1.3.0 [c oef,score,latent,tsquare] = princomp (spect(:,50:125)); % function princomp %calculating representativity Y_expl=cumsum(latent)./sum(latent); %display of Y explaining by each component comp=1:1:length(Y_expl); figure; plot(comp,Y_expl,'-b',comp,Y_expl,'*r') grid on; box off xlabel("Component"); ylabel("Y cumuled explained by component") %plotting score of components figure, plot(score(:,1),score(:,2),'+');grid on; box off xlabel("1st Principal Component") ylabel("2nd Principal Component")

150

Luis Condezo-Hoyos and Wilson Castro

%calculating new spectrum using coeficients spectN=spect(:,50:125); ncomp=6; specPC=zeros(r,ncomp); % for ncomp principal components for i=1:r for j=1:ncomp specPC(i,j)=sum(spectN(i,:)'.*coef(:,j)); endfor endfor specPC=cat(2,specPC,spect(:,[152, 153, 151, 155])); %save matrix of spectrum of principal components save specPC specPC save ncomp ncomp

F. SCRIPT FOR ANN MODELING % ============================================================== % script for ANN modeling (creation, training and validation) % ============================================================== %Clean window and memory clc; clear %Loading packages for data reading pkg load statistics pkg load optim %Reading data load specPC load ncomp dim=size(specPC); %Data for validation cnd=0; DataV1=zeros(20,dim(2)); for T=0:4 [rT1 cT1]=find(specPC(:,dim(2)-1)==T); DataVT=specPC(rT1,:); for m=1:4 cnd=cnd+1; [rm1 cm1]=find(DataVT(:,dim(2))==m); DataV1(cnd,:)=median(DataVT(rm1,:)); end end DataV=DataV1; %Preparing data for modelling rat=[0.7 0.3]; % ratio for modelling and validation for i=0:4 [r  c]=find(specPC(:,dim(2)-1)==i); % finding the rows with information for week rm = [1, round(length(r)*rat(1))]+min(r)-1; % columns for modelling rv  = [round(length(r)*rat(1))+1, length(r)]+min(r)-1; % columns for validation

ANNs and HSI for Quality Control in Foods

151

%data for modelling and validation if i==0 datM=specPC(rm(1):rm(2),:);datV=specPC(rv(1):rv(2),:); else M=cat(1,datM,specPC(rm(1):rm(2),:));datV=cat(1,datV,specPC(rv(1): dat rv(2),:)); endif endfor %Creating and training neuronal network for each hardness value for i=1:2 figure %dim=size(datM); Xm=datM(:,1:ncomp); ym=datM(:,ncomp+i); X_v=DataV(:,1:ncomp); y_v=DataV(:,ncomp+i); %Neuronal network creating, training and validation Pr=min_max(Xm'); net= newff(Pr, [ncomp, round(sqrt(ncomp)), 1],{"purelin", "purelin", "purelin"}, "trainlm", "learngdm", "mse"); net =train(net, Xm', ym'); Y_p=sim(net,X_v');Y_p=Y_p'; %Ploting hardness real and predicted hold on plot(y_v,Y_p,'*r'); plot([0 1], [0 1],'-b'); hold off if i==1 title("Hardness (H_1)") else title("Hardness (H_2)") endif grid on; box off; xlim([0.3 0.9]);ylim([0.3 0.9]) xlabel('Real hardness kg_f / cm^2'); ylabel('Predicted hardness kg_f / cm^2'); %r square calculated for H1 and H2 [p,e_var,r,p_var,y_var] = LinearRegression (y_v,Y_p); text (0.6, 0.5, strcat("R^2 = ", num2str(p*p))); endfor

REFERENCES 1. J. H. Flores. Focus on Artificial Neural Networks. Nova Science Publisher, Inc. New York. 399 pp, 2011. 2. D. Graupe. Principle of Artificial Neural Networks. World Scientific Publishing Co.: Singapore, 2nd edition. 301 pp, 2007.

152

Luis Condezo-Hoyos and Wilson Castro

3. B. Yegnanarayana. Artificial Neural Network. Prentice Hall. New Delhi. 461 pp, 2005. 4. F. Marini. Artificial neural networks in foodstuff analyses: Trends and perspectives A review. Analytica Chimica Acta 635:121–131, 2009. 5. T. Kohonen. Essentials of the self-organizing map. Neural Networks 37:52–65, 2013. 6. R. C. Brereton. Self organising maps for visualising and modelling. Chemical Central Journal 6(Suppl 2):S1, 2012. 7. T. Lillesand, R. Kiefer, and J. Chipman. Remote Sensing and Image Interpretation. John Wiley & Sons: Hoboken, NJ, 2014. 8. D. Wu and D. Sun. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review—part i: Fundamentals. Innovative Food Science & Emerging Technologies, 19:1–14, 2013. 9. H. He, D. Wu, and D. Sun. Potential of hyperspectral imaging combined with chemometric analysis for assessing and visualising tenderness distribution in raw farmed salmon fillets. Journal of Food Engineering, 126:156–164, 2014. 10. G. K. Naganathan, K. Cluff, A. Samal, C. Calkins, D. Jones, C. Lorenzen, and J. Subbiah. A prototype on-line AOTF hyperspectral image acquisition system for tenderness assessment of beef carcasses. Journal of Food Engineering, 154:1–9, 2015. 11. G. K. Naganathan, K. Cluff, A. Samal, C. Calkins, D. Jones, C. Lorenzen, and J. Subbiah. Hyperspectral imaging of ribeye muscle on hanging beef carcasses for tenderness assessment. Computers and Electronics in Agriculture, 116:55–64, 2015. 12. C. Everard, M. Kim, and H. Lee. A comparison of hyperspectral reflectance and fluorescence imaging techniques for detection of contaminants on spinach leaves. Journal of Food Engineering, 143:139–145, 2014. 13. A. Fernandes, C. Franco, A. Mendes-Ferreira, A. Mendes-Faia, P. Costa, and P.  Melo-Pinto. Brix, pH and anthocyanin content determination in whole Port wine grape berries by hyperspectral imaging and neural networks. Computers and Electronics in Agriculture, 115:88–96, 2015. 14. G. ElMasry, N. Wang, and C. Vigneault. Detecting chilling injury in Red Delicious apple using hyperspectral imaging and neural networks. Postharvest Biology and Technology, 52(1):1–8, 2009. 15. J. Gómez-Sanchis, J. Martín-Guerrero, E. Soria-Olivas, M. Martínez-Sober, R. Magdalena-Benedito, and J. Blasco. Detecting rottenness caused by Penicillium genus fungi in citrus fruits using machine learning techniques. Expert Systems with Applications, 39(1):780–785, 2012. 16. R. Zhou, S. Kaneko, F. Tanaka, M. Kayamori, and M. Shimizu. Image-based field monitoring of cercospora leaf spot in sugar beet by robust template matching and pattern recognition. Computers and Electronics in Agriculture, 116:65–79, 2015. 17. X. Fu, M. Kim, K. Chao, J. Qin, J. Lim, H. Lee, A. Garrido-Varo, D. Pérez-Marín, and Y. Ying. Detection of melamine in milk powders based on NIR hyperspectral imaging and spectral similarity analyses. Journal of Food Engineering, 124:97–104, 2014. 18. R. Lu and D. Ariana. Detection of fruit fly infestation in pickling cucumbers using a hyperspectral reflectance/transmittance imaging system. Postharvest Biology and Technology, 81:44–50, 2013.

ANNs and HSI for Quality Control in Foods

153

19. R. Leardi. Nature-Inspired Methods in Chemometrics: Genetic Algorithms and Artificial Neural Networks, Elsevier: Amsterdam, The Netherlands, 2003. 20. D. Ariana, D. Guyer, and B. Shrestha. Integrating multispectral reflectance and fluorescence imaging for defect detection on apples. Computers and Electronics in Agriculture, 50(2):148–161, 2006. 21. X. Qiao, J. Jiang, X. Qi, H. Guo, and D. Yuan. Utilization of spectral-spatial characteristics in shortwave infrared hyperspectral images to classify and identify fungicontaminated peanuts. Food Chemistry, 220:393–399, 2017. 22. R. Choudhary, S. Mahesh, J. Paliwal, and D. Jayas. Identification of wheat classes using wavelet features from near infrared hyperspectral images of bulk samples. Biosystems Engineering, 102(2):115–127, 2009. 23. S. Mahesh, A. Manickavasagan, D. S. Jayas, J. Paliwal, and N. White. Feasibility of near-infrared hyperspectral imaging to differentiate Canadian wheat classes. Biosystems Engineering, 101(1):50–57, 2008. 24. L. Liu, M. Ngadi, S. Prasher, and C. Gariépy. Categorization of pork quality using Gabor filter-based hyperspectral imaging technology. Journal of Food Engineering, 99(3):284–293, 2010. 25. C. Singh, D. Jayas, J. Paliwal, and N. White. Identification of insect-damaged wheat kernels using short-wave near-infrared hyperspectral and digital colour imaging. Computers and Electronics in Agriculture, 73(2):118–125, 2010. 26. A. Fernandes, P. Oliveira, J. Moura, A. Oliveira, V. Falco, M. Correia, and P. MeloPinto. Determination of anthocyanin concentration in whole grape skins using hyperspectral imaging and adaptive boosting neural networks. Journal of Food Engineering, 105(2):216–226, 2011. 27. M. Atas, Y. Yardimci, and A. Temizel. A new approach to aflatoxin detection in chili pepper by machine vision. Computers and Electronics in Agriculture, 87:129– 141, 2012. 28. P. Baranowski, W. Mazurek, and J. Pastuszka-Woźniak. Supervised classification of bruised apples with respect to the time after bruising on the basis of hyperspectral imaging data. Postharvest Biology and Technology, 86:249–258, 2013. 29. J. Gómez-Sanchis, J. Blasco, E. Soria-Olivas, D. Lorente, P. Escandell-Montero, J. Martínez-Martínez, M. Martínez-Sober, and N. Aleixos. Hyperspectral LCTFbased system for classification of decay in mandarins caused by Penicillium digitatum and Penicillium italicum using the most relevant bands and non-linear classifiers. Postharvest Biology and Technology, 82:76–86, 2013. 30. J. Cheng, D. Sun, H. Pu, X. Chen, Y. Liu, H. Zhang, and J. Li. Integration of classifiers analysis and hyperspectral imaging for rapid discrimination of fresh from cold-stored and frozen-thawed fish fillets. Journal of Food Engineering, 161:33–39, 2015. 31. Q. Dai, J. Cheng, D. Sun, H. Pu, X. Zeng, and Z. Xiong. Potential of visible/nearinfrared hyperspectral imaging for rapid detection of freshness in unfrozen and frozen prawns. Journal of Food Engineering, 149:97–104, 2015. 32. S. Huang, L. Qi, X. Ma, K. Xue, W. Wang, and X. Zhu. Hyperspectral image analysis based on BoSW model for rice panicle blast grading. Computers and Electronics in Agriculture, 118:167–178, 2015. 33. C. Liu, W. Liu, W. Chen, J. Yang, and L. Zheng. Feasibility in multispectral imaging for predicting the content of bioactive compounds in intact tomato fruit. Food chemistry, 173:482–488, 2015.

154

Luis Condezo-Hoyos and Wilson Castro

34. F. Tao, Y. Peng, C. Gomes, K. Chao, and J. Qin. A comparative study for improving prediction of total viable count in beef based on hyperspectral scattering characteristics. Journal of Food Engineering, 162:38–47, 2015. 35. Y. Yang, D. Sun, and N. Wang. Rapid detection of browning levels of lychee pericarp as affected by moisture contents using hyperspectral imaging. Computers and Electronics in Agriculture, 113:203–212, 2015. 36. Q. Dai, J. Cheng, D. Sun, Z. Zhu, and H. Pu. Prediction of total volatile basic nitrogen contents using wavelet features from visible/near-infrared hyperspectral images of prawn (Metapenaeus ensis). Food Chemistry, 197:257–265, 2016. 37. U. Khulal, J. Zhao, W. Hu, and Q. Chen. Nondestructive quantifying total volatile basic nitrogen (TVB-N) content in chicken using hyperspectral imaging (HSI) technique combined with different data dimension reduction algorithms. Food Chemistry, 197, Part B:1191–1199, 2016. 38. L. Pan, Q. Zhang, W. Zhang, Y. Sun, P. Hu, and K. Tu. Detection of cold injury in peaches by hyperspectral reflectance imaging and artificial neural network. Food Chemistry, 192:134–141, 2016. 39. W. Su and D. Sun. Multivariate analysis of hyper/multi-spectra for determining volatile compounds and visualizing cooking degree during low-temperature baking of tubers. Computers and Electronics in Agriculture, 127:561–571, 2016. 40. W. Cheng, D. Sun, H. Pu, and Q. Wei. Chemical spoilage extent traceability of two kinds of processed pork meats using one multispectral system developed by hyperspectral imaging combined with effective variable selection methods. Food Chemistry, 221:1989–1996, 2017. 41. V. Gomes, A. Fernandes, P. Martins-Lopes, L. Pereira, F. Mendes, and P. MeloPinto. Characterization of neural network generalization in the determination of pH and anthocyanin content of wine grape in new vintages and varieties. Food Chemistry, 218:40–46, 2017. 42. S. Mireei, S. Amini-Pozveh, and M. Nazeri. Selecting optimal wavelengths for detection of insect infested tomatoes based on SIMCA-aided CFS algorithm. Postharvest Biology and Technology, 123:22–32, 2017. 43. M. Sun, D. Zhang, L. Liu, and Z. Wang. How to predict the sugariness and hardness of melons: A near-infrared hyperspectral imaging method. Food Chemistry, 218:413–421, 2017. 44. D. Wu, D. Sun, and Y. He. Novel non-invasive distribution measurement of texture profile analysis (TPA) in salmon fillet by using visible and near infrared hyperspectral imaging. Food Chemistry, 145:417–426, 2014. 45. G. A. Leiva-Valenzuela, R. Lu, and J. M. Aguilera. Prediction of firmness and soluble solids content of blueberries using hyperspectral reflectance imaging. Journal of Food Engineering, 115(1):91–98, 2013. 46. L. Darnay, F. Králik, G. Oros, Á. Koncz, and F. Firtha. Monitoring the effect of trans-glutaminase in semi-hard cheese during ripening by hyperspectral imaging. Journal of Food Engineering, 196:123–129, 2017. 47. S. Fukuda, E. Yasunaga, M. Nagle, K. Yuge, V. Sardsud, W. Spreer, and J. Müller. Modelling the relationship between peel colour and the quality of fresh mango fruit using Random Forests. Journal of Food Engineering, 131:7–17, 2017. 48. W. Lee, M. Kim, H. Lee, S. Delwiche, H. Bae, D. Kim, and B. Cho. Hyperspectral near-infrared imaging for the detection of physical damages of pear. Journal of Food Engineering, 130:1–7, 2014.

ANNs and HSI for Quality Control in Foods

155

49. L. Ravikanth, C. Singh, D. Jayas, and N. White. Classification of contaminants from wheat using near-infrared hyperspectral imaging. Biosystems Engineering, 135:73–86, 2015. 50. D. Liu, D. Sun, and X. Zeng. Recent advances in wavelength selection techniques for hyperspectral image processing in the food industry. Food and Bioprocess Technology, 7(2):307–323, 2013. 51. T. Masters. Practical Neural Network Recipes in C++. Morgan Kaufmann, Elsevier: The Netherlands, 1993.

Section

III

Applications

Chapter

12

Recent Advances for Rapid Detection of Quality and Safety of Fish by Hyperspectral Imaging Analysis Chao-Hui Feng The University of Tokyo Chengdu University Sichuan Agricultural University

Yoshio Makino and Masatoshi Yoshimura The University of Tokyo

Francisco J. Rodríguez-Pulido University of Seville

CONTENTS 12.1 Introduction: Background and Driving Forces 159 12.2 Freshness 160 12.3 Physical Properties 164 12.4 Chemical Compositions 165 12.5 Nematodes Inspection 168 12.6 Microbial Spoilage Inspection 169 12.7 Conclusions 169 Acknowledgments 170 References 170

12.1  INTRODUCTION: BACKGROUND AND DRIVING FORCES Fish has been one of the most important components of several and nutritious diets in the world. Its contribution to human health is well documented, being an essential topic for researchers. The reason of its importance lays in its nutritional composition. Fish is

159

160

Chao-Hui Feng, et al.

an important source of protein, vitamins, trace elements, and other nutritional components, with the most significant being, without doubt, the polyunsaturated fatty acids which are related to the long-chain ω-3 eicosapentaenoic acid and docosahexaenoic acid (DHA) (Hernández-Martínez et al., 2013). ω-3 fatty acids are extremely important for the neural development in the infant in utero and the first few years after birth (Uauy et al., 2000). Fish and seafood are the major elements for many dietary guidelines, such as Mediterranean and Japanese diets. Humans lack the enzymes necessary to produce ω-3 fatty acids that have to be obtained from the diet or produced in vivo from diet-derived ω-3 fatty acid precursors such as α-linolenic acid. Similarly, DHA, which is a primary structural component of the human brain, cerebral cortex, skin, and retina, is an essential nutrition for infants and can be greatly obtained from fish or other types of marinederived food (Montaño et al., 2001). Fishes such as salmon, tuna, mackerel, and sardines are particularly rich in polyunsaturated fatty acids (PUFAs). Nowadays, high quality of the fish is highly demanded from both consumers and fishery industries. Significant efforts have been made by the industries to enhance the quality and safety of the aquatic and seafood products by using new technologies such as novel cooling (Feng et al., 2013a, b; 2014a, b, c; 2016; Feng and Sun, 2014; Feng and Li, 2015), freezing (Crane et al., 2016; Sánchez-Alonso et al., 2018), and packaging (Kuswandi et al., 2012; Heide and Olsen, 2017). Meanwhile, manufactures, consumers, and government regulators increasingly and imperatively require reliable, rapid, and practical analytical methods and techniques for safety detection and assessment of the aquatic products (Kamruzzaman et al., 2015a). However, high-quality and safe fish products are also closely related to human health and dietary benefits. For example, the nematode, as a common parasite in fish muscle, exerts a negative impact on consumers’ purchase decision (Heia et al., 2007). Anisakis, as a genus of parasitic nematodes that have lifecycles involving fish and marine mammals, can cause anisakiasis or an allergic reaction (Berger and Marr, 2006). This parasite poses a high risk to human health via intestinal infection with worms from the eating of uncooked or underprocessed fish. Within a few hours of ingestion of fish with Anisakis, the parasitic worm tries to burrow from the intestinal wall to escape. As it cannot penetrate and gets stuck and eventually dies in the human body, it arouses an immune response from our body and forms a ball-like structure surrounding the dead worms, leading to severe abdominal pain and vomiting due to the block of the digestive system caused by the balllike structure. If the worm (larvae) passes into the bowel or large intestine, it may cause a severe eosinophilic granulomatous response and symptoms mimicking Crohn’s disease. Herein, a noninvasive and rapid analysis and evaluation of fish and other seafood quality is highly demanded to ensure the humans health. Hyperspectral imaging (HSI), as an emerging and innovative tool, has been intensively exploited and investigated for nondestructively detecting the food and food products during the past few years (Kamruzzaman et al., 2015a, b, c; 2016a, b, c, d; Feng et al., 2018). In this chapter, the detection of fish freshness, evaluation of physical properties and chemical composition and inspection of microbial spoilage in fish by HSI are discussed.

12.2 FRESHNESS Fish freshness is always regarded as one of the most important integrated quality attributes for evaluating the quality of the fish. This critical index affects safety, nutritional quality, and edibility of fish, which may be caused by physical, chemical, microbiological, and biochemical processes (Cheng et al., 2013). Table 12.1 summarizes the recent

TABLE 12.1  Application of Hyperspectral Imaging for the Detection of Fish Properties Applications Fish classification Fish storage prediction

Wavelength Range (nm) 400–1,100 400–1,000 430–1,000

Key Wavelengths (nm) 606, 636, 665, 705, 764 433, 489, 500, 867, 900, 922, 944 —

892–2,495

—

Accuracy

PLSR SVM KNN PLSR

88.3 ± 4.5% Rcv2 = 0.87 R2 = 0.98; 0.99 R2 = 0.92

PLSR

Rp2 = 0.94

LS-SVM

R2 = 0.94

432, 455, 588, 635, 750, 840, 970

400–1,000

441, 560, 598, 639, 684

897–1,753

980, 1,220, 1,450

MLR

Textural prediction

400–1,000

PLSR

Waterholding capacity prediction Gross energy density values prediction

400–1,000; 897–1,753 900–1,700

Hardness: 408, 417, 435, 457, 523, 555, 569, 585, 605, 619, 728, 915 Gumminess: 408, 435, 457, 552, 569, 585, 607, 619, 643, 782, 915, 989 Chewiness: 417, 435, 469, 555, 569, 585, 607, 623, 782, 915, 948, 989 440, 450, 520, 595, 600, 615, 765, 830, 885, 925, 975, 995 931, 1,001, 1,135, 1,168

Rp2 = 0.87 (L*), 0.74 (a*), 0.80 (b*) Rp2 = 0.85 (hardness), 0.80 (gumminess), 0.85 (chewiness)

Sone et al. (2012) Ivorra et al. (2016) Washburn et al. (2017) Whitworth et al. (2010) Cheng et al. (2015b)

Cheng and Sun (2015a) Wu et al. (2012)

Ma et al. (2017)

CARS-PLSR

Rp2 = 0.94

Wu and Sun (2013a)

Stepwise-MLR

Rp2 = 0.91

Xu et al. (2016a) (Continued)

161

400–1,000

K value (freshness) prediction in grass carp and silver carp fillets QIS prediction of grass carp fish fillet Color prediction

Reference

Detection of Quality and Safety of Fish

Modeling

Applications

Wavelength Range (nm)

Distribution of blood

430–1,000

Lipid oxidation prediction

400–1,000 900–1,700

400–1,000

Moisture

400–1,000

Salt content Nematodes inspection

760–1,040 400–1,100 400–1,000

TVC

400–1,000 400–1,000

—

444, 475, 553, 577, 590, 623, 710, 795, 847, 937 1,061, 1,071, 1,078, 1,151, 1,155, 1,158, 1,335, 1,338, 1,342, 1,352, 1,395, 1,398, 1,408, 1,489, 1,529, 1,535, 1,552, 1,565 420, 466, 523, 552, 595, 615, 717, 850, 955 414, 490, 520, 563, 580, 593, 634, 709, 972 — 500–550 540, 547, 550, 576, 646

495, 535, 550, 585, 625, 660, 785, 915 410, 488, 553, 665, 750, 825, 960

Modeling Nonnegativity constrained spectral unmixing PLSR

Accuracy —

Reference Skjelvareid et al. (2017)

Rp2 = 0.83

Cheng et al. (2015a)

Forward stepwise-MLR

Rp2 = 0.81

Xu et al. (2016b)

LS-SVM

Rp2 = 0.92

Cheng et al. (2014)

PLSR

Rp2 = 0.90

Qu et al. (2017)

PLSR SIMCA Gaussian maximum likelihood classifier CARS-PLSR

Rp2 = 0.92 — 60.30%

Segtnan et al. (2009) Wold et al. (2001) Sivertsen et al. (2012)

Rp2 = 0.96

Wu and Sun (2013b)

SPA-PLSR

Rp2 = 0.90

Cheng and Sun (2015b)

Chao-Hui Feng, et al.

TVB-N

Key Wavelengths (nm)

162

TABLE 12.1 (Continued)  Application of Hyperspectral Imaging for the Detection of Fish Properties

Detection of Quality and Safety of Fish

163

applications to freshness detection in fish. Although the conventional well-established methods and analytical techniques such as sensory analysis, high performance liquid chromatographic, spectrophotometric, and electrochemical approaches are able to detect the fish freshness, the aforementioned methods are generally time-consuming, tedious, and relatively expensive (He and Sun, 2015a). Moreover, a skilled personnel is required to perform the experiments, which is difficult to be used in on-field applications (Cheng et al., 2014). Compared with traditional methods, HSI is a environmental friendly, toxicfree, noninvasive, time-saving technique (Kamruzzaman et al., 2016a; Feng et al., 2018). The spectroscopic changes in fresh salmon stored under different atmospheres (air, modified atmosphere packaging, and 90% vacuum) were studied using HSI (400–1,100 nm) (Sone et al., 2012). The main spectral changes occurred at wavelengths of 606 and 636 nm, which could be used to classify fresh salmon fillets with different types of packaging used for the storage (Sone et al., 2012). The authors attributed the spectroscopic changes to the different oxidation states of the heme proteins in the salmon (Sone et al., 2012). Similarly, HSI with the short wave near-infrared SW-NIR range was employed to evaluate the shelf life of the vacuum-packed chilled smoked salmon (Ivorra et al., 2016). The partial least square (PLS) regression model using fat tissue presented a better cross validation outcome (Rcv2 = 0.80). A support vector machine model successfully classified the samples stored for 0, 10, 20, 40, and 60 days, and the prediction accuracy being 87.2% (Ivorra et al., 2016). Fillets of cod under different programs of freezing, thawing, and storage were investigated by HSI (Washburn et al., 2017). The results showed that samples frozen to −40°C and −20°C can be accurately predicted using the region of 450–600 nm, with accuracy rate of 98% and 99%, respectively. The reason for differentiating with fresh, once thawed, and twice thawed samples is because of the increased light scattering with each freeze-thaw cycle. It was proposed that the increased light scattering is as the result of the denaturation of proteins during freezing and thawing (Love, 1962). K value, as an important freshness index, is widely used for the assessment of chemical spoilage and nucleotide degradation (Cheng et al., 2015b). As the increase in K value indicates the endogenous enzyme actions and bacterial spoilage (Lowe et al., 1993), the K value of 20% is regarded as the optimal freshness limit, while the K value of 60% being the rejection point (Ehira, 1976). The K values in grass carp and silver carp fillets were investigated by using HSI (400–1,000 nm). Seven optimal wavelengths, i.e., 432, 455, 588, 635, 750, 840, and 970 nm, were selected in the study by Cheng et al. (2015b). The simplified PLS model using the optimal wavelengths shows excellent performance ability (Rp2 = 0.935; Root Mean Square Error of Prediction (RMSEP) = 5.17%), with comparable accuracy to the original models using the whole wavelengths. According to the distribution map of fish fillet (Figure 12.1), the storage for 2 days at 4°C ± 1°C showed the limit freshness for the fish and it started to spoil from the bottom of the fish sample (Cheng et al., 2015b). Quality index method (QIM) is a standard to evaluate and describe the fish freshness. Although it is the one of the most wholesome and straightforward ways to describe the fish freshness, it is subjective, time-consuming, and not of practical use for the largescale commerce. Cheng and Sun (2015a) predicted sensory quality index scores (QIS) of grass carp fish fillet by using HSI in tandem with data fusion technique. Whitworth et al. (2010) also measured QIM by HSI. They acquired images from exterior surface and fillets of cod and measured the average spectra from areas having demonstrated the greatest potential for discrimination of changes along time. After applying a segmentation criterion for selecting the region of interest, they divided the fillets into head, middle, and tail pieces of equal size for considering the average spectra. Then, the authors used PLS

164

Chao-Hui Feng, et al.

FIGURE 12.1  Distribution maps of K value in fish fillets at four different K values (freshness): (a) K = 24.2%, storage for 0 days; (b) K = 45.6%, storage for 2 days; (c) K = 78.1%, storage for 4 days; (d) K = 89.8%, storage for 6 days (cited from Cheng et al., 2015b).

regression with full cross validation for predicting some reference measurements of freshness, such as storage time and Torry scores for raw and cooked cod. They proved that HSI improves the results obtained by visible/NIR spectroscopy. In fact, the best results were obtained for the head end of the fillet (R 2 = 0.92 and SECV = 1.66 days). Rigor mortis is also related to freshness.

12.3  PHYSICAL PROPERTIES Color is an important criterion for evaluating the quality of the foodstuffs as consumers use color as the first visual attribute to judge the quality of the foods, resulting in purchasing intention. For example, redder salmon is preferable for consumers as it is associated with better quality, higher flavor, and freshness (Troy and Kerry, 2010). Wu et al. (2012) investigated the color of salmon from Scotland, Ireland, and Norway using HSI with the spectral range of 897–1,753 nm. Three local absorption maxima were reported to appear at 980, 1,220, and 1,450 nm, which may due to the presence of water, fat, and protein in the sample, respectively. The PLSR model using full spectral range achieved Rp2 of 0.864, 0.736, and 0.798 for L*, a*, and b*, respectively. For the reduced wavelengths selection, the final prediction models were considered as the multiple linear regression (MLR) model, with Rp2 of 0.869, 0.729, and 0.788 for L*, a*, and b*, respectively. Approximate 98.5%, 98.5%, and 96.5% of wavelengths were reduced for L*, a*, and b* during wavelengths selection, respectively, which will significantly reduce the high dimensionality with redundancy and collinearity among spectral wavelengths and facilitate the designing of a multispectral imaging system (Wu et al., 2012). Apart from the color that influences the consumer’s preference, texture is also an important index to evaluate food or processed food products to control different processing operations (Ma et al., 2017). The textural properties (Warner–Bratzler shear force [WFSF], hardness, gumminess, and chewiness) of fish fillets for vacuum freezedrying at different durations (3, 6, 12, 18, 24, 30, and 36 h) were investigated using HSI (400–1,000 nm). The authors stated that choosing optimal wavelengths did not significantly reduce the accuracy of the model for predicting WBSF in comparison with the full

Detection of Quality and Safety of Fish

165

wavelengths. The Rp2 of model using the selected wavelengths even improved a little bit (0.81%) as compared to the full wavelengths. The predicting ability in the study by Ma et al. (2017) was reported to be better than the other studies (ElMasry et al., 2012; Kamruzzaman et al., 2013), which may be due to the greater textural changes occurred during the drying process. Selected wavelengths displayed an acceptable accuracy, with Rp2 of 0.85, 0.80, and 0.85 for the attributes of hardness, gumminess, and chewiness, respectively (Ma et al., 2017). Water holding capacity (WHC) is an important indicator for describing quality evolution in food and food products. As it is the ability of muscle to retain water or resist water loss, fish becomes tougher with the reduction of WHC (Ocano-Higuera et al., 2009). Therefore, it is critically important to determine the WHC of fish products to maintain the high quality of the products. Wu and Sun (2013a) applied visible and NIR HSI to visualize the distribution of WHC in salmon flesh. The authors concluded that the visible and SW-NIR range of 400–1,000 nm performed better than that the long-wave NIR range of 897–1,753 nm for the WHC prediction (Wu and Sun, 2013a). Twelve important wavelengths were selected out from 121 wavelengths using competitive adaptive reweighted sampling (CARS) algorithm. The established CARSPLSR model achieved a high correlation coefficient of prediction of 0.937 with a lower RMSEP of 0.964%. The gross energy density values of salmon fillets were studied using HSI (900–1,700 nm) in tandem with multivariate data analysis (Xu et al., 2016a). Four optimal wavelengths (931, 1,001, 1,135, and 1,168 nm) were selected by stepwise-MLR variable selection method with a high coefficient prediction (Rp2 = 0.91). The model established in this research successfully illustrated the distribution of energy density values of different salmon fillets, providing accuracy calorie counts in food nutrition labeling (Xu et al., 2016a). Fat content in salmon fillets proved to exert an important impact on the calorie counts (Xu et al., 2016a). The blood in whitefish fillets was detected by HSI with a range of 430–1,000 nm (Skjelvareid et al., 2017). The method used in the research was based on diffuse reflectance and unmixing of measured absorbance spectra into known spectra for hemoglobin, water, and muscle tissue. It was found that the peak at approximately 605 nm might be correlated to cytochrome oxidase (Skjelvareid et al., 2017). The unmixing model achieved a good fit between measured and modeled spectra, which was of practice use for the classification of fillets based on blood defects. Fillet shown in Figure 12.2a and e can be classified as a high-quality fillet while fillet shown in Figure 12.2d and h can be considered as a low-quality fillet due to the elevated blood levels throughout the muscle and several blood spots (Skjelvareid et al., 2017).

12.4  CHEMICAL COMPOSITIONS As fish is rich in long-chain omega-3 PUFAs, the unsaturated carbon–carbon bonds are easily oxidized which is the major cause of food deterioration (Xu et al., 2015). As a result, lipid oxidation detection is vital to evaluate fish quality. Cheng et al. (2015a) monitored lipid oxidation in grass carp fillets (Ctenopharyngodon idella) during cold storage at 4°C for 0, 2, 5, and 8 days using HSI. Ten optimal wavelengths (444, 475, 553, 577, 590, 623, 710, 795, 847, and 937 nm) were selected according to weighted regression coefficients. The optimized MLR model provided even better results (Rp2 = 0.834; RMSEP = 0.115) than that of using full spectral wavelengths (Rp2 = 0.833; RMSEP = 0.117) (Cheng et al., 2015a). Distribution

166

Chao-Hui Feng, et al.

FIGURE 12.2  Color (a–d) and blood concentration images (e–h) for four fillets (cited

from Skjelvareid et al., 2017).

map clearly distinguished different TBA values in fish fillets (Figure 12.3). Similarly, lipid oxidation in Atlantic salmon (Salmo salar) fillets stored for 12 days was investigated based on the thiobarbituric acid (TBA) values (Xu et al., 2016b). Eighteen optimal wavelengths were selected with forward stepwise-MLR variable selection method with the coefficient of determination in prediction of 0.814 and RMSEP of 2.081. According to the distribution map, the distribution density of TBA was nonuniform and asymmetric (Xu et al., 2016b), which is consistent with the study of Cheng et al. (2015a). The ventral section was observed to be more severe than that in dorsal part, which may be due to the different contents of lipid in the different parts (Xu et al., 2016b).

Detection of Quality and Safety of Fish

167

FIGURE 12.3  Distribution maps of thiobarbituric (TBA) value in fish fillets at four different TBA values: (a) TBA = 0.2355 mg/kg; (b) TBA = 0.4602 mg/kg; (c) TBA = 0.9524 mg/kg; (d) TBA = 1.1213 mg/kg (cited from Cheng et al., 2015a).

Total volatile basic nitrogen (TVB-N), as a kind of alkali compound containing nitrogen, generates during the decomposition of protein. Apart from the aforementioned TBA and K value to evaluate the freshness of fish, TVB-N is also recognized as an important indicator to assess the freshness of fish. Cheng et al. (2014) determined the TVB-N contents of grass carp fillets during different frozen storage conditions by using HSI in the range of 400–1,000 nm. The results revealed that least squares support vector machine (LS-SVM) model developed for predicting TVB-N using full wavelengths performed better than that of PLSR model, with Rp2 of 0.916 for LS-SVM and 0.905 for PLSR. Subsequently, TVB-N prediction ability in grass carp fish fillet stored at 4°C for up to 8 days was improved by using Physarum network (PN) combined with genetic algorithm (GA) (Cheng et al., 2017). Six optimal wavelengths (i.e., 428, 550, 601, 655, 775, and 980 nm) were selected, and the prediction performance increased to 0.956 and 0.947 for using PN-GA-PLSR and PN-GA-LS-SVM, respectively (Cheng et al., 2017). Moisture, as one of the most important component in meat, has a strong relation to the quality, shelf-life, and economic profits of fish products. Qu et al. (2017) studied the moisture content in grass carp slices under different freeze drying durations. The spectra data were pretreated by multiplicative scatter correction and standard normal variate. Nine wavelengths (i.e., 414, 490, 520, 563, 580, 593, 634, 709, and 972 nm) were chosen as the optimal wavelengths, and the simplified PLSR model can achieve good prediction results of 0.902. It is reported that there is no need to process spectral pretreatments as it seems to be no improvement (Qu et al., 2017). One of the advantages of hyperspectral is that imaging the generated data can be used for generating the distribution map and knowing where and how much the quality attributes change with regard to different treatments (in this case is the different freeze drying periods). In conventional measurements, it is only possible to measure the average moisture of each grass carp slice instead of measuring each point on the surface. With the help of visualization map, the gradient and migration of moisture content during the drying process were clearly displayed. This will render people to better understand the moisture content variations during the processing of drying. Salt content is also an interesting parameter that can be evaluated by HSI. Salted and smoked salmon are the traditional Norwegian food, and are consumed around the

168

Chao-Hui Feng, et al.

world. Due to its antimicrobial properties, NaCl enlarges the shelf-life of salmon fillets up to 6 weeks approximately. Anyway, a high intake of salt has negative effects on human health. Therefore, its evaluation arises as a property to be evaluated. Segtnan et al. (2009) built robust models by merging HSI with X-ray CT for predicting both salt and fat content. They reached a correlation of R = 0.92 and RMSECV = 0.40% NaCl by comparing reference and predicted values.

12.5  NEMATODES INSPECTION As mentioned above, nematodes in fish are harmful to the health of the human beings and the existence of parasites in fish will definitely cause the rejection of the products by consumer. It is thus of great interest and high demand for the fish industry to apply an instrument to rapid and efficiently detect parasites in fish products. Wold et al. (2001) first utilized a multispectral imaging system (400–1,100 nm) to detect parasites in cod fillets. A significant difference in the spectral properties was found between parasites at different depths and those of normal flesh. By combining spectral imaging and soft independent modeling of class analogy (SIMCA), parasitic nematodes in cod fillets can be automatically detected (Figure 12.4). The results showed that the parasites embedded deeper than 6 mm can be detected by using this method (Wold et al., 2001). Subsequently, Heia et al. (2007) distinguished good fish muscle from parasites for the same cod species. The detected depth has reported to be extended to 8 mm below the fillet surface (Heia et al., 2007), which is 2–3 mm deeper than that observed by the previous studies (Wold et al., 2001). Encouragingly, Sivertsen et al. (2012) applied the Gaussian maximum likelihood classifier to automatically detect nematodes in cod fillets. Results reveal that the Parasites

Skin remnants

(a)

(b)

(c)

(d)

(e)

(f)

FIGURE 12.4  The nematodes are naturally at 1–4 mm depth of three different fillets (a–c); the same images with classification results, the soft independent modeling of class analogy (SIMCA) was based on pixels within both the white areas in (d) and (f). The black points indicate pixels classified as parasite (d–f) (cited from Wold et al., 2001).

Detection of Quality and Safety of Fish

169

detection rate for the pale nematodes were increased to 60.3%, compared to only 46% of the detection rate in the study of Sivertsen et al. (2011). This is the good industrial application as the slow moving conveyor belt (400 mm/s) meets industrial required speed of assessment while the inspection rate is comparable or better to that for manual inspection under industrial conditions (Sivertsen et al., 2012).

12.6  MICROBIAL SPOILAGE INSPECTION Total viable count (TVC) of salmon muscle during spoilage process was detected by HSI with the region of 400–1,700 nm. The prediction was established using PLSR and LS-SVM models. Eight important wavelengths (495, 535, 550, 585, 625, 660, 785, and 915 nm) were selected using CARS algorithm. Encouragingly, TVC prediction of salmon fillet muscle achieved a satisfied result: with Rp2 of 0.96 and residual predictive deviation of 5.13 (Wu and Sun, 2013b). Similarly, HSI in the range of 400–1,000 nm was applied to evaluate the microbial spoilage of fish fillets (Cheng and Sun, 2015b). The authors applied successive projection algorithm (SPA) to select the optimal wavelengths, and the model established based on the seven selected wavelengths presented the equivalent prediction effectiveness and accuracy (Rp2 = 0.90) compared with the models based on the original models (Rp2 = 0.93) (Cheng and Sun, 2015b). The selected optimal wavelengths were useful for developing the multispectral imaging system, simplifying the data handled in the high dimension of HSI, and were more suitable for industrial online application (Cheng and Sun, 2015b). Similarly, the total counts of Enterobacteriaceae and Pseudomonas spp. on edible salmon flesh were detected by using NIR HSI (900–1,700 nm) (He and Sun, 2015b). Nine important wavelengths (i.e., 931, 1,138, 1,175, 1,242, 1,359, 1,628, 1,641, 1,652, and 1,655 nm) were selected from the reflectance spectra. The correlation coefficients of prediction (Rp2) of PLSR model using important wavelengths of reflectance, absorbance, and Kubelka-Munck can achieve as high as 0.95, 0.96, and 0.93, respectively. The accurate prediction of harmful microbial contamination is vital for warranting fish quality and safety. A multispectral imaging system, which is comparably lower cost, could be designed based on the selected wavelengths and could be refined further for online applications.

12.7 CONCLUSIONS This chapter addresses the progress made in recent years in the development of HSI technology for the fish industry. Both quality attributes and safety properties have been extensively discussed. Although HSI is a promising and sound technique that can rapidly detect the safety and quality of various seafood commodities without destructing the food structure, it still faces some inherent limits in terms of its application, complex data handling, and initial investment. Although many research studies have been done in relation to evaluation of quality, nutrition, and authenticity (fresh, freeze, thaw-frozen) of the seafood, the application of HSI to identify protein content in seafood products is lacking. As fish is one of the most important quality protein sources to provide 40% of protein intake approximately in our diet, relevant studies including prediction of some individual protein or amino acids merit attention. Also, limit researches about pigments prediction of seafood products by using

170

Chao-Hui Feng, et al.

HSI have not been exploited yet. More studies are considered to be carried out in this area to fill the gap of this domain. HSI integrates the information of spectroscopy and imaging simultaneously, which provides spectral and spatial information at the same time. As a rapid, noninvasive, environmental friendly technique, HSI can be widely applied for fish and fish products. More research is needed to elucidate the quality evolution during storage using HSI technique.

ACKNOWLEDGMENTS Chao-Hui Feng wishes to thank for the financial support of her research work under the Japan Society for the Promotion of Science (No. P16104) and a Grant-in-Aid for Scientific Research (JSPS No. 16F16104), National Natural Science Foundation of China (No. 31501550), Natural Science Foundation of Sichuan Provincial Department of Education (No. 16ZA0033), Talent Project from Sichuan Agricultural University (No. 03120301), and the key research project (Grant No. 15-R06) funded by the Meat Processing Key Laboratory of Sichuan Province sponsored by Chengdu University. The authors would also like to thank the anonymous reviewers for their constructive comments.

REFERENCES Berger, S. A., & Marr, J. S. (2006). Human Parasitic Diseases Sourcebook. Jones and Bartlett Publishers: Sudbury, MA. Cheng, J. H., Dai, Q., Sun, D.-W., Zeng, X.-A., Liu, D., & Pu, H.-B. (2013). Applications of non-destructive spectroscopic techniques for fish quality and safety evaluation and inspection. Trends in Food Science & Technology, 34, 18–31. Cheng, J. H., & Sun, D. W. (2014). Hyperspectral imaging as an effective tool for quality analysis and control of fish and other seafoods: current research and potential applications. Trends in Food Science & Technology, 37, 78–91. Cheng, J. H., & Sun, D.-W. (2015a). Data fusion and hyperspectral imaging in tandem with least squares-support vector machine for prediction of sensory quality index scores of fish fillet. LWT-Food Science and Technology, 63, 892–898. Cheng, J. H., & Sun, D.-W. (2015b). Rapid and non-invasive detection of fish microbial spoilage by visible and near infrared hyperspectral imaging and multivariate analysis. LWT-Food Science and Technology, 62, 1060–1068. Cheng, J. H., Sun, D.-W., Pu, H. B., Wang, Q.-J., & Chen, Y.-N. (2015a). Suitability of hyperspectral imaging for rapid evaluation of thiobarbituric acid (TBA) value in grass carp (Ctenopharyngodon idella) fillet. Food Chemistry, 171, 258–265. Cheng, J. H., Sun, D.-W., Pu, H. B., & Zhu, Z. W. (2015b). Development of hyperspectral imaging coupled with chemometric analysis to monitor K value for evaluation of chemical spoilage in fish fillets. Food Chemistry, 185, 245–253. Cheng, J. H., Sun, D.-W., & Wei, Q. Y. (2017). Enhancing visible and near-infrared hyperspectral imaging prediction of TVB-N level for fish fillet freshness evaluation by filtering optimal variables. Food Analytical Methods, 10, 1888–1898. Cheng, J. H. Sun, D.-W., Zeng, X.-A., & Pu, H.-B. (2014). Non-destructive and rapid determination of TVB-N content for freshness evaluation of grass carp (Ctenopharyngodon idella) by hyperspectral imaging. Innovative Food Science & Emerging Technologies, 21, 179–187.

Detection of Quality and Safety of Fish

171

Crane, D. P., Killourhy, C. C., & Clapsadl, M. D. (2016). Effects of three frozen storage methods on wet weight of fish. Fisheries Research, 175, 142–147. Ehira, S. (1976). A biochemical study on the freshness of fish. Bulletin of Tokai Regional Fisheries Research Laboratory. ElMasry, G., Sun, D.-W., & Allen, P. (2012). Near-infrared hyperspectral imaging for predicting colour, pH and tenderness of fresh beef. Journal of Food Engineering, 110, 127–140. Feng, C. H., Drummond, L., & Sun, D.-W. (2014a). Modeling the growth parameters of lactic acid bacteria and total viable count in vacuum-packaged Irish cooked sausages cooled by different methods. International Journal of Food Science & Technology, 49, 2659–2667. Feng, C. H., Drummond, L., Sun, D.-W., & Zhang, Z. H. (2014b). Evaluation of natural hog casings modified by surfactant solutions combined with lactic acid by response surface methodology. LWT-Food Science and Technology, 58, 427–438. Feng, C. H., Drummond, L., Zhang, Z. H., & Sun, D.-W. (2013a). Effects of processing parameters on immersion vacuum cooling time and physico-chemical properties of pork hams. Meat Science, 95, 425–432. Feng, C. H., Drummond, L., Zhang, Z. H., & Sun, D.-W. (2014c). Evaluation of innovative immersion vacuum cooling with different pressure reduction rates and agitation for cooked sausages stuffed in natural or artificial casing. LWT-Food Science and Technology, 59, 77–85. Feng, C. H., García Martín, J. F., Li, C., Liu, B.-L., Song, X.-Y., Dong, Q.-L., Wang, W., & Yang, Y. (2016). Evaluation of physicochemical properties and microbial attributes of cooked sausages stuffed in casing modified by surfactants and lactic acid after immersion vacuum cooling and long-term storage. International Journal of Food Science & Technology, 51, 2270–2279. Feng, C. H., & Li, C. (2015). Immersion vacuum-cooling as a novel technique for cooling meat products: Research advances and current state-of-the art. Comprehensive Reviews in Food Science and Food Safety, 14, 785–795. Feng, C. H., Makino, Y., Oshita, S. & García-Martín, J. F. (2018). Hyperspectral imaging and multispectral imaging as the novel techniques for detecting defects in raw and processed meat products: Current state-of-the-art research advances. Food Control, 84, 165–176. Feng, C. H., & Sun, D.-W. (2014). Optimization of immersion vacuum cooling operation and quality of Irish cooked sausages by using response surface methodology. International Journal of Food Science & Technology, 49, 1850–1858. Feng, C. H., Sun, D.-W., García Martín, J. F., & Zhang, Z. H. (2013b). Effects of different cooling methods on shelf-life of cooked jumbo plain sausages. LWT-Food Science and Technology, 54, 426–433. He, H.-J. & Sun, D.-W. (2015a). Hyperspectral imaging technology for rapid detection of various microbial contaminants in agricultural and food products. Trends in Food Science & Technology, 46, 99–109. He, H.-J. & Sun, D.-W. (2015b). Inspection of harmful microbial contamination occurred in edible salmon flesh using imaging technology. Journal of Food Engineering, 150, 82–89. Heia, K., Sivertsen, A. H., Stormo, S. K., Elvevoll, E., Wold, J. P., & Nilsen, H. (2007). Detection of nematodes in cod (Gadus morhua) fillets by imaging spectroscopy. Journal of Food Science, 72, E011–E015.

172

Chao-Hui Feng, et al.

Heide, M., & Olsen, S. O. (2017). Influence of packaging attributes on consumer evaluation of fresh cod. Food Quality and Preference, 60, 9–18. Hernández-Martínez, M., Gallardo-Velázquez, T., Osorio- Revilla, G., Almaraz-Abarca, N., Ponce-Mendoza, A., & Vásquez-Murrieta, M. S. (2013). Prediction of total fat, fatty acid composition and nutritional parameters in fish fillets using MID-FTIR spectroscopy and chemometrics. LWT-Food Science and Technology, 52, 12–20. Ivorra, E., Sánchez, A. J., Verdú, S., Barat, J. M., & Grau, R. (2016). Shelf life prediction of expired vacuum-packed chilled smoked salmon based on a KNN tissue segmentation method using hyperspectral images. Journal of Food Engineering, 178, 110–116. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2013). Non-destructive assessment of instrumental and sensory tenderness of lamb meat using NIR hyperspectral imaging. Food Chemistry, 141, 389–396. Kamruzzaman, M., Makino, Y., & Oshita, S. (2015a). Non-invasive analytical technology for the detection of contamination, adulteration, and authenticity of meat, poultry, and fish: A review. Analytica Chimica Acta, 853, 19–29. Kamruzzaman, M., Makino, Y., & Oshita, S. (2015b). Hyperspectral imaging in tandem with multivariate analysis and image processing for non-invasive detection and visualization of pork adulteration in minced beef. Analytical Methods, 7, 7496–7502. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016a). Parsimonious model development for real-time monitoring of moisture in red meat using hyperspectral imaging. Food Chemistry, 196, 1084–1091. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016b). Online monitoring of red meat colour using hyperspectral imaging. Meat Science, 116, 110–117. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016c). Hyperspectral imaging for realtime monitoring of water holding capacity in red meat. LWT-Food Science and Technology, 66, 685–691. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016d). Rapid and non-destructive detection of chicken adulteration in minced beef using visible near-infrared hyperspectral imaging and machine learning. Journal of Food Engineering, 170, 8–15. Kamruzzaman, M., Makino, Y., Oshita, S., & Liu, S. (2015c). Assessment of visible nearinfrared hyperspectral imaging as a tool for detection of horsemeat adulteration in minced beef. Food Bioprocess Technology, 8, 1054–1062. Kuswandi, B., Jayus, J., Restyana, A., Abdullah, A., Heng, L. Y., & Ahmad, M. (2012). A novel colorimetric food package label for fish spoilage based on polyaniline film. Food Control, 25, 184–189. Love, R. M. (1962). Protein denaturation in frozen fish. VI.—Cold-storage studies on cod using the cell fragility method. Journal of the Science of Food and Agriculture, 13, 269–278. Lowe, T., Ryder, J., Carragher, J. F., & Wells, R. M. G. (1993). Flesh quality in snapper, pagrcrs auratus, affected by capture stress. Journal of Food Science, 58, 770–773. Ma, J., Sun, D.-W., Qu, J.-H., & Pu, H. B. (2017). Prediction of textural changes in grass carp fillets as affected by vacuum freeze drying using hyperspectral imaging based on integrated group wavelengths. LWT-Food Science and Technology, 82, 377–385. Montaño, N., Gavino, G., & Gavino, V. C. (2001). Polyunsaturated fatty acid contents of some traditional fish and shrimp paste condiments of the Philippines. Food Chemistry, 75, 155–158.

Detection of Quality and Safety of Fish

173

Ocano-Higuera, V. M., Marquez-Rios, E., Canizales-Davila, M., Castillo-Yanez, F. J., Pacheco-Aguilar, R., Lugo-Sanchez, M. E., Garcia-Orozco, K. D., & GracianoVerdugo, A. Z. (2009). Food Chemistry, 116, 933–938. Qu, J. H., Sun, D.-W., Cheng, J.-H., & Pu, H. B. (2017). Mapping moisture contents in grass carp (Ctenopharyngodon idella) slice under different freeze drying periods by Vis-NIR hyperspectral imaging. LWT-Food Science and Technology, 75, 529–536. Sánchez-Alonso, I., Carballeda-Sangiao, N., González-Muñoz, M., Navas, A., Arcos, S. C., Mendizábal, A., Teijada, M., & Careche, M. (2018). Pathogenic potential of Anisakis L3 after freezing in domestic freezers. Food Control, 84, 61–69. Segtnan, V. H., Høy, M., Sørheim, O. Kohiler, A., Lundby, F., Wold, J. P., & Ofstad, R. (2009). Noncontact salt and fat distributional analysis in salted and smoked salmon. Journal of Agricultural and Food Chemistry, 57, 1705–1710. Sivertsen, A. H., Heia, K., Hindberg, K., & Godtliebsen, F. (2012). Automatic nematode detection in cod fillets (Gadus morhua L.) by hyperspectral imaging. Journal of Food Engineering, 111, 675–681. Sivertsen, A. H., Heia, K., Stormo, S. K., Elvevoll, E., & Nilsen, H. (2011). Automatic nematode detection in cod fillets (Gadus Morhua) by transillumination hyperspectral imaging. Journal of Food Science, 76, S77–S83. Skjelvareid, M. H., Heia, K., Olsen, S. H. & Stormo, S. K. (2017). Detection of blood in fish muscle by constrained spectral unmixing of hyperspectral images. Journal of Food Engineering, 212, 252–261. Sone, I., Olsen, R. L., Sivertsen, A. H., Eilertsen, G., & Heia, K. (2012). Classification of fresh Atlantic salmon (Salmo salar L.) fillets stored under different atmospheres by hyperspectral imaging. Journal of Food Engineering, 109, 482–489. Troy, D. J., & Kerry, J. P. (2010). Consumer perception and the role of science in the meat industry. Meat Science, 86, 214–226. Uauy, R., Mena, P., Wegher, B., Nieto, S., & Salem, N. J. (2000). Long chain polyunsaturated fatty acid formation in neonates: Effect of gestational age and intrauterine growth. Pediatric Research, 47, 127–135. Washburn, K. E., Stormo, S. K., Skjelvareid, M. H., & Heia, K. (2017). Non-invasive assessment of packaged cod freeze-thaw history by hyperspectral imaging. Journal of Food Engineering, 205, 64–73. Whitworth, M. B., Millar, S. J., & Chau, A. (2010). Food quality assessment by NIR hyperspectral imaging. Proc. SPIE 7676, Sensing for Agriculture and Food Quality and Safety II, 7676, 1–12. https://doi.org/10.1117/12.852170 Wu, D., & Sun, D.-W. (2013a). Application of visible and near infrared hyperspectral imaging for non-invasively measuring distribution of water-holding capacity in salmon flesh. Talanta, 116, 266–276. Wu, D., & Sun, D.-W. (2013b). Potential of time series-hyperspectral imaging (TS-HSI) for non-invasive determination of microbial spoilage of salmon flesh. Talanta, 111, 39–46. Wu, D., Sun, D.-W., & He, Y. (2012). Application of long-wave near infrared hyperspectral imaging for measurement of colour distribution in salmon fillet. Innovative Food Science & Emerging Technologies, 16, 361–372. Wold, J. P., Westad, F., & Heia, K. (2001). Detection of parasites in cod fillets by using SIMCA classification in multispectral images in the visible and NIR region. Applied Spectroscopy, 55, 1025–1034. Xu, J.-L., Riccioli, C., & Sun, D.-W. (2015). An overview on non-destructive spectroscopic techniques for lipid and lipid oxidation analysis in fish and fish products. Comprehensive Reviews in Food Science and Food Safety, 14, 466–477.

174

Chao-Hui Feng, et al.

Xu, J.-L., Riccioli, C., & Sun, D.-W. (2016a). Development of an alternative technique for rapid and accurate determination of fish caloric density based on hyperspectral imaging. Journal of Food Engineering, 190, 185–194. Xu, J.-L., Riccioli, C., & Sun, D.-W. (2016b). Efficient integration of particle analysis in hyperspectral imaging for rapid assessment of oxidative degradation in salmon ­fillet. Journal of Food Engineering, 169, 259–271.

Chapter

13

Applications of Hyperspectral Imaging for Meat Quality and Authenticity Mohammed Kamruzzaman Bangladesh Agricultural University

CONTENTS 13.1 Introduction 175 13.2 Overview of the Chapter 176 13.3 Application of Hyperspectral Imaging for Red Meat Quality 177 13.3.1 Real-Time Multispectral Imaging System for Predicting Color Parameters in Red Meat179 13.3.2 Real-Time Multispectral Imaging System for Predicting Water Holding Capacity in Red Meat180 13.3.3 Real-Time Multispectral Imaging System for Predicting Water Holding Capacity in Red Meat181 13.3.3.1 Hyperspectral Imaging for Predicting Quality and Authenticity of Lamb Meat183 13.3.3.2 Application of Hyperspectral Imaging for Adulteration Detection in Minced Meat185 13.3.3.3 Adulteration Detection in Minced Lamb185 13.3.3.4 Hyperspectral Imaging for Adulteration Detection in Beef 187 13.4 Conclusions 188 References 189

13.1 INTRODUCTION Meat is a global product and it is traded between regions, countries, and continents. The meat-processing industry is one of the largest agricultural and food-processing industries in the world. Red meat such as beef, lamb, and pork are the most important food products throughout the world from the perspective of the human diet and commercial activity in both developing and developed countries (Verbeke et al., 2011). Consumers are the driving force in the meat market and their expectation for meat quality and safety is increasing with access to information and life style changes. For this reason, determination of quality characteristics in raw and processed red meat is very important to facilitate the supply of consistently superior quality meat and meat products at an affordable price to

175

176

Mohammed Kamruzzaman

the consumer (ElMasry et al., 2012a). Indeed, most consumers are willing to pay a higher price for quality meat. Basically, high quality is a key element for modern meat industry in today’s hypercompetitive marketplace. Therefore, to meet consumer demand, it is very important that meat is provided to fulfill market requirements and customer needs. To realize these needs, it is crucial for the meat industry to assess meat quality attributes by deploying modern techniques to facilitate quick, accurate, and easy measurements. All of these concerns have promoted the development of objective measurement systems in the meat industry. In particular, there is a great interest in developing nondestructive optical sensing technologies that have the capability of monitoring quality attributes and safety parameters in real-time assessment (Kamruzzaman et al., 2015a). Recently, hyperspectral imaging has been introduced to integrate both spectroscopic and imaging techniques in one system for providing both spectral and spatial information simultaneously. Hyperspectral imaging techniques provide spatial information, as regular imaging systems, along with spectral information for each pixel in the image. This information then forms a three-dimensional (3D) data which can be analyzed to detect, identify, and quantify the physical and chemical features of the imaged objects and to characterize the products in more detail than is possible using the ordinary imaging or spectroscopy techniques. Each pixel in a hyperspectral image contains a spectrum (also called spectral signature or spectral fingerprint) representing the light absorbing and/or scattering properties of such a pixel. In essence, spectral signatures can be used to uniquely characterize, identify, and discriminate among classes/types of any given material(s) in the image. Hyperspectral imaging can be used to know the chemical composition, how much and where they are located in the sample under study. It has recently been accepted as one of the most powerful nondestructive imaging technologies for predicting quality and safety attributes in different meat species as well as for building chemical images to show the distribution maps of these constituents in a direct and easy way. In its ample applications, hyperspectral imaging technique has proved its potential for predicting quality, safety, and authenticity in beef (Naganathan et al., 2008a, b; ElMasry et al., 2011b, 2012b; Peng et al., 2011; Wu et al., 2012a), pork (Barbin et al., 2012a, b, c, d; Qiao et al., 2007; Tao et al., 2012; Wang et al., 2010), lamb (Kamruzzaman et al., 2011; 2012a, b, c; 2013a, b; Pu et al., 2014a, b), chicken, (Feng et al., 2013; Feng & Sun, 2013a, b; Park et al., 2011; Nakariyakul and Casasent, 2008, 2009), Turkey (ElMasry et al., 2011a; Iqbal et al., 2013a), and fish (ElMasry and Wold, 2008; He et al., 2015a, b; Sivertsen et al., 2011; Sone et al., 2012; Wu et al., 2012; Wu et al., 2014). The application of hyperspectral imaging in meat is huge. The application is still being investigated extensively to determine quality, safety, contamination, authenticity, and adulteration detection in meat and meat products. Although facing many challenges, hyperspectral imaging is still expected to become one of the most promising analytical tools in inspecting qualities of meat. It is expected that the technique will dominate in the future in quality evaluation, safety assessment, and process monitoring of food and agro-products.

13.2  OVERVIEW OF THE CHAPTER The application of hyperspectral imaging techniques for meat (beef, lamb, pork, chicken, and poultry) quality is huge, and all of these applications cannot be covered in a single chapter. Therefore, all of these applications are not discussed here. These have already been highlighted in several reviews and book chapters (Feng & Sun, 2012; Iqbal et al., 2013; Kamruzzaman and Sun, 2016; Kamruzzaman, 2016; Kamruzzaman et al., 2015d;

HSI for Meat Quality and Authenticity

177

Pu et al., 2015; Sun, 2010; Wu & Sun, 2013a; Xiong et al., 2014, 2015). In this chapter, some selected special issues are discussed. Recently, several researchers have investigated hyperspectral imaging as a precursor for selection of important wavelengths for designing online multispectral imaging instruments for quality (Barbin et al., 2012a; ElMasry et al., 2011a, b; Kamruzzaman et al., 2011; Pu et al., 2014a), safety (Barbin et al., 2012d; Feng et al., 2013), and authenticity (Kamruzzaman et al., 2011, 2012c) of red meat (beef, lamb, and pork). Surprisingly, different combinations of important wavelengths were selected for the same constituent in different types of red meat, although some of these studies were performed with the same system using identical reference and data analysis methods. Accordingly, for convenient industrial applications, it is desirable to conduct a comprehensive study combining all red meats for selecting important wavelengths to design real-time spectral imaging instruments for specific applications for the meat industry, instead of selecting different sets of important wavelengths for each red meat. Therefore, some applications on comprehensive study combining all red meats are discussed in this chapter. Although, lamb is an important constituent of red meat, the potential of hyperspectral imaging technology has not yet been sufficiently exploited for lamb meat quality. Therefore, in this chapter, the application of hyperspectral imaging for lamb meat quality is also discussed. Additionally, adulteration in minced meat has always been an international problem and there is a constant requirement for robust analytical methods to be developed in order to detect adulteration in minced meat and meat products. Therefore, this chapter also describes the potential of hyperspectral imaging for rapid authentication and detection of adulteration in minced meat. It is hoped that this will encourage the researchers to explore this emerging detection technology not only in meat but also in all agro-food products so that the consumer can benefited further from this emerging technology to determine quality and to detect adulteration.

13.3  APPLICATION OF HYPERSPECTRAL IMAGING FOR RED MEAT QUALITY Recently, various studies have investigated hyperspectral imaging for selecting effective wavelengths aimed at encouraging the manufacture of online multispectral imaging instruments for beef (ElMasry et al., 2011a; Wu et al., 2012), lamb (Kamruzzaman et al., 2011; 2012a; Pu et al., 2014a), pork (Barbin et al., 2012b, c; Tao et al., 2012), and chicken (Feng et al., 2013). In particular, one research group headed by Professor Sun investigated near infrared (NIR) hyperspectral imaging (900–1,700 nm) and proposed feature wavelengths for designing a multispectral imaging system for online monitoring of various quality attributes of beef (ElMasry et al., 2011b, 2012b), pork (Barbin et al., 2012b, c), and lamb (Kamruzzaman et al., 2013a, b). For instance, eight wavelengths (934, 1,048, 1,108, 1,155, 1,185, 1,212, 1,265, and 1,379 nm), six wavelengths (960, 1,057, 1,131, 1,211, 1,308, and 1,394 nm), and seven wavelengths (927, 950, 1,047, 1,211, 1,325, 1,513, and 1,645 nm) were selected for water content in beef (ElMasry et al., 2012c), lamb (Kamruzzaman et al., 2012b), and pork (Barbin et al., 2012c), ­respectively. The same situation has been observed for other meat quality attributes of beef, lamb, and pork (Barbin et al., 2012a, b; ElMasry et al., 2011, 2012b; Kamruzzaman, 2012a, c). Although all these studies were performed using a common system with an identical reference and data analysis method, surprisingly different combinations of optimum wavelengths were selected for the same constituent  in

178

Mohammed Kamruzzaman

different types of red meat. This implies that to develop a multispectral system, different combinations of wavelengths need to be used to determine the same attribute in different types of red meat, which is not convenient and feasible for the processors. To address this problem, Kamruzzaman et al. (2016a, b, c) conducted a comprehensive study combining all red meats for selecting feature wavelengths to design real-time spectral imaging instruments for specific applications for the meat industry, instead of selecting different sets of important feature wavelengths for each red meat. The authors used a visible near-infrared (VNIR) system in the spectral range of 400–1,000 nm for designing a multispectral system. The main components of this system, as shown schematically in Figure 13.1, are a 12-bit charge-coupled device (CCD) camera, a linescanning spectrograph coupled with a C-mount lens with fixed distance of 330 mm from sample surface, an illumination unit with one 50-W tungsten halogen and one xenon lamp adjusted at a 45° angle to illuminate the camera’s field of view, a translation stage driven by a stepping motor with a user-defined speed, and a computer with data acquisition software. Each sample was placed on a black background with very low reflectance to obtain good contrast between the sample and background and conveyed to the field of view of the camera to be scanned line by line to create a 3-D hypercube. The system scans the sample line by line at a constant speed of 1.08 cm/s. The movement of the translation stage is synchronized with the camera to obtain spectral images with a spatial resolution of 0.749 mm/pixel. The authors used this system for selecting feature wavelengths for designing real-time multispectral system for predicting color parameters, water holding capacity (WHC), and moisture content in red meats. The step-by-step procedure to select feature wavelengths for designing multispectral realtime imaging for color, WHC, and moisture content is depicted in Figure 13.2. These applications are described in Sections 13.3.1–13.3.3.

FIGURE 13.1  Schematic diagram of the hyperspectral imaging system (Kamruzzaman

et al., 2016a).

HSI for Meat Quality and Authenticity

Sample set

Image

Hyperspectral image

acquisition

Calibrated image

179

Extraction of raw spectral data

Region of interest identification Reference analysis

Anomalies or outliers detection

No Model validation

Good?

Good?

Yes

Building calibration model

Yes

No

Optimizing preprocessing tool

Yes Selecting feature wavelengths

Spectral preprocessing required?

No

No Calibration model at feature wavelengths Design of multispectral system

Good?

Feature wavelengths for online screening

Yes Yes

Model validation

Good?

No

FIGURE 13.2  Step-by-step procedure for selecting feature wavelengths for designing

multispectral real-time imaging system (Kamruzzaman et al., 2016c). 13.3.1 Real-Time Multispectral Imaging System for Predicting Color Parameters in Red Meat Color is an important factor that is commonly used as a quality index to the meat industry and meat science research. It is considered a fundamental physical property that reflects the correlation between physical, chemical, and sensorial indicators of meat and meat products (Iqbal et al., 2010). Generally, consumer first judge a meat from its color. Consequently, color plays a major role in quality assessment of meat (Wu & Sun, 2013b). Color has been reported to be one of the most important meat quality attributes and significantly influences purchasing decisions, because consumers use discoloration as an indication of lack of freshness and wholesomeness (Mancini & Hunt, 2005). Color is also important from the economic point of view as the industry loses money due to undesirable color (Hughes et al., 2014). Lightness information (L* value) also allows the detection of certain defects in meat such as DFD (dark, firm, and dry) and PSE (pale, soft exudative) (Warriss et al., 2006). Both DFD and PSE are unattractive to consumers. Therefore, rapid and accurate monitoring color is essential for the meat industry. Kamruzzaman et al. (2016a) developed an online monitoring system for red meat (beef, lamb, and pork) color (L*a*b*) in the meat industry by selecting a set of feature  wavelengths from a VNIR hyperspectral imaging system in the spectral range of 400–1,000 nm. A total of 132 samples including 44 each of beef, lamb, and pork were used for the  investigation. Out of 132 samples, 87 fresh samples (29 each) and 45 frozen samples (15 from each) were collected from the loin muscle (m. longissimus

180

Mohammed Kamruzzaman

thoracis et lumborum). Frozen samples were originated from different geographical regions (Australia, Canada, Mexico, New Zealand, and USA) to ensure that the model would apply both locally and globally. To ensure the credibility and reliability of wavelength selection to design a multispectral online imaging system, samples were collected from four different slaughter batches as well as from different quality grades to ensure wide variations in color values that may be found during routine sampling. Only six feature wavelengths (450, 460, 600, 620, 820, and 980 nm) were selected using SPA to predict all color parameters in all red meat for convenient industrial applications. A quantitative linear relationship was established between the spectral data and corresponding color parameters based on these six selected wavelengths using MLR. The following MLR quantitative functions were obtained for L*, a*, and b*, respectively, and these functions can be used to predict color parameters in an online system. These functions predicted L*, a*, and b* with coefficients of determination (Rp2) of 0.97, 0.84, and 0.82, and root mean square error of prediction of 1.72, 1.73, and 1.35, respectively. A multispectral vision system can be easily developed using only six band pass filters to monitor red meat color in the meat industry. If it can be properly optimized under industry standard conditions, a VNIR hyperspectral imaging system would be invaluable for real-time inspection of meat color and other quality parameters:





yˆ L* = 41.94 + 130.25 × λ450 − 307.91 × λ460 + 82.83 × λ600 (13.1) + 131.98 × λ620 − 88.76 × λ820 + 52.83 × λ980 yˆ a* = 15.86 − 141.05 × λ450 + 101.97 × λ460 − 52.71 × λ600 + 14.85 × λ620 − 19.05 × λ820 + 34.47 × λ980 yˆ b* = 2.83 − 193.05 × λ450 + 88.41 × λ460 + 2.10 × λ600 + 41.70 × λ620 − 41.81 × λ820 + 59.76 × λ980

(13.2)

(13.3)

where λ is the spectral reflectance profile with subscripts indicating the specific wavelengths and yˆ is the predicted values. Finally, distribution maps of the predicted color parameters L*, a*, and b* were generated by applying the corresponding MLR model to each pixel in the spectral image. Figure 13.3 shows examples of distribution maps of L*, a*, and b* on some tested samples. Although it was difficult to distinguish color differences among samples with the naked eye of meat samples, the multivariate models had the capacity to display the spatial distribution of color parameters in a simple visual form. The results obtained from this technique can be applied for automating the inspection and quality grading based on color values of red meat through the integration of efficient image-processing algorithms in industrial machine-vision systems. 13.3.2 Real-Time Multispectral Imaging System for Predicting Water Holding Capacity in Red Meat Another important red meat quality attribute is water holding capacity. Water holding capacity (WHC) is the ability of the postmortem muscle (meat) to retain all or part of its own water even though external pressures are applied to it. WHC is economically

HSI for Meat Quality and Authenticity

181

FIGURE 13.3  Prediction maps of L*a*b* for beef, lamb, and pork. The value in the

­bottom of each prediction map represents the average value of the sample (Kamruzzaman et al., 2016a).

important for the meat processing industry as meat is sold by weight. Therefore, controlling WHC is very important for the meat industry to maximize yield and quality. For designing multispectral real-time imaging system for predicting WHC in red meat, Kamruzzaman et al. (2016b) explored two popular and widely used variable ­selection strategies such as RCs and CARS. The distribution of these selected feature wavelengths in the whole spectral range of 400–1,000 nm is shown in Figure 13.4. Eight wavelengths were selected using RCs (545, 610, 705, 765, 805, 900, 940, and 970 nm) and CARS (485, 545, 640, 670, 705, 765, 770, and 800 nm). It is clear that CARS identifies relevant wavelengths in the spectral range up to 800 nm whereas RCs tend to spread relevant wavelengths across the entire spectral range. Based on these feature wavelengths, PLSR and LS-SVM models were developed. Although the number of variables was substantially reduced from 121 to 8, the performances of the new models at selected feature wavelengths were tolerably degraded to those models developed using the full spectral range of 121 variables, indicating that the methods for wavelength selection using RCs and CARS were effective. Although both PLSR and LS-SVM models obtained good results based on reduced spectra in both calibration and prediction conditions for predicting WHC in red meat, it is confirmed that the LS-SVM models displayed a better performance than that of PLSR models. Therefore, the best LS-SVM model (RCs-LS-SVM) was subsequently applied to each pixel in the image to obtain prediction maps of WHC as shown in Figure 13.5 (Table 13.1). 13.3.3 Real-Time Multispectral Imaging System for Predicting Water Holding Capacity in Red Meat Moisture is one of the most important chemical parameters in meat and meat products. Moisture accounts for more than 70% of muscle composition in meat, and controlling this parameter is important in meat preservation processes such as curing, smoking, and drying. Moisture content profoundly influences meat quality and safety. It also has a strong relationship with lipids that affect the eating quality of meat, as it may affect

182

Mohammed Kamruzzaman

FIGURE 13.4  Selection of feature wavelengths using RCs (top) and CARS (bottom). In

CARS, the selected feature wavelengths are indicated by vertical lines on top of a representative spectrum. In the RCs process, the black vertical line indicates the selected feature wavelengths (Kamruzzaman et al., 2016b).

FIGURE 13.5  Prediction maps of WHC for (a) beef, (b) lamb, and (c) pork. The value in the bottom of each prediction map represents the average value of WHC of the sample (Kamruzzaman et al., 2016b).

characteristics such as flavor, juiciness, texture, and appearance (Weeranantanaphan et al., 2011). Moisture is a key factor that affects microbial growth, resulting in influencing the shelf life of meat. Thus, accurate monitoring of moisture content is essential for the meat industry. Robust analytical methods having the capability of real-time assessment are required in the meat industry for continuous monitoring of meat moisture content.

HSI for Meat Quality and Authenticity

183

TABLE 13.1  Calibration and Prediction Statistics for Predicting WHC Using PLSR and LS-SVM Based on Full Wavelengths Range (121 Variables) and Feature Wavelengths Selected By RCs (545, 610, 705, 765, 805, 900, 940, and 970 nm) and CARS (485, 545, 640, 670, 705, 765, 770, and 800 nm) (Kamruzzaman et al., 2016b) Calibration

Validation

Model

LVs

Rc

RMSEC (%)

Rp

RMSEP (%)

RPD

RER

PLSR LS-SVM CARS-PLSR CARS-LS-SVM RC-PLSR RC-LS-SVM

10 / 5 / 7 /

0.94 0.97 0.91 0.93 0.90 0.94

0.52 0.36 0.67 0.56 0.68 0.53

0.92 0.94 0.89 0.91 0.90 0.93

0.63 0.50 0.74 0.63 0.72 0.56

3.63 4.58 3.09 3.63 3.18 4.09

13.23 16.67 11.26 13.63 11.57 14.88

2

2

The best model indicated in bold for both full and selected wavelengths.

For designing multispectral real-time imaging for predicting moisture content in red meat, Kamruzzaman et al. (2016c) used the weighted regression coefficients (BW) resulting from the best PLSR model for selection of feature-related wavelengths. Variables with large regression coefficients (irrespective of sign) play an important role in the regression model. By means of this approach, only ten feature wavelengths (440, 480, 575, 620, 655, 680, 725, 780, 955, and 980 nm) were selected from the full spectral range of 121 wavelengths. MLR model was then developed with these ten bands and the following MRL function was obtained. The MLR model showed good performance in predicting moisture content, with Rp2 of 0.97, RMSEP of 2.19%, and RPD of 4.04 even though 91.74% (from 121 to 10) of the variables were eliminated. yˆ = 57.64 + 352.61 × λ440 − 302.50 × λ480 + 152.95 × λ575 + 37.11 × λ620

+ 104.60 × λ655 − 330.23 × λ680 + 99.92 × λ725 + 67.91 × 780

(13.4)

+ 549.46 × λ955 − 557.46 × λ980 where λ is the spectral reflectance profile with subscripts indicating the specific wavelengths and yˆ is the predicted moisture content (%). The MLR model (equation 13.4) was applied to each pixel of the image to predict moisture content over the full surface of the meat sample. Figure 13.6 shows the distribution maps of moisture content. The power of these distribution maps resides in the rapid and easy access they afford to the spatial distribution of moisture content and their relative concentrations as indicated in the color bar. The difference in moisture content from sample to sample even within the same sample was very interesting and easily visualized in the concentration maps. The distribution maps of moisture contents obtained in this study reveal advantages of hyperspectral imaging that cannot be achieved by either conventional imaging or conventional spectroscopy alone. 13.3.3.1 Hyperspectral Imaging for Predicting Quality and Authenticity of Lamb Meat Although lamb is an important constituent among red meat, the potential of hyperspectral imaging has not yet been sufficiently exploited for lamb meat quality attributes. One research group headed by Professor Da-Wen Sun first explored the potential of hyperspectral imaging for lamb meat quality. Hyperspectral imaging technique was evaluated

184

Mohammed Kamruzzaman

FIGURE 13.6  Prediction maps of moisture content for (a) beef, (b) lamb, and (c) pork.

The value in the bottom of each prediction map represents the average percentage of moisture content of the sample (Kamruzzaman et al., 2016c). as a tool for predicting quality and authenticity in lamb meat including muscles discrimination, prediction of pH, color, drip loss, chemical composition (i.e., water, fat, and protein), and tenderness (i.e., instrumental and sensory), identification and authentication of lamb meat from other red meats. The authors used hyperspectral imaging system in the NIR range (900–1,700 nm) for evaluating quality and authenticity of lamb meat. The system consists of a 12-bit CCD camera, a spectrograph, a standard C-mount lens, an illumination unit of two 500-W tungsten halogen lamps, a translation stage, and a computer supported with data acquisition software. The area CCD array detector of the camera has 320 × 256 (spatial × spectral) pixels, and the spectral resolution spectral resolution was 6 nm. The system scans the sample line by line at a constant speed of 2.8 cm/s. The movement of the translation stage is synchronized with the camera to obtain spectral images with a spatial resolution of 0.58 mm/pixel. Initial investigation was carried to discriminate three types of lamb muscles from different anatomical locations. Principal component analysis (PCA) was used for dimensionality reduction, wavelength selection, and visualizing hyperspectral data. The results showed that it was possible to discriminate lamb muscles with overall accuracy of 100% using NIR hyperspectral reflectance spectra (Kamruzzaman et al., 2011). Later, partial least-squares regression (PLSR) models were developed to correlate the NIR reflectance spectra with some quality attributes of the tested muscles. The models performed well for predicting pH, color, and drip loss with the coefficient of determination (Rcv2) of 0.65, 0.91, and 0.77, respectively (Kamruzzaman et al., 2012a). In another study, PLSR models were also developed for prediction of chemical composition in lamb meat. The developed models performed well for predicting water, fat, and protein with determination coefficient (Rp2) of 0.88, 0.88, and 0.63, respectively (Kamruzzaman et al., 2012b). NIR hyperspectral imaging was also evaluated for predicting instrumental and sensory tenderness in lamb meat. It was found that NIR hyperspectral imaging combined with PLSR could be used to determine lamb meat tenderness with a reasonable accuracy (Rcv = 0.84 for WBSF and 0.69 for sensory tenderness). Moreover, the study also confirmed that NIR hyperspectral imaging could become an interesting screening tool to quickly categorize lamb steaks into good (i.e., tender) and bad (i.e., tough) grades based on WBSF values and sensory scores with overall accuracy about 94.51% and 91.00%, respectively (Kamruzzaman et al., 2013a). Among the quality attributes tested (pH, color, drip loss, tenderness, water, fat, and protein); the best prediction results were obtained for

HSI for Meat Quality and Authenticity

185

TABLE 13.2  Optimum Wavelengths Selected Through Hyperspectral Data Analysis for Different Applications in Lamb Meat Application

Methods

Muscle discrimination L* value measurement

PCA PLS

Prediction of water Prediction of fat Prediction of protein Red meat discrimination WBSF measurement

PLS PLS PLS 2nd D SPA

Selected Wavelengths (nm) 934, 974, 1,074, 1,141, 1,211, 1,308 940, 980, 1,037, 1,104, 1,151, 1,258, 1,365, 1,418 960, 1,057, 1,131, 1,211, 1,308, 1,394 960, 1,057, 1,131, 1,211, 1,308, 1,394 1,008, 1,211, 1,315, 1,445, 1,562, 1,649 957, 1,071, 1,121, 1,144, 1,368, 1,394 934, 964, 1,017, 1,081, 1,144, 1,215, 1,265, 1,341, 1,455, 1,615, 1,655

No  6  8  6  6  6  6 11

predicting water, fat, and L* values than those for pH, drip loss, tenderness, and protein. However, the models for pH, drip loss, tenderness, and protein could be enhanced with inclusion of more samples to encompass more variability in the measured reference data. Indeed, it was possible to enhance the model for protein prediction (Rp2 = 0.63 vs. 0.85) by creating wider ranges in the reference data (Kamruzzaman et al., 2012b). The NIR hyperspectral imaging was also very effective for categorization and authentication of longissimus dorsi muscle of pork, beef, and lamb with 98.67% overall classification accuracy in the validation sets (Kamruzzaman et al., 2013b). Interestingly, the model developed with intact samples was applied successfully to detect and identify pork, beef, and lamb when meat is minced as shown in Figure 13.7 with a high degree of accuracy (69 out of 70 identified). The authors also addressed the most important challenge to develop and implement hyperspectral imaging system for online monitoring of meat quality by selecting some optimum wavelengths. These optimum wavelengths are summarized in Table 13.2. 13.3.3.2 Application of Hyperspectral Imaging for Adulteration Detection in Minced Meat Food adulteration is a common practice all over the world. Major food adulteration events appear to regularly occur (Ellis et al. 2012). In 2013, the horsemeat scandal in UK and Ireland drew huge attention to the issue of meat adulteration, both locally and globally. The shocking issue involved meat labeled as beef was found to contain undeclared horsemeat as well as meat from pork. Such adulteration not only constitutes consumer fraud and commercial malpractice, but there is also concern with individuals allergic to particular meat species, or those with religious taboos or ethical aversions (Zhao et al., 2014). Despite being a major concern for researchers, consumers, industries, and regulatory agencies at all levels of the production process, it is difficult to detect without resorting to highly sophisticated analytical techniques. Hyperspectral imaging has the potential to fight against adulteration. 13.3.3.3 Adulteration Detection in Minced Lamb The possibility of developing a hyperspectral imaging in the spectral rang of 900–1,700 nm was investigated for the first time by (Kamruzzaman et al., 2013b) to detect the level of adulteration in minced lamb meat. To identify the most potential adulterant in lamb meat, a total of 200 samples (5 classes × 40 samples from each class) were prepared

186

Mohammed Kamruzzaman

FIGURE 13.7  RGB images and corresponding classification maps of independent testing

set containing both intact and minced meat samples (Kamruzzaman et al., 2013b). using pure minced lamb meat and lamb meat mixed with a range of potential adulterants including pork, heart, kidney, and lung in 20% w/w proportions. They performed a PCA and identified pork, among others (heart, kidney, and lung), as the most potential adulterant in minced lamb. Minced lamb meat samples were then adulterated with minced pork in the range 2–40% (w/w) at approximately 2% increments. PLSR model

HSI for Meat Quality and Authenticity

187

FIGURE 13.8  RGB images (top) and corresponding prediction maps (bottom) of adulter-

ation at different levels from 4% to 40% (left to right) with 4% increments. RGB images were synthesized by combining calibrated hyperspectral images at the wavelengths of 950, 1,250 and 1,300 nm (Kamruzzaman et al., 2013b). was developed to predict the level of pork adulteration. Good PLSR model was obtained using the whole spectral range (910–1,700 nm) with a coefficient of determination (Rcv2) of 0.99 and root-mean-square errors estimated by cross validation (RMSECV) of 1.37%. Out of 237 wavelengths, only four feature wavelengths (940, 1,067, 1,144 and 1,217 nm) were selected using weighted RC of PLSR and a MLR model with Rcv2 = 0.98 was developed. Finally, the developed MLR model was applied to each pixel in the image to obtain prediction maps to visualize the distribution of the level of adulteration in the samples. The distribution maps for some tested quality attributes are presented in Figure 13.8. Although it was difficult to identify the difference in the adulteration level with the naked eye from the RGB images, the generated prediction map clearly revealed the change in adulteration from sample to sample and even from spot to spot within the same sample. 13.3.3.4 Hyperspectral Imaging for Adulteration Detection in Beef A hyperspectral imaging system was used in the spectral range of 400–1,000 nm with 5nm intervals to detect horsemeat (Kamruzzaman et al., 2015c), pork (Kamruzzaman et al., 2016d), and chicken (Kamruzzaman et al., 2015b) adulteration in minced beef. The minced beef samples were separately adulterated by mixing horsemeat, chicken, and pork in the range of 2%–50% (w/w), at approximately 2% increments. The minced beef and adulterate (i.e., horsemeat, chicken, or pork) were individually weighed and thoroughly mixed and homogenized to obtain a total sample weight of 32 g. The minced meat was placed in a circular metal can and imaged using the hyperspectral system. For predicting horsemeat adulteration in minced beef, PLSR models were developed with raw and pre-treated spectra (MSC, SNV, first derivative, and second derivative). PLSR models developed using the raw spectra were very accurate with R c2 of 0.99, SEC of 1.14%, Rcv2 of 0.99, and SECV of 1.56%. When this developed model was applied to an independent validation set, the level of adulteration in mince beef was predicted with an Rp2 of 0.98 and SEP of 2.23%. To develop optimized PLSR model, four important

188

Mohammed Kamruzzaman

FIGURE 13.9  RGB images (top) and corresponding prediction maps (bottom) of adulteration at different levels from 2% to 50% (left to right) with 6% increments. RGB images were synthesized by combining calibrated hyperspectral images at the wavelengths of 660, 580, and 455 nm (Kamruzzaman et al., 2015c).

wavelengths at 515, 595, 650, and 880 nm were selected corresponding to the highest absolute values of BW. By using only these particular wavelengths, new PLSR models were created. The new PLSR model had a good performance in predicting adulteration with Rc2 of 0.99, SEC of 1.21%, Rcv2 of 0.99, and SECV of 1.60%. This PLSR model, when applied to an independent validation set, was capable of predicting the level of adulteration with Rp2 of 0.98 and SEP of 2.20%. The optimized model was transferred in each pixel of the image to create prediction maps or distribution maps. Figure 13.9 shows the prediction maps of some samples with their corresponding RGB images. It was quite clear to aptly figure out how the levels of adulteration vary from sample to sample and even within the same sample. Similar results were obtained for predicting pork and chicken adulteration in minced beef. The interested readers are referred to the published articles for the detail modeling results (Kamruzzaman et al., 2015b, 2016d). In short, for predicting pork adulteration in minced beef, only four wavelengths centered at 430, 605, 665, and 705 nm were selected as the important wavelengths to build an MLR model. The MLR model had a good performance with a R c of 0.992, a SEC of 1.831%, a Rp of 0.985, and a SEP of 4.172%. Although the variable numbers needed for prediction were substantially reduced from 117 to 4, however, the prediction ability of MLR model with only four important wavelengths was better than the original PLSR or PCR models with the full 117 wavelengths (Kamruzzaman et al., 2016d). For predicting chicken adulteration in minced beef, PLSR model was developed to relate the absorbance spectral profile with the adulteration levels of the tested samples (Kamruzzaman et al., 2015b). The PLSR model was then validated using different independent data sets, and obtained the coefficient of determination (Rp2) of 0.97, with root mean square error in prediction (RMSEP) of 2.45 (w/w). To reduce the high dimensionality of the hyperspectral data, five important wavelengths (555, 595, 670, 920, and 980 nm) were selected using stepwise regression. PLSR model was again created using these important wavelengths and the model (Rp2 of 0.97, RMSEP of 2.61%, and RPD of 5.86) was then transferred in each pixel in the image to obtain prediction map.

13.4 CONCLUSIONS The various applications described here confirm that hyperspectral imaging provide an alternative solution for meat quality, safety, authenticity, and adulteration detection.

HSI for Meat Quality and Authenticity

189

Therefore, the laborious and time-consuming conventional analytical techniques could be replaced by spectral data to provide a rapid and nondestructive testing technique. Because of the combined efforts of academia and industry, tremendous developments and remarkable improvements have been made during the last decade in hyperspectral imaging technology. Although hyperspectral imaging technology is currently suffering from some drawbacks, the future development of hyperspectral imaging instruments such as lower purchase costs and improved processing speed along with progress in data analysis techniques will lead this technology to more substantial and have widespread applications in the future. Once the technology is optimized and developed for real-time sensing, the system can be easily installed over the conveyor belt with the minimum modifications of the existing industrial set-up with little financial or manpower investment.

REFERENCES Barbin, D., ElMasry, G., Sun, D.-W., & Allen, P. (2012a). Near-infrared hyperspectral imaging for grading and classification of pork. Meat Science, 90, 259–268. Barbin, D., ElMasry, G., Sun, D.-W., & Allen, P. (2012b). Predicting quality and sensory attributes of pork using near-infrared hyperspectral imaging. Analytica Chimica Acta, 719, 30–42. Barbin, D. F., ElMasry, G., Sun, D.-W., & Allen, P. (2012c). Non-destructive determination of chemical composition in intact and minced pork by near-infrared hyperspectral imaging. Food Chemistry, 138, 1162–1171. Barbin, D. F., ElMasry, G., Sun, D.-W., Allen, P., & Noha, M. (2012d). Non-destructive assessment of microbial contamination in porcine meat using NIR hyperspectral imaging. Innovative Food Science & Emerging Technologies, 17, 180–191. Ellis, D. I., Brewster, V. L., Dunn, W. B., Allwood, J. W., Golovanov, A. P., & Goodacre, R. (2012). Fingerprinting food: Current technologies for the detection of food ­adulteration and contamination. Chemical Society Review, 41, 5706–5727. ElMasry, G., Iqbal, A., Sun, D.-W., Allen, P., & Ward, P. (2011a). Quality classification of cooked, sliced turkey hams using NIR hyperspectral imaging system. Journal of Food Engineering, 103, 333–344. ElMasry, G., Kamruzzaman, M., Sun, D.-W., & Allen, P. (2012a). Principles and applications of hyperspectral imaging in quality evaluation of agro-food products: A review. Critical Reviews in Food Science and Nutrition, 52, 999–1023. ElMasry, G., Sun, D.-W., & Allen, P. (2011b). Non-destructive determination of waterholding capacity in fresh beef by using NIR hyperspectral imaging. Food Research International, 44, 2624–2633. ElMasry, G., Sun, D.-W., & Allen, P. (2012b). Near-infrared hyperspectral imaging for predicting colour, pH and tenderness of fresh beef. Journal of Food Engineering, 110, 127–140. ElMasry, G., Sun, D.-W. & Allen, P. (2012c). Chemical-free assessment and ­mapping of major constituents in beef using hyperspectral imaging. Journal of Food Engineering, 117, 235–246. ElMasry, G., & Wold, J. P. (2008). High-speed assessment of fat and water content distribution in fish fillets using online imaging spectroscopy. Journal of Agricultural and Food Chemistry, 56, 7672–7677.

190

Mohammed Kamruzzaman

Feng, Y.-Z., ElMasry, G., Sun, D.-W., Scannell, A. G. M., Walsh, D., & Morcy, N. (2013). Near-infrared hyperspectral imaging and partial least squares regression for rapid and reagentless determination of Enterobacteriaceae on chicken fillets. Food Chemistry, 138, 1829–1836. Feng, Y.-Z., & Sun, D.-W. (2012). Application of hyperspectral imaging in food safety inspection and control, a review. Critical Reviews in Food Science & Nutrition, 52(11), 1039–1058. Feng, Y.-Z., & Sun, D.-W. (2013a) Near-infrared hyperspectral imaging in tandem with partial least squares regression and genetic algorithm for non-destructive determination and visualization of Pseudomonas loads in chicken fillets. Talanta, 109, 74–83. Feng, Y.-Z., & Sun, D.-W. (2013b). Determination of total viable count (TVC) in chicken breast fillets by near-infrared hyperspectral imaging and spectroscopic transforms. Talanta, 105, 244–249. He, H.-J., & Sun, D.-W. (2015a). Toward enhancement in prediction of pseudomonas counts distribution in salmon fillets using NIR hyperspectral imaging. LWT-Food Science and Technology, 62, 11–18. He, H.-J., & Sun, D.-W. (2015b). Inspection of harmful microbial contamination occurred in edible salmon flesh using imaging technology. Journal of Food Engineering, 150, 82–89. Hughes, J. M., Oiseth, S. K., Purslow, P. P., & Warner, R. D. (2014). A structural approach to understanding the interactions between colour, water-holding capacity and tenderness. Meat Science, 98, 520–532. Iqbal, A., Sun, D.-W., & Allen, P. (2013a). Prediction of moisture, color and pH in cooked, pre-sliced turkey hams by NIR hyperspectral imaging system. Journal of Food Engineering, 117, 42–51. Iqbal, A., Sun, D.-W., & Allen, P. (2013b). An overview on principle, techniques and application of hyperspectral imaging with special reference to ham quality evaluation and control. Food Control, 46, 242–254. Iqbal, A., Valous, N. A., Mendoza, F., Sun, D.-W., & Allen, P. (2010). Classification of pre-sliced pork and Turkey ham qualities based on image colour and textural features and their relationships with consumer responses. Meat Science, 84, 455–465. Kamruzzaman, M. Food adulteration and authenticity. In Food Safety-Basic Concepts, Recent Issues, and Future Challenges. Edited by S. Jinap and S. Z. Iqbal. Springer International Publishing. 2016, 127–148. Springer, Germany. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2011). Application of NIR hyperspectral imaging for discrimination of lamb muscles. Journal of Food Engineering, 104, 332–340. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2012a). Prediction of some quality attributes of lamb meat using near infrared hyperspectral imaging and ­multivariate analysis. Analytica Chimica Acta, 714, 57–67. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2012b). Non-destructive prediction and visualization of chemical composition in lamb meat using NIR hyperspectral imaging and multivariate regression. Innovative Food Science and Emerging Technologies, 16, 218–226. Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2012c). Potential of hyperspectral imaging and pattern recognition for categorization and authentication of red meat. Innovative Food Science and Emerging Technologies, 16, 316–235.

HSI for Meat Quality and Authenticity

191

Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2013a). Non-destructive assessment of instrumental and sensory tenderness of lamb meat by NIR hyperspectral imaging. Food Chemistry, 141, 389–396. Kamruzzaman, M., Makino, Y., & Oshita, S. (2015a). Non-invasive analytical technology for the detection of contamination, adulteration, and authenticity of meat, poultry, and fish: A review. Analytica Chimica Acta, 853, 19–29. Kamruzzaman, M., Makino, Y., & Oshita, S. (2015b). Hyperspectral imaging in tandem with multivariate analysis and image processing for non-invasive detection and visualization of pork adulteration in minced beef. Analytical Methods, 7, 7496–7502. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016a). Online monitoring of red meat color using hyperspectral imaging. Meat Science, 116, 110–117. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016b). Hyperspectral imaging for realtime monitoring of water holding capacity in red meat. LWT-Food Science and Technology, 66, 685–691. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016c). Parsimonious model development for real-time monitoring of moisture in red meat using hyperspectral imaging. Food Chemistry, 196, 1084–1091. Kamruzzaman, M., Makino, Y., & Oshita, S. (2016d). Rapid and non-destructive detection of chicken adulteration in minced beef using visible near-infrared hyperspectral imaging and machine learning. Journal of Food Engineering, 170, 8–15. Kamruzzaman, M., Makino, Y., Oshita, S., & Liu, S. (2015c). Assessment of visible nearinfrared hyperspectral imaging as a tool for detection of horsemeat adulteration in minced. Food & Bioprocess Technology, 8, 1054–1062. Kamruzzaman, M., Nakauchi, S., & ElMasry, G. On-line screening of meat and poultry products using hyperspectral imaging. In High Throughput Screening for Food Safety Assessment—Biosensor Technologies, Hyperspectral Imaging and Practical Applications. Edited by A. K. Bhunia, M. S. Kim and C. R. Taitt. 2015d, 425–466. Woodhead Publishing Limited, Cambridge, UK. Kamruzzaman, M., & Sun, D.-W. Introduction to hyperspectral imaging. In Computer Vision Technology for Food Quality Evaluation. Edited by D.-W. Sun. Elsevier. 2016, 111–139. Academic Press, London. Kamruzzaman, M., Sun, D.-W., ElMasry, G., & Allen, P. (2013b). Fast detection and visualization of minced lamb meat adulteration using NIR hyperspectral imaging and multivariate image analysis. Talanta, 103, 130–136. Mancini, R. A., & Hunt, M. C. (2005). Current research in meat color. Meat Science, 71, 100–121. Naganathan, G. K., Grimes, l. M., Subbiah, J., Calkins, C. R., Samal, A., & Meyer, G. E. (2008a). Visible/near-infrared hyperspectral imaging for beef tenderness prediction. Computers & Electronics in Agriculture, 64, 225–233. Naganathan, G. K., Grimes, l. M., Subbiah, J., Calkins, C. R., Samal, A., & Meyer, G. E. (2008b). Partial least squares analysis of near-infrared hyperspectral images for beef tenderness prediction. Sensing & Instrumentation for Food Quality & Safety, 2, 178–188. Nakariyakul, S., & Casasent, D. (2008). Hyperspectral waveband selection for contaminant detection on poultry carcasses. Optical Engineering, 47, 087202–087209. Nakariyakul, S., & Casasent, D. (2009). Fast feature selection algorithm for poultry skin tumor detection in hyperspectral data. Journal of Food Engineering, 94, 358–365.

192

Mohammed Kamruzzaman

Park, B., Yoon, S.-C., Windham, W., Lawrence, K., Kim, M., & Chao, K. (2011). Linescan hyperspectral imaging for real-time in-line poultry fecal detection. Sensing & Instrumentation for Food Quality & Safety, 5, 25–32. Peng, Y., Zhang, J., Wang, W., Li, Y., Wu, J., Huang, H., Gao, X., & Jiang, W. (2011). Potential prediction of the microbial spoilage of beef using spatially resolved hyperspectral scattering profiles. Journal of Food Engineering, 102, 163–169. Pu, H., Xie, A., Sun, D. W., Kamruzzaman, M., & Ma, J. (2014a). Application of wavelet analysis to spectral data for categorization of lamb muscles. Food and Bioprocess Technology, 8, 1–16. Pu, H.-B., Kamruzzaman, M., Sun, D.-W. (2015). Selection of feature wavelengths for developing multispectral imaging systems for quality, safety and authenticity of muscle foods-a review. Trends in Food Science and Technology, 45, 86–104. Pu, H.-B., Sun, D.-W., Ma, J., Liu, D., Kamruzzaman, M. (2014b). Hierarchical variable selection for predicting chemical constituents in lamb meats using hyperspectral imaging. Journal of Food Engineering, 143, 44–52. Qiao, J., Wang, N., Ngadi, M. O., Gunenc, A., Monroy M., Gariepy, C., & Prasher, S. O. (2007). Prediction of drip-loss, pH and color for pork using a hyperspectral imaging technique. Meat Science, 76, 1–8. Sivertsen, A. H., Heia, K., Stormo, S. K., Elvevoll, E., & Nilsen, H. (2011). Automatic nematode detection in cod fillets (Gadus morhua) by transillumination hyperspectral imaging. Journal of food science, 76, S77–S83. Sone, I., Olsen, R. L., Sivertsen, A. H., Eilertsen, G. & Heia, K. (2012). Classification of fresh Atlantic salmon (Salmo salar L.) fillets stored under different atmospheres by hyperspectral imaging. Journal of Food Engineering, 109, 482–489. Sun, D.-W. Hyperspectral Imaging for Food Quality Analysis and Control. Academic Press/Elsevier, San Diego, CA, USA. 2010. Tao, F., Peng, Y., Li, Y., Wang, W., Chao, K. & Dhakal, S. (2012). Simultaneous determination of tenderness and Escherichia coli contamination of pork using hyperspectral scattering technique. Meat Science, 90, 851–857. Verbeke, W., Pérez-Cueto, F. J. A., & Grunert, K. G. (2011). To eat or not to eat pork, how frequently and how varied? Insights from the quantitative Q-Pork Chains consumer survey in four European countries. Meat Science, 88, 619–626. Wang, W., Peng, Y., & Zhang, X. (2010). Study on modeling method of total viable count of fresh pork meat based on hyperspectral imaging system. Spectroscopy and Spectral Analysis, 30, 411–415. Warriss, P. D., Brown, S. N., & Paściak, P. (2006). The colour of the adductor as a predictor of pork quality in the loin. Meat Science, 73, 565–569. Weeranantanaphan, J., Downey, G., Allen, P., & Sun, D. W. (2011). A review of near infrared spectroscopy in muscle food analysis: 2005–2010. Journal of Near Infrared Spectroscopy, 19, 61–104. Wu, D., Shi, H., Wang, S., He, Y., Bao, Y., & Liu, K. (2012a). Rapid prediction of moisture content of dehydrated prawns using online hyperspectral imaging system. Analytica Chimica Acta, 726, 57–66. Wu, D., & Sun, D.-W. (2013a). Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review—Part I: Fundamentals. Innovative Food Science and Emerging Technologies, 19, 1–14. Wu, D., & Sun, D.-W. (2013b). Colour measurements by computer vision for food quality control: A review. Trends in Food Science & Technology, 29, 5–20.

HSI for Meat Quality and Authenticity

193

Wu, D., Sun, D.-W., & He, Y. (2014). Novel non-invasive distribution measurement of texture profile analysis (TPA) in salmon fillet by using visible and near infrared hyperspectral imaging. Food Chemistry, 145, 417–426. Wu, J., Peng, Y., Li, Y., Wang, W., Chen, J., & Dhakal, S. (2012b). Prediction of beef quality attributes using VIS/NIR hyperspectral scattering imaging technique. Journal of Food Engineering, 109, 267–273. Xiong, A., Sun, D.-W., Zeng, X.-A., & Xie, A. (2014). Recent developments of hyperspectral imaging systems and their applications in detecting quality attributes of red meats: A review. Journal of Food Engineering, 132, 1–13. Xiong, A., Xie, A., Sun, D.-W., Zeng, X.-A., & Liu, D. (2015). Applications of hyperspectral imaging in chicken meat safety and quality detection and evaluation: A review. Critical Reviews in Food Science and Nutrition, 55, 1278–1291. Zhao, M., Downey, G., & O’Donnell, C. (2014). Detection of adulteration in fresh and frozen beef burger products by beef offal using mid-infrared ATR spectroscopy and multivariate data analysis. Meat Science, 96, 1003–1011.

Chapter

14

Hyperspectral Imaging Applications in Analysis of Fruits for Quality and Safety Anoop A. Krishnan Export Inspection Agency—Kochi (Laboratory)

S.K. Saxena Export Inspection Council

CONTENTS 14.1 Introduction 195 14.2  Application of HSI in Fruit Quality 197 14.2.1 HSI System 197 14.2.2 Application in Analysis of Quality of Fruits 197 14.3 Conclusion 199 References 202

14.1 INTRODUCTION Over the years, optical sensing technologies have been investigated as potential tools for nondestructive evaluation and inspection for food quality and safety. Nondestructive evaluation of the internal quality of fruits is an important field of research to improve their export potential. With the current need for reduced cost of production, the food industry is facing a number of challenges, including maintenance of high-quality ­standards and assurance of food safety while avoiding liability issues. Meeting these challenges has become crucial in regards to grading food products for different markets. Fruit quality is defined as a measure of characters or attributes that determine the suitability of the fruit to be eaten as fresh or stored for reasonable period without deterioration. Fruit quality could be considered as a multiple concept encompassing the physical, physiological, nutritional, and pathological attributes that affect fruit shelf life. Quality of a fresh produce includes appearance (size, shape, color, gloss, and freedom from defects and decay), texture (firmness, crispness, and toughness), flavor (sweetness, sourness, aroma, and off-flavors), and nutritive value (vitamins, minerals, nutrients, and carbohydrates).

195

196

Anoop A. Krishnan and S.K. Saxena

Currently, fruits are sorted manually or automatically on the basis of their external quality features. In manual sorting processes, there is a relatively high risk of human error, and decisions made by workers can be affected by psychological factors such as fatigue or acquired habits. However, internal quality attributes such as dry matter content, total soluble solid content (SSC), sugar content, and juice acidity are very important in the modern quality evaluation industries. Most instrumental techniques to measure these properties are destructive and involve a considerable amount of manual work, time  consuming, and nonobjective. Therefore, applying a rapid, accurate, and noninvasive technique in food analytical procedure for quality evaluation and safety inspection has become an urgent need. To speed up grading and classifying efficiency and to reduce artificial error and lower analysis cost, extensive efforts have been made recently to find solutions for rapid, accurate, and nondestructive determination of food products. Various researchers have reported on the development of different nondestructive sensing techniques for assessing the postharvest quality of horticultural crops; they include mechanical force/deformation, sonic, impact, optical, and electrical techniques (Abbott et al., 1997). Technologies that can sort fruit for appearance, texture, taste, flavor and/or nutritive value would assure fruit quality and consistency, increase consumer confidence and satisfaction, and enhance the competitiveness and profitability of the fruit industry (Lu and Ariana, 2002). Machine vision and NIR spectroscopy are two of the more extensively applied methods for food quality and safety assessment (Huang et al., 2014). Machine vision techniques has been used and applied to evaluate the external characteristics of food while spectroscopy was popularly used for quantification of the chemical components in food. The requirement or the weakness of the above techniques for the application in nonhomogenous samples introduced the hyperspectral imaging (HSI). Spectroscopic and HSI systems have many advantages when compared with classical chemical and physical analytical methods. It has a short measuring time with limited sample preparation, is chemical-free, and can be applied to estimate more than one attribute at the same time (Lammertyn et al., 1998). Near-infrared spectroscopy (NIRS) has become a useful technique for measuring fruit internal quality, especially SSC. Applications of NIRS have already been found for quality evaluation of food products, fruits, and vegetables. Quality evaluation with the visible/near-infrared (VIS/NIR) region of spectra is being used to extract information from a small area or point from the food object. Considerable researches have reported on using NIRS to measure fruit internal quality such as sugar content or SSC (Choi et al., 1997; Dull et al., 1989; Kawano et al., 1992; Lu et al., 2000; Lammertyn et al., 1998; Moons et al., 1997; Slaughter, 1995). NIRS has also been demonstrated to have the potential for measuring other related flavor attributes for apple and other fruits (Lovett et al., 2005; Kang et al., 2004; Saranwong et al., 2004). Commercial NIRS systems are recently available for sorting and grading of fresh fruits. The spectroscopic method has a great drawback compared with the HSI because it acquires the spectral data from a single point or from a small portion of the tested fruit. The HSI, on the contrary, has advantages of receiving spatially distributed spectral responses at each pixel of a fruit image. The HSI technique has been implemented in several applications such as inspection of poultry carcasses (Chao et al., 2001; Park et al., 2004) and defect detection or quality determination of apples, eggplants, pears, cucumber, and tomatoes (Cheng et al., 2004; Kim et al., 2002; Liu et al., 2006; Li et al., 2002; Polder et al., 2002). The successful attempts to evaluate internal properties nondestructively were accomplished using spectral technology for prediction sugar content (Bellon et al., 1993), soluble

HSI in Analysis of Fruits for Quality and Safety

197

solids (Park et al., 2003; Peiris et al., 1999), firmness (Park et al., 2003; Peirs et al., 2002), moisture content (Katayama et al., 1996), acidity (Lammertyn et al., 1998; Peirs et al., 2002) and so many other applications.

14.2  APPLICATION OF HSI IN FRUIT QUALITY The quality of a particular fruit is defined by a set of characteristics, such as color, size, weight, texture, hardness, and sweetness, which make it more or less attractive to the consumer. HSI is a nondestructive, rapid, and chemical-free method, and is now an emerging analytical tool which simultaneously offering spatial information and spectral signals from one object (Pu et al., 2015). The main goal of an HSI system in food industry is to provide a far more extensive source of information in order to maximize the quality of each product. Statistical learning approaches, such as principal component analysis (PCA), artificial neural networks (ANN), partial least square (PLS), and linear discriminant analysis (LDA), are employed for the detection and the classification of defects in fruits. 14.2.1 HSI System The basic principle of HSI is based on the fact that all materials, due to the difference of their chemical composition and inherent physical structure, reflect, scatter, absorb, and emit electromagnetic energy in distinctive patterns at specific wavelengths. This characteristic is called spectral signature or spectral fingerprint or simply the spectrum. A spectral signature is a unique characteristic of an object. For a given material, if the percentage of reflectance (also absorbance or transmittance) is plotted against wavelengths, the resulting curve is referred to as the spectral signature for that material. Each material has a distinctive spectrum that tells about its chemical composition. In essence, the spectral signature can be used to uniquely characterize, identify, and discriminate between classes/types of any given material(s) in each pixel of the image (Shaw and Manolakis, 2002). When the hyperspectral data are appropriately processed, it is possible to automatically identify the location of features that display specific spectral signatures and to map the gradient and spatial distribution of specific attributes. Typical HSI system comprises hardware and software. The specific configuration may vary depending on the object to be assessed and the technique of image acquisition. Most hardware platform of HSI systems share common basic components as shown in Figure 14.1: an illumination to provide light source (usually produced by halogen lamps); light irradiation either directly or delivered by optical fiber; detector (e.g., CCD or CMOS camera, InGaAs-based array detector) which obtains both spectral and spatial resolution simultaneously; spectrograph to disperse the wavelengths of the light and deliver signals to the photosensitive surface of the detector; an objective lens to adjust the range of light acquisition; an objective table fixed to a conveyer belt to hold and transport the sample; and a computer to compose and store the three-dimensional hypercube. 14.2.2 Application in Analysis of Quality of Fruits Exploring the possibility of using the optical property of fruits and vegetables for food product quality evaluation is gaining momentum. One example is NIRS, which

198

Anoop A. Krishnan and S.K. Saxena

FIGURE 14.1  Configuration of a hyperspectral imaging system.

has become a useful technique for measuring fruit internal quality, especially SSC. Applications of NIRS have already been found for quality evaluation of food products, fruits, and vegetables. Quality evaluation with the VIS/NIR region of spectra is being used to extract information from a small area or point from the food object. NIRS was also used for measuring fruit firmness (Lu and Peng, 2004; Lammertyn et al., 1998; McGlone et al., 2002) and other properties such as acidity (Downey and Kelly, 2004; Schmilovitch et al., 2000); however, the results are much less satisfactory. Also, NIRS is still unable to provide consistent and accurate measurement of other quality attributes such as fruit firmness (McGlone and Kawano, 1998; Lu et al., 2000). Recently, hyperspectral or multispectral imaging technique was investigated for measuring fruit firmness and sugar content (Polder et al., 2002). This new approach resulted in better prediction of fruit qualities than that by NIRS. Pu et al. (2015) gave a comprehensive review on the recent applications on textural characterization, biochemical component detection, and safety feature assessment using HSI on fruits. In the literature reviewed, the researchers worldwide have been effectively working in a variety of texture analysis algorithms for different applications such as detection, recognition, classification, segmentation, and clustering of different chemicals present in fruits. A number of products studied with HSI are fruits, e.g., apples, citruses, pears, and peaches. The majority of these studies were conducted in reflectance mode and in the VIS-NIR range (400–1,100 nm), while some recent research has been carried out in the NIR range (900–1,700 nm) (Wang et al., 2009; Sugiyama et al., 2010). For apple, HSI was used to measure quality attributes such as firmness, SSC, and mealiness (Peng and Lu, 2008; Huang and Lu, 2010). These studies demonstrated that hyperspectral scattering technique was potentially useful for the nondestructive detection of apple quality

HSI in Analysis of Fruits for Quality and Safety

199

attributes. The same spatially resolved diffuse reflectance HSI system was also used to study optical properties of fruits and vegetables including apple, pear, cucumber, and tomato (Qin and Lu, 2008). This research reinforced the potential of HSI technique as a convenient attribute classification tool for fruits as well as vegetables. HSI has been an emerging analytical tool for ensuring food safety and its quality. Table 14.1 presents a summary of typical papers published in hyperspectral detection in fruits since 2008. Majority of work has been done on various types of fruits as shown in Table 14.1 in reflectance mode, VIS/NIR (400–1,100 nm) region. Most of the fruit that was studied over the period are apple, citrus, pear, peach, orange, blueberry, citrus, grape seed, and strawberry. Peng and Lu (2008) designed a reflectance system to detect apple firmness and SSC using stable object stage. Ten MLD (modified Lorentzian function) functions were proposed to fit the spectral scattering profiles and the best one was chosen as the ideal method for predicting fruit firmness and SSC using multiple linear regression (MLR). Mendoza et al. (2011) employed integrated spectral scattering and image characteristics to predict the firmness and SSC in apples. The results indicated the need for a more robust prediction model for firmness and SSC of apples. Leiva-Valenzuela et al. (2013) used VIS/NIR HSI (500–1,000 nm) to determine the SSC in blueberries, reaching prediction accuracies of 0.87 and 0.79 for firmness and SSC, respectively. Another quality attribute of fruit evaluated using HSI is mealiness. Huang et al. (2012) examined the relationship between reflectance hyperspectral line images and apple mealiness. The spectral scattering profiles at individual wavelengths of apples undergoing different time, images were obtained and correlated to different mealiness levels. The mealiness of the apple was determined by the hardness and juiciness. Its correlation with hyperspectral scattering profiles was predicted using PLS. Classification models with two-class or more class was built using partial least squares discriminant analysis (PLSDA). The best classification accuracy was obtained in the classification of “nonmealy” and “mealy” apples, with an accuracy of 75%. This study demonstrated that hyperspectral scattering technique was potentially useful for nondestructive detection of apple mealiness and suggested that further research should focus on improving the classification accuracy especially for the discrimination of less severe mealy apples.

14.3 CONCLUSION HSI is an emerged technique invented to integrate both optical spectroscopy and traditional imaging. With the essential advantage for providing both spatial and spectral details, HSI systems introduce new inspection facilities that enable better evaluation of different agro-food products. Though HSI in association with powerful chemometrics releases us from the laborious measurement and burdensome computation during food quality assessment, there are still some obstacles to be surmounted in applying HSI into real-time applications. First of all, the equipment is still too expensive, especially for those with high spatial and spectral resolutions. Second, when the prebuilt model is applied to another HSI system, model transformation should be carefully conducted. Third, the characteristics of samples may also pose some problems for quality classification and prediction. On the one hand, spectral variations due to morphological changes of most fruits and vegetables (for example, round or cylinder-shaped objects) diminish the power of models; on the other hand, the interferences that samples possess (for example, the stem and calyx to bruise detection, the fluorescence compound

Product

Spectral Coverage

Apple

600–1,000

Quantitative Thresholding (TH)

CCD

Apple

450–1,000

Quantitative Not mentioned

CCD

Apple

400–1,000

Qualitative

Not mentioned

Apple

600–1,000

Qualitative

EMCCD (electronmultiplying charge-coupled device) EMCCD

Apple

400–1,000

Apple

Not mentioned

Apple

PLSR Mendoza Quantitative Run length matrix et al. (2011) (RLM), directional fractal dimension analysis, and multi-resolution wavelet transform Soft independent modeling Baranowski 400–1,000, Qualitative PCA (principal et al. (2012) 1,000–2,500 component analysis), class analogy (SIMCA), linear discriminant minimum noise analysis (LDA), SVM fraction (MNF)

Mode

Camera

Reflectance CCD (chargecoupled device)

Analysis Type

Image Processing

Locally linear embedding (LLE) Quantitative First derivative, and multiresolution wavelet transform

Partial least squares regression (PLSR), partial least squares discriminant analysis (PLSDA) Stepwise multi linear regression (SMLR) Artificial neural networks (ANN) Support Vector Machines (SVM), PLSDA Partial least square regression (PLSR)

Reference Huang and Lu (2010)

Peng and Lu (2008) Elmasry et al. (2009) Huang et al. (2012) Mendoza et al. (2012)

500–1,000

(Continued )

Anoop A. Krishnan and S.K. Saxena

TH

Modeling

200

TABLE 14.1  Summary of Measurement Mode, Product Type, Analysis Type, Wavelength and Modeling Algorithm in Representative Papers Published on Hyperspectral Imaging of Fruits Since 2008

TABLE 14.1 (Continued )  Summary of Measurement Mode, Product Type, Analysis Type, Wavelength and Modeling Algorithm in Representative Papers Published on Hyperspectral Imaging of Fruits Since 2008 Camera CCD

Product

Spectral Coverage

Analysis Type

Image Processing

Modeling

Quantitative TH

Manual analysis

CCD

Apple, peach, 500–1,000 kiwifruit, plum Citrus 400–1,000

Qualitative

EMCCD

Citrus

450–930

Qualitative

Genomic factor correction (GFC) TH

CCD

Mandarin

320–1,100

Qualitative

GFC

InGaAs (indium gallium arsenide) InGaAs

Strawberry

1,000–1,600

Qualitative

TH

Digital elevation model (DEM) Spectral information divergence (SID) imagining LDA, classification and regression trees (CART) LDA

Strawberry

1,000–1,600

Qualitative

Multiband image segmentation

Multiband multivariate classifiers, uniband univariate classifiers, multiband decision fusion classification

Reference Qin and Lu (2008) Gomez et al. (2008b) Qin et al. (2009) Gomez et al. (2008a) Sugiyama et al. (2010) Nanyam et al. (2012)

HSI in Analysis of Fruits for Quality and Safety

Mode

201

202

Anoop A. Krishnan and S.K. Saxena

to feces identification) might deteriorate the classification accuracy. In future research, spectral and image processing techniques in dealing with morphological influence are required. From the aspect of data analysis, extraction of ROIs (regions of interest) (feature wavebands or areas) for modeling is generally conducted by manual observation and operation, which accounts for the majority of analyzing time. Automatic recognition of representative ROIs based on computer software to improve analytical efficiency is another challenge. With technological advances in both hardware and software, HSI systems with low-cost and fast-detecting properties are anticipated. Currently, multispectral imaging based on critical wavebands derived from HSI receives great attention. It is expected that multispectral imaging system would be of broad use in widespread industrial applications due to its relative low instrument cost and high analytical speed. Additionally, HSI can be performed in different modes, and each mode has its own superiority on providing specific sample information. Thus, future HSI systems performing in a multimode manner would have a good potential by offering more comprehensive information on the same object. This review provides the evidence that the HSI system has the potential to fill the gap between spectroscopy and imaging techniques by recording full spectrum to each individual pixel in the image. The simultaneous exploration of both spectral and spatial information concomitant with appropriate multivariate analyses enables displaying chemical composition and ingredients of food samples in forms of concentration profiles. It is envisaged that this technology will undoubtedly play indispensable roles in research and industry for food quality evaluation and safety providing that more improvements in terms of expeditiousness, sensitivity, scanning rate, and portability that occurs to meet the requirements of the existing process control and sorting systems. The striking advantage of this technique is its chemical-free and nondestructive detecting nature together with the abundant information provided, which enables multiple quality features to be examined. Further research should concentrate on improving the processing speed of the large amount of hyperspectral information. With continued technical innovations in manufacturing and computing, HSI performing in a low-cost and high-speed way for online and real-time detection of various products is foreseen.

REFERENCES Abbott, J. A., Lu, R. F., Upchurch, B. L., Stroshine, R. L. (1997). “Technologies for nondestructive quality evaluation of fruits and vegetables”, in Horticultural Reviews, Volume 20 (Ed. J. Janick), John Wiley and Sons, New York, pp. 1–120. Baranowski, P., Mazurek, W., Wozniak, J., Majewska, U. (2012). Detection of early bruises in apples using hyperspectral data and thermal imaging. Journal of Food Engineering 110: 345–355. Bellon, V., Vigneau, J. L., Leclercq, M. (1993). Feasibility and performance of a new, multiplexed, fast and low-cost fiber-optic NIR spectrometer for the on-line measurement of sugars in fruits. Applied Spectroscopy 47(7): 1079–1083. Chao, K., Chen, Y. R., Hruschka, W. R., Park, B. (2001). Chicken heart disease characterization by multi-spectral imaging. Transactions of the ASAE 17(1): 99–106. Cheng, X., Chen, Y. R., Tao, Y., Wang, C. Y., Kim, M. S., Lefcourt, A. M. (2004). A novel integrated PCA and FLD method on hyperspectral image feature extraction for cucumber chilling damage inspection. Transactions of the ASAE 47(4): 1313–1320.

HSI in Analysis of Fruits for Quality and Safety

203

Choi, C. H., Abbott, J. A., Park, B., Chen, Y. R. (1997). Prediction of soluble solids and firmness in apples by visible/near infrared spectroscopy. In: Proceeding of the Fifth International Symposium of Fruit, Nut and Vegetable Production Engineering, Davis, CA, 3–10. Downey, G., Kelly, J. D. (2004). Detection and quantification of apple adulteration in diluted and sulfited strawberry and raspberry purees using visible and near-infrared spectroscopy. Journal of Agricultural and Food Chemistry 52: 204–209. Dull, G. G., Birth, G. S., Smittle, D. A., Leffler, R. G. (1989). Near infrared analysis of soluble solids in intact cantaloupe. Journal of Food Science 54(2): 393–395. Elmasry, G., Wang, N., Vigneault, C. (2009). Detecting chilling injury in red delicious apple using hyperspectral imaging and neural networks. Postharvest Biology and Technology 52: 1–8. Gómez-Sanchis, J., Gómez-Chova, L., Aleixos, N., Camps-Valls, G., Montesinos-Herrero, C., Moltó, E., Blasco, J. (2008a). Hyperspectral system for early detection of rottenness caused by Penicillium digitatumin mandarins. Journal of Food Engineering 89: 80–86. Gómez-Sanchis, J., Moltó, E., Camps-Valls, G., Gómez-Chova, L., Aleixos, N., Blasco, J. (2008b). Automatic correction of the effects of the light source on spherical objects. An application to the analysis of hyperspectral images of citrus fruits. Journal of Food Engineering 85: 191–200. Huang, H., Liu, L., Ngadi, M. O. (2014). Recent developments in hyper spectral imaging for assessment of food quality and safety. Sensors 14: 7248–7276. Huang, M., Lu, R. F. (2010). Apple mealiness detection using hyperspectral scattering technique. Postharvest Biology and Technology 58(3): 168–175. Huang, M., Zhu, Q., Wang, B., Lu, R. (2012). Analysis of hyperspectral scattering images using locally linear embedding algorithm for apple mealiness classification. Computers and Electronics in Agriculture 89: 175–181. Kang, S., Lee, K. J., Choi, W., Son, L. R., Choi, D. S., Kim, G. (2004). A near infrared sensing technique for measuring the quality of potatoes, Mich: ASAE, St. Joseph, ASAE Paper No: 033137. Katayama, K., Komaki, K., Tamiya, S. (1996). Prediction of starch, moisture, and sugar in sweet potato by near infrared transmittance. Hort Science 31(6): 1003–1006. Kawano, S., Watanabe, H., Iwamoto, M. (1992). Determination of sugar content in intact peaches by near infrared spectroscopy with fiber optics in interactance mode. Journal of the Japanese Society for Horticultural Science 61(2): 445–451. Kim, M. S., Lefcourt, A. M., Chao, K., Chen, Y. R., Kim, I., Chan, D. E. (2002). Multispectral detection of fecal contamination on apples based on hyperspectral imagery: Part I. Application of visible and near-infrared reflectance imaging. Transactions of the ASAE 45(6): 2027–2037. Lammertyn, J., Nicolai, B., Ooms, K., De Smedt, V., De Baerdemaeker, J. (1998). Nondestructive measurement of acidity, soluble solids, and firmness of Jonagold apples using NIR-spectroscopy. Transactions of the ASAE 41(4): 1089–1094. Leiva-Valenzuela, G., Lu, R., Aguilera, J. M. (2013). Prediction of firmness and soluble solids content of blueberries using hyperspectral reflectance imaging. Journal of Food Engineering 115(1): 91–98. Li, Q., Wang, M., Gu, W. (2002). Computer vision based system for apple surface defect detection. Computers and Electronics in Agriculture 36(2): 215–223. Liu, Y., Chen, Y. R., Wang, C. Y., Chan, D. E., Kim, M. S. (2006). Development of hyperspectral imaging technique for the detection of chilling injury in cucumbers; spectral and image analysis. Applied Engineering in Agriculture 22(1): 101–111.

204

Anoop A. Krishnan and S.K. Saxena

Lovett, D. K., Deaville, E. R., Givens, D. I., Finlay, M., Owen, E. (2005). Near infrared reflectance spectroscopy (NIRS) to predict biological parameters of maize silage: effects of particle comminution, oven drying temperature and the presence of residual moisture. Animal Feed Science and Technology 120: 323–332. Lu, R. F., Ariana, D. (2002). A near-infrared sensing technique for measuring internal quality of apple fruit. Applied Engineering in Agriculture 18: 585–590. Lu, R. F., Guyer, D. E., Beaudry, R. M. (2000). Determination of firmness and sugar content of apples using near-infrared diffuse reflectance. Journal of Texture Studies 31: 615–630. Lu, R. F., Peng, Y. K. (2004). “Hyperspectral scattering for assessing peach fruit firmness”, Mich: ASAE, St. Joseph, ASAE Paper No. 043007. McGlone, V. A., Jordan, R. B., Martinsen, P. J. (2002). VIS/NIR estimation at harvest of pre- and post-storage quality indices for ‘Royal Gala’ apple. Postharvest Biology and Technology 25: 135–144. McGlone, V. A., Kawano, S. (1998). Firmness, dry-matter and soluble-solids assessment of postharvest kiwifruit by NIR spectroscopy. Postharvest Biology and Technology 13: 131–141. Mendoza, F., Lu, R., Ariana, D., Cen, H., Bailey, B. (2011). Integrated spectral and image analysis of hyperspectral scattering data for prediction of apple fruit firmness and soluble solids content. Postharvest Biology and Technology 62: 149–160. Mendoza, F., Lu, R. F., Cen, H. Y. (2012). Comparison and fusion of four nondestructive sensors for predicting apple fruit firmness and soluble solids content. Postharvest Biology and Technology 73: 89–98. Moons, E., Dardenne, P., Dubois, A., Sindic, M. (1997). Nondestructive visible and NIR spectroscopy measurement for the determination of apple internal quality. Acta Horticulturae 517: 441–448. Nanyam, Y., Choudhary, R., Gupta, L., Paliwal, J. (2012). A decision-fusion strategy for fruit quality inspection using hyperspectral imaging. Biosystems Engineering 111: 118–125. Park, B., Abbott, J. A., Lee, K. J., Choi, C. H., Choi, K. H. (2003). Near-infrared diffuse reflectance for quantitative and qualitative measurement of soluble solids and firmness of Delicious and Gala apples. Transactions of the ASAE 46(6): 1721–1731. Park, B., Windham, W. R., Lawrence, K. C., Smith, D. P. (2004). Hyperspectral image classification for fecal and ingest identification by spectral angle mapper. ASAE/ CSAE meeting, Ottawa, Ontario, Canada, ASAE Paper No. 043032. Peiris, K. H. S., Dull, G. G., Leffler, R. G., Kays, S. J. (1999). Spatial variability of soluble solids or dry-matter content with in individual fruits, bulbs, or tubers: Implications for the development and use of NIR spectrometric technique. Hort Science 34(1): 114–118. Peirs, A., Scheerlinck, N., Touchant, K., Nicolaı ,̈ B. M. (2002). Comparison of Fourier transform and dispersive near-infrared reflectance spectroscopy for apple quality measurements. Biosystems Engineering 81(3): 305–311. Peng, Y., Lu, R. (2008) Analysis of spatially resolved hyperspectral scattering images for assessing apple fruit firmness and soluble solids content. Postharvest Biology and Technology 48: 52–62. Polder, G., Van Der Heijden, W. A. M., Young, I. T. (2002). Spectral image analysis for measuring ripeness of tomatoes. Transactions of the ASAE 2002(45): 1155–1161.

HSI in Analysis of Fruits for Quality and Safety

205

Pu, Y.-Y., Feng, Y.-Z., Sun, D.-W. (2015). Recent progress of hyperspectral imaging on quality and safety inspection of fruits and vegetables: A review. Comprehensive Review in Food Sciences and Food Safety 14(2): 176–188. Qin, J., Burks, T. F., Ritenour, M. A., Bonn, W. G. (2009). Detection of citrus canker using hyperspectral reflectance imaging with spectral information divergence. Journal of Food Engineering 93: 183–191. Qin, J. W., Lu, R. F. (2008). Measurement of the optical properties of fruits and vegetables using spatially resolved hyperspectral diffuse reflectance imaging technique. Postharvest Biology and Technology 49(3): 355–365. Saranwong, S., Sornsrivichai, J., Kawano, S. (2004). Prediction of ripe-stage eating quality of mango fruit from its harvest quality measured nondestructively by near infrared spectroscopy. Postharvest Biology and Technology 31: 137–145. Schmilovitch, Z., Mizrach, A., Hoffman, A., Egozi, H., Fuchs, Y. (2000). Determination of mango physiological indices by near-infrared spectrometry. Postharvest Biology and Technology 19: 245–252. Shaw, G., Manolakis, D. (2002). Signal processing for hyperspectral image exploitation. IEEE Signal Proc. Mag. pp. 12–16. Slaughter, D. C. (1995). Nondestructive determination of internal quality in peaches and nectarines. Transactions of the American Society of Agricultural Engineers 38(2): 617–623. Sugiyama, T., Sugiyama, J., Tsuta, M., Fujita, K., Shibata, M., Kokawa, M., Araki, T., Nabetani, H., Sagara, Y. (2010). NIR spectral imaging with discriminant analysis for detecting foreign materials among blueberries. Journal of Food Engineering 101(3): 244–252. Tan, J. L. (2004). Meat quality evaluation by computer vision. Journal of Food Engineering 61: 27–35. Wang, W. L., Li, C. Y., Gitaitis, R., Tollner, E. W., Yoon, S. C. (2009). Detection of Sour Skin Diseases in Vidalia Sweet Onions Using Near-infrared Hyperspectral Imaging. Reno, Nevada: The American Society of Agricultural and Biological Engineers. www.asabe.org. Paper number 096364, ASAE Annual Meeting.

Chapter

15

Applications in Vegetables Leo M.L. Nollet University College Ghent

Hong-Ju He and Hui Wang Henan Institute of Science and Technology (HIST)

CONTENTS 15.1  Soybean 207 15.2  Mushroom 209 15.3  Potato 213 15.4  Tomato 215 15.5  Cucumber 217 15.6  Lettuce 221 15.7  Spinach 223 15.8  Peppers 225 15.9  Onion 226 15.10 Broccoli 226 Abbreviations 227 References 227

15.1 SOYBEAN Soybean (Glycine max L.) is the most widely grown oilseed crop in the world. Applications of hyperspectral imaging (HSI) on soybeans are detection of insect damage, analysis of pigments, sweetness, color, and moisture, and checking for isoflavones and amino content. Hyperspectral spectrometry and a backpropagation neural network (BPNN) model were used to detect the cowpea weevils (Callosobruchus maculatus (F.)) in soybean [1]. Spectrum of each sample was measured using a ASD FieldSpec® 3 Spectroradiometer fitted with a high intensity contact probe. Spectra data were processed by analysis of variance (ANOVA) and BPNN using MATLAB®. After the optimum eigenvalues were determined based on the spectral curves, they are used as input vectors to create the BPNN model. Results showed that the sensitive bands 780–900 nm, 920–1,000 nm, and 1,205–1,560 nm have the potential to detect the infestation caused by cowpea weevils in soybeans. The eigenvalues, such as the crest or trough positions of the spectral curves, and the slope degree of the edges of the first derivative spectrum were found to be useful and optimum eigenvalues for differentiating the infested soybean samples caused by cowpea weevils from noninfested soybean samples. The correct classification of the obtained

207

208

Leo M.L. Nollet, et al.

BPNN model arrived 87.5% for the testing samples set and 93.5% for the total samples set model. Huang, Wan, Zhang, and Zhu [2] attempted to use HSI in wavelength of 400– 1,000 nm to detect insect-damaged vegetable soybeans. By extracting four statistical image features (minimum, maximum, mean, and standard deviation) and applying support ­vector data description (SVDD) classifier, 97.3% and 87.5% classification accuracy were achieved for the normal and insect-damaged samples, respectively, with an overall classification accuracy of 95.6%. Within the same wavelength range, Ma, Huang, Yang, and Zhu [3] proposed to select the region of interest through automatic threshold segmentation and optimal wavelength selection using the fuzzy-rough set model to discriminate the insect-damaged soybean, based on acquired hyperspectral images of 362 bean samples. As a result, 100.0% and 91.7% accuracy were achieved for the normal and insect-­damaged bean samples, respectively, with an overall classification accuracy of 98.8%. The accuracy improved 2.8% and 4.8% for the normal and insect-damaged bean samples, respectively, as well as 3.3% for overall accuracy, compared with the study of Huang, Wan, Zhang, and Zhu [2]. In another study conducted by Chelladurai, Karuppiah, Jayas, Fields, and White [4], a specific pest called cowpea weevil, which always infests soybeans and causes extensive storage losses, was detected by HSI technology combined with soft X-ray. The near-infrared (NIR) hyperspectral images (960–1,700 nm) of soybeans infested by three stages of C. maculatus (egg, larva, and pupa) were acquired for analysis. By establishing pair-wise linear discriminant analysis models based on the NIR hyperspectral data, 86% and 87% classification accuracy were obtained for uninfested and infested soybean samples, respectively. Three wavelengths (960, 1,030, and 1,440 nm) were identified to explain more than 99% of spectral information variability, after principal component analysis (PCA). It was concluded that the combination of NIR HSI and X-ray could be more effective and reliable for the classification of pestinfested soybeans. Soybeans are always dried to store and processed into snack food after harvest. Color and moisture are two important parameters in the quality evaluation of dried soybeans. As is well-known, color and moisture are usually measured by conventional methods that are time-consuming and labor-intensive. In addition to insect detection, Huang, Wang, Zhang, and Zhu [5] also used the HSI to assess the color and moisture content (MC) of soybeans simultaneously during drying process. Partial least squares regression was applied for 270 dried soybean samples based on mean reflectance spectra and image entropy parameters. Better prediction performance was found by using mean reflectance spectra alone, with correlation coefficients in prediction set (R P) of 0.86 and root mean square errors of prediction (RMSEP) of 1.04 for color as well as R P of 0.97 and RMSEP of 4.7% for moisture. Significant potential of using HSI has been demonstrated in the quality evaluation of soybeans, and much work is still needed to improve and ensure the higher accuracy, either internal damage or external quality. Accurate prediction of leaf pigments from spectral reflectance is important because it allows nondestructive, rapid assessment of crop-N status under field conditions [6]. Canopy reflectance and leaf pigments (chlorophyll and carotenoids concentrations) were measured on 385 field-grown soybean genotypes during flowering and seed development stages each in 2009 and 2010. Spectral features related to pigments were extracted based on several known spectral indices and using a number of analytical methods to develop prediction models incorporating reflectance data at single, two (simple-ratio), or more (multiple linear regression, MLR) wavebands. Among the tested methods, fitness and accuracy (measured as coefficient of determination, R 2; root mean square error, RMSE;

Applications in Vegetables

209

and relative error, % RE) of the prediction models developed using MLR was greatest. The accuracy of known indices such as the Maccioni-index and canopy chlorophyll content index showed potential for the estimation of pigment concentrations using soybean canopy reflectance data. However, models developed using transformed spectra outperformed the original reflectance spectra irrespective of the analytical method used. In general, the validation of the MLR models revealed limited accuracy across sampling dates and types of spectra used. Continuous wavelet transformed spectra using “Mexican hat” wavelet family (CWT-mexh) produced the best model with the highest accuracy. The selected wavebands in the models primarily consisted of the visible (400–750 nm) as compared to the NIR (750–1,350 nm) spectrum. A general-purpose MLR model using ­C WT-mexh spectra that was strongly related to pigment concentrations (R 2 = 0.86, RMSE = 2.12, and RE = 12.5%; chlorophyll and R 2 = 0.83, RMSE = 0.56, and RE = 12.7%; carotenoids) was developed. A hyperspectral image data-processing method was investigated to predict the sweetness and amino acid content of soybean crops [7]. Regression models based on artificial neural networks were developed in order to calculate the level of sucrose, glucose, fructose, and nitrogen concentrations, which can be related to the sweetness and amino acid content of vegetables. A performance analysis was conducted comparing regression models obtained using different preprocessing methods, namely, raw reflectance, second derivative, and principal components analysis. High-resolution hyperspectral data of wavelengths ranging from the visible to the NIR acquired from an experimental field of green vegetable soybeans were collected. The best predictions were achieved using a nonlinear regression model of the second derivative transformed dataset. Glucose could be predicted with greater accuracy, followed by sucrose, fructose, and nitrogen. Instead of traditional method relying on many reagents, HSI shows its nondestructive advantage and was proposed to detect the isoflavone content in soybean [8]. Five wavelengths such as 1,516, 1,572, 1,691, 1,716 and 1,760 nm within 1,000–2,500 nm range were selected as feature wavelengths to predict the isoflavone content of 40 varieties of soybeans using the artificial neural network (ANN) algorithm, leading to the coefficient of determination (R 2) of 0.97 and mean square error (MSE) of 0.11.

15.2 MUSHROOM White button mushroom (Agaricus bisporus) is a medicinal edible and a main commercial variety of fungi for wide consumption by people around the world in form of raw or processed food. HSI is used to check damage, MC, and shelf life of mushrooms. Further HSI analyses may be quality and bacterial control. In the study of Gowen [9], the potential application of HSI for damage detection on the caps of white mushrooms (Agaricus bisporus) was investigated. Mushrooms were damaged by controlled vibration to simulate damage caused by transportation. Hyperspectral images were obtained using a pushbroom line-scanning HSI instrument, operating in the wavelength range of 400–1,000 nm with spectroscopic resolution of 5 nm. The effective resolution of the CCD detector was 580 × 580 pixels by 12 bits. Two data reduction methods were investigated: in the first, PCA was applied to the hypercube of each sample, and in the second, PC (PC 2) score image was used for the identification of bruise-damaged regions on the mushroom surface; in the second method, PCA was applied to a dataset comprising average spectra from regions normal and bruise-damaged

210

Leo M.L. Nollet, et al.

tissue. In this case, it was observed that normal and bruised tissues were separable along the resultant first principal component (PC 1) axis. Multiplying the PC 1 eigenvector by the hypercube data allowed reduction of the hypercube to a 2-D image, which showed maximal contrast between normal and bruise-damaged tissues. The second method performed better than the first when applied to a set of independent mushroom samples. Mushrooms have a thin and porous epidermal structure and are sensitive to handling and transportation practices [10]. Mechanical damage triggers a browning process within the tissue changing its metabolic state. Different levels of physical perturbation on the mushroom pilei, using NIR spectral images and machine learning approaches were quantified. An ANN classifier is implemented, whose input is a small set of vectors containing representative information, and output is the set of categorical labels that correspond to different levels of mechanical vibration. For obtaining a salient dataset for classifying the images, the Harris corner detection algorithm is employed. The advantage of using interest points is to replace an exhaustive search over the entire image space by a computation over a concise set of highly informative points. A frame-based classification approach is proposed and shown to produce an increase in the classification accuracy, since feature vectors regarded as single instances may not always carry sufficient discriminant information. Comparisons with statistical features computed from wavelet coefficients showed that interest points are more suitable in assessing mechanical perturbation. Comparisons on a classifier level with support vector machines showed that ANNs perform better for  the specific application, implying a connection between the classification method and the underlying learning problem. Overall, the frame-based classification scheme reduced the misclassification rate. This approach is suited for challenging c­ lassification problems where the degree of class separation is variable, i.e., assessment of mechanical damage in mushrooms. White mushrooms were subjected to mechanical injury by controlled shaking in a plastic box at 400 rpm for different times (0, 60, 120, 300 and 600 s) [11]. Immediately after shaking, hyperspectral images were obtained using two pushbroom line-scanning HSI instruments, one operating in the wavelength range of 400–1,000 nm with spectroscopic resolution of 5 nm, the other operating in the wavelength range of 950–1,700 nm with spectroscopic resolution of 7 nm. Different spectral and spatial pretreatments were investigated to reduce the effect of sample curvature on hyperspectral data. Algorithms based on chemometric techniques (principal component analysis and partial least squares discriminant analysis) and image processing methods (masking, thresholding, morphological operations) were developed for pixel classification in hyperspectral images. In addition, correlation analysis, spectral angle mapping, and scaled difference of sample spectra were investigated and compared with the chemometric approaches. The potential use of HSI for early detection of freeze damage in white button mushrooms was investigated [12]. Hyperspectral images of mushroom samples were obtained using a pushbroom line-scanning HSI instrument in the wavelength range of 400–1,000 nm. Reflectance spectra from each mushroom sample were obtained from various positions on the mushroom surface and pretreated using the standard normal variate (SNV) transformation. Weight loss (WL) and Hunter L-value of mushroom samples were also measured over the storage period. At early stages of thawing, frozen samples were similar in appearance, WL and L-value to undamaged mushrooms, while substantial changes in WL and L-value were evident in freeze-damaged mushrooms after 24-hour thawing. A procedure based on principal components analysis and linear discriminant analysis was developed for classification of both selected spectra and whole mushrooms into undamaged and freeze-damaged groups. Applying this procedure to an independent test set, 100% and 97.9% of whole undamaged mushrooms and freeze-damaged samples,

Applications in Vegetables

211

respectively, were correctly classified. Using this method, freeze-damaged mushrooms could be classified with high accuracy (>95% correct classification) after only 45 min thawing (at 23°C ± 2°C) at which time freeze–thaw damage was not visibly evident. Physical stress (i.e., bruising) during harvesting, handling, and transportation triggers enzymatic discoloration of mushrooms, a common and detrimental phenomenon largely mediated by polyphenol oxidase (PPO) enzymes [13]. The objective of this study was to assess the ability of HSI to predict the activity of PPO on mushroom caps. Hyperspectral images of mushrooms subjected to various damage treatments were taken, followed by enzyme extraction and PPO activity measurement. Principal component regression models (each with three PCs) built on raw reflectance and multiple scatter-corrected (MSC) reflectance data were found to be the best modeling approach. Prediction maps showed that the MSC model allowed for compensation of spectral differences due to sample ­curvature and surface irregularities. The potential application of HSI for quality prediction of white mushroom slices during storage at 4°C and 15°C was investigated [14]. Mushroom slice quality was measured in terms of MC, color (CIE Lightness, L* and yellowness, b*) and texture (hardness, H and chewiness, Ch). Hyperspectral images were obtained using a pushbroom line-­scanning HSI instrument, operating in the wavelength range of 400–1,000 nm with spectroscopic resolution of 5 nm. Multiple linear regression and principal component regression models were developed to investigate the relationship between reflectance and the various quality parameters measured. Twenty optimal wavelengths for quality prediction were selected after performing an exhaustive search for the best subsets of predictor variables on a calibration set of 84 samples. PCR applied to the set of optimal wavelengths gave the best performance as compared to MLR and PCR on the entire wavelength range. When applied to an independent validation set of samples, PCR models developed on the calibration set were capable of predicting MC with RMSEP of 0.74% w.b. and R 2 of 0.75, L* with RMSEP of 0.47 and R 2 of 0.95, b* with RMSEP of 0.66 and R 2 of 0.75, H with RMSEP of 0.49 N and R 2 of 0.77, and Ch with RMSEP of 0.27 N and R 2 of 0.72. Virtual images showing the distribution of MC on the mushroom surface were generated from the estimated PCR model. A method for mushroom quality grading based on hyperspectral image analysis in the wavelength range of 400–1,000 nm is presented by A. A. Gowen et al. [15]. Different spectral and spatial pretreatments were investigated to reduce the effect of sample curvature on hyperspectral data. Algorithms based on chemometric techniques (principal component analysis and partial least squares discriminant analysis) and image processing methods (masking, thresholding, morphological operations) were developed for pixel classification in hyperspectral images. A HSI system with a spectral range of 380–1,000 nm was used for the detection of four levels of skin browning on button mushroom [16]. Samples were stored at standard condition (3°C ± 1°C and 92% ± 2% RH) to generate different levels of browning development on cap surface. After acquisition of hypercubes, the extracted spectra were ­preprocessed by the Savitzky–Golay and mean normalization treats. Inter- and ­intravariations of different levels of browning were calculated using spectral similarity measure. The competitive adaptive reweighted sampling algorithm was applied to extract the browning-specific wavelengths, leading to the partial least square-discriminant analysis classification accuracy of 80.6% and 80.3% for calibration and testing stages, respectively. Classification maps were generated using browning-specific wavelengths and compared to the classification maps generated from the full spectral points, PCA of full spectral points, and conventional RGB (red, green, blue) imaging bands.

212

Leo M.L. Nollet, et al.

In the study of M. Taghizadeh et al. [17], the potential of HSI to predict white button mushroom MC was investigated. Mushrooms were subjected to dehydration at 45°C ± 1°C for different time periods (0, 30, 60, and 120 min) to obtain representative samples at different moisture levels (93.40% ± 0.62%, 82.76% ± 2.11%, 73.20% ± 2.60%, and 60.89% ± 4.32% wet basis [wb]). Hyperspectral images of the mushrooms were obtained using a pushbroom system operating in the wavelength range of 400–1,000 nm. Hunter L, a, and b color values of the mushrooms were also measured. The average reflectance spectra of samples at different MC levels were obtained and partial least square regression models were built to predict mushroom MC. To reduce the spectral variability caused by factors unrelated to MC such as scattering effects and differences in sample height, different spectral pretreatments were applied. The SNV transformation was found to be the best approach among the wavelength range studied, resulting in the greatest reduction in root mean square error of cross-validation (RMSECV) and root mean square error of prediction (RMSEP) for a four-component PLSR model. RMSECV of 5.50 (% wb) and RMSEP of 5.58 (% wb) were obtained for the calibration and test sets of data, respectively. Prediction maps were generated from hyperspectral data to show the predictive model performance at pixel level. The shelf life of mushrooms packaged using different polymer top-films (PVC, PET with different levels of perforations) was investigated using HSI [18]. Packaged mushrooms were stored at 4°C ± 0.2°C for 14 days, and WL, Hunter L, a, b values, maturity index, and in-pack gas composition (% CO2 and O2) were also measured. The results obtained showed that the PET film perforated with small holes (1 mm in diameter) was generally superior in terms of maintaining overall mushroom quality. Regression models were built to correlate HSI data with measured quality parameters. Prediction maps were generated from hyperspectral data to show the model performance at pixel level. Brown blotch, caused by pathogenic Pseudomonas tolaasii, is the most problematic bacterial disease in Agaricus bisporus mushrooms [19]. Although it does not cause any health problems, it reduces the consumer appeal of mushrooms in the market place, generating important economic losses worldwide. The objective of this study was to investigate the use of HSI for brown blotch identification and discrimination from mechanical damage on mushrooms. Hyperspectral images of mushrooms subjected to (1) no treatment, (2) mechanical damage, or (3) microbiological spoilage were taken during storage and spectra representing each of the classes were selected. Partial least squares-discriminant analysis (PLS-DA) was carried out in two steps: (1) discrimination between undamaged and damaged mushrooms and (2) discrimination between damage sources (i.e., mechanical or microbiological). The models were applied at a pixel level and a decision tree was used to classify mushrooms into one of the aforementioned classes. A correct classification of >95% was achieved. Mycophilic fungi of anamorphic genus Sepedonium infect and parasitize sporomata of boletes [20]. The obligated hosts such as Boletus edulis and allied species (known as “porcini mushrooms”) are among the most valued and prized edible wild mushrooms in the world. Sepedonium infections have a great morphological variability: at the initial state, contaminated mushrooms present a white coating covering tubes and pores; at the final state, Sepedonium forms a deep and thick hyphal layer that eventually leads to the total necrosis of the host. Up to date, Sepedonium infections in porcini mushrooms have been evaluated only through macroscopic and microscopic visual analysis. In this study, in order to implement the infection evaluation as a routine methodology for industrial purposes, the potential application of HSI and PCA for detection of Sepedonium presence on sliced and dried B. edulis and allied species was investigated. Hyperspectral

Applications in Vegetables

213

images were obtained using a pushbroom line-scanning HSI instrument, operating in the wavelength range between 400 and 1,000 nm with 5 nm resolution. PCA was applied on normal and contaminated samples. To reduce the spectral variability caused by factors unrelated to Sepedonium infection, such as scattering effects and differences in sample height, different spectral pretreatments were applied. A supervised rule was then developed to assign spectra recorded on new test samples to each of the two classes, based on the PC scores. This allowed to visualize directly—within false-color images of test samples—which points of the samples were contaminated. The results achieved may lead to the development of a nondestructive monitoring system for a rapid on-line screening of contaminated mushrooms.

15.3 POTATO Different parameters are detected by HSI in potatoes. A. Dacal-Nieto et al. [21] present a new method to detect the presence of the hollow heart, an internal disorder of the potato tubers, using HSI technology in the infrared region. A set of 468 hyperspectral cubes of images has been acquired from Agria variety potatoes, which have been cut later to check the presence of a hollow heart. The authors developed several experiments to recognize hollow heart potatoes using different Artificial Intelligence and Image Processing techniques. The results showed that Support Vector Machines (SVM) achieved an accuracy of 89.1% of correct classification. Cooking of potatoes causes changes in the microstructure and composition of starch [22]. These changes affect the interaction of light with the starch granules at different regions inside the potato. The potential of HSI in the wavelength range of 400–1,000 nm in combination with chemometric tools and image processing for contactless detection of the cooking front in potatoes was investigated. Partial least squares discriminant analysis (PLSDA) was employed to discriminate between the pixel spectra for the cooked regions and those for the remaining raw regions. In a next step, image processing techniques were applied to detect the cooking front in the images obtained by the PLSDA pixel classification. From each of the resulting images with detected cooking fronts, the ratio of the remaining raw part area over the total potato area was then calculated. Finally, the effect of the cooking time on this ratio was modeled to be able to predict the optimal cooking time. Detection of starch content in potato is studied applying HSI technique by W. Jiang et al. [23]. The original and preprocessing spectra were processed with partial least square (PLS) method to build prediction model of starch content. The original spectra between 400 and 1,000 nm were preprocessed with smoothing, second derivation, and multiplicative scatter correction (MSC). Prediction model was built with preprocessing spectra by applying PCA. Known from the result, the model based on the preprocessing spectra preprocessed with smoothing and PCA is the best of all prediction models built in research. The determination coefficient (R 2) of calibration set and prediction set was 0.8234 and 0.9031 respectively. The root mean square error of calibration set (RMSEC) and root mean square error of validation set (RMSEV) were 0.5633 and 0.502, respectively. It indicated that this method could be applied in detection of starch content in potato. The study could offer theoretical and practical reference for further study in the future. Visible/NIR (VNIR) hyperspectral reflectance imaging was evaluated as a technique toward rapid prediction of the glucose and sucrose percentages in two common fresh use and chipping potato cultivars [24, 25]. Tubers were sampled and held in multiple storage temperatures in an attempt to develop uniform and broad constituent distributions.

214

Leo M.L. Nollet, et al.

Each tested sample was a 12.7-mm-thick slice cut uniformly from all tubers. Multiple features were extracted from samples including mean reflectance spectra and curve feature parameters yielded from an exponential model. Both glucose and sucrose ratios were measured using the Megazyme sucrose/d-glucose assay procedure as reference values. Partial least squares regression (PLSR), feed forward neural networks (FFNN), radial basis functions neural networks (RBFNN), and exact design radial basis functions (RBFNNE) neural networks were used for building calibration and prediction models. PLSR results demonstrated strongly correlated models built using mean reflectance spectra for glucose. Sucrose models showed less correlation performance. Wavelength selection/prediction using interval partial least squares (IPLS) and genetic algorithm (GA) was conducted on the data, and PLSR and NN results were close to the full-wavelength models for the glucose and sucrose of both cultivars with a preference given to IPLS as it yields less selected variables than GA. Applying K-nearest neighbor (Knn) and partial least squares discriminant analysis (PLSDA) on mean reflectance spectra resulted in glucose misclassification. However, classification errors were higher for sucrose indicating lower accuracy for this sugar. Blackspot is a subsurface potato damage resulting from impacts during harvesting. This type of bruising represents substantial economic losses every year. As the tubers do not show external symptoms, bruise detection in potatoes is not straightforward. Therefore, a nondestructive and accurate method capable of identifying bruised tubers is needed. HSI has been shown to be able to detect other subsurface defects such as bruises in apples. This method is nondestructive, fast, and fully automated. Therefore, its potential for nondestructive detection of blackspot in potatoes has been investigated in this study [26]. Two HSI setups were used, one ranging from 400 to 1,000 nm, named visibleNear Infrared (Vis-NIR), and another covering the 1,000–2,500 nm range, called Short Wave Infrared (SWIR). A total of 188 samples belonging to three different varieties were divided in two groups. Bruises were manually induced and samples were analyzed 1, 5, 9, and 24 h after bruising. PCA, SIMCA and PLS-DA were used to build classifiers. The PLS-DA model performed better than SIMCA, achieving an overall correct classification rate above 94% for both hyperspectral setups. Furthermore, more accurate results were obtained with the SWIR setup at the tuber level (98.56% vs. 95.46% CC), allowing the identification of early bruises within 5 h after bruising. Moreover, the pixel based PLS-DA model achieved better results in the SWIR setup in terms of correctly classified samples (93.71% vs. 90.82% CC) suggesting that it is possible to detect blackspot areas in each potato tuber with high accuracy. The aim of the study of Y. Sun et al. was to investigate the feasibility of HSI in measuring MC and freezable water content during drying process [27]. Hyperspectral images were acquired for purple-fleshed sweet potato slices during contact ultrasound assisted hot drying (CUHAD) process, and the corresponding mean reflectance spectra from regions of interest in visible and NIR (371–1,023 nm) regions were extracted. Moving average, Savitzky–Golay smoothing filter (S–G filter), and multiplicative scatter correction (MSC) were investigated to preprocess the raw spectra and partial least square regression (PLSR) calibration model was established to analyze the relationship between the extracted spectral data and measured quality attributes. Comparing the performance of model based on different preprocessing methods, the PLSR model with MSC pretreatment presented better results with coefficients of determination for prediction (R 2 P) of 0.9837 and 0.9323 for MC and freezable water content, respectively. Instead of selecting full range spectra data, optimal wavelengths were identified based on the regression coefficients (RC) method.

Applications in Vegetables

215

Then, two linear calibration algorithms named PLSR and multiple linear regression (MLR) and a nonlinear calibration algorithm named backpropagation (BP) neural network were used to establish models to predict quality attributes of samples simultaneously. The results showed that the RC-MLR with R 2 P of 0.9359 and 0.8592 was considered as the best for determining MC and freezable water content of sweet potato slices. A. Kjaer et al. [28] investigated the use of HSI to detect and quantify chlorophyll (Chl) and total glycoalkaloid concentrations (TGA) in potatoes. To create a set of tubers with different concentrations of Chl and TGA, potatoes of four varieties were wounded or treated with red, blue, red/blue, UV-a, UV-b, or UV-c light. HSI analyses were performed with a reflection based setup implemented in an industrial potato sorting machine. After hyperspectral analyses, the peel was sampled, and the concentrations of Chl and total TGA were determined. Results showed that the HSI system predicted the concentrations of Chl with a relatively high degree of accuracy, and a prediction R 2 value of 0.92. Prediction of TGA concentrations performed to a much lesser extent, and the overall prediction R 2 value was only 0.21. Moderate soil covers only affected the prediction powers to a minor degree.

15.4 TOMATO Parameters checked by HSI on tomatoes are ripening stages, damage, pests, color, and chemical or physical compounds. Tomatoes (Lycopersicon esculentum, Mill. cv. Capita F1) were harvested at different ripening stages [29]. Spectral images from 400 to 700 nm with a resolution of 1 nm were recorded. After recording, samples were taken from the fruit wall and the lycopene, lutein, β-carotene, chlorophyll-a and chlorophyll-b concentrations were measured using HPLC. The relation between the compound concentrations measured with HPLC and the spectral images was analyzed using partial least squares regression. The Q 2 error of the predicted lycopene concentration, determined from the PLS procedure, was 0.95 on a pixel basis and 0.96 on a tomato basis. The Q 2 error of the other compounds varied from 0.73 to 0.84. Pixel-based regression made it possible to construct concentration images of the tomatoes, which showed nonuniform ripening. Lycopene is a major carotenoid in tomatoes and detecting changes in its content can be used to monitor the ripening of tomatoes. In the study of J. Qin et al. [30], a benchtop point-scan Raman chemical imaging system was developed to detect and visualize internal lycopene distribution during postharvest ripening of tomatoes. Tomato samples at different ripeness stages (i.e., green, breaker, turning, pink, light red, and red) were selected and cut open for imaging. Hyperspectral Raman images were acquired from fruit cross-sections in the wavenumber range of 200–2,500 cm−1 with a spatial resolution of 1 mm. A polynomial curve-fitting method was used to correct for the underlying fluorescence background in the original spectra. A hyperspectral image classification method was developed based on spectral information divergence to identify lycopene in the tomato cross-sections. Raman chemical images were created to visualize the spatial distribution of the lycopene for different ripeness stages. The system was also configured to test the feasibility of utilizing spatially offset Raman spectroscopy (SORS) technique for subsurface detection of a Teflon slab placed under samples of outer pericarp cut in 5-mm and 10-mm thick slices from green and breaker tomatoes. The results showed that the Raman spectrum of Teflon can be extracted from the SORS measurements of the pericarps placed over the Teflon, demonstrating the potential of the future development

216

Leo M.L. Nollet, et al.

of a Raman-based nondestructive approach for the subsurface detection of lycopene as an indicator of tomato maturity. It is a challenge to detect surface defects on tomatoes in an automatic sorting line. In the paper of J. Sing et al. [31], the efforts are described for recognizing bruises on tomatoes in this paper. Only areas that are perceived as soft spots are considered as bruises. Due to the subtle color contrast, it is not an easy job for a conventional machine vision system to distinguish the bruised tissues from the sound ones. Therefore, an attempt was made for bruise detection on tomatoes by using a HSI setup in the wavelength region between 400 and 1,000 nm. The chemometrics tools were used to extract the effective wavebands for surface defects detection and for identifying the stem-end. The orientation of the tomatoes in the field of view of the camera may affect the final classification accuracy. Cuticle cracks on tomatoes are potential sites of pathogenic infection that may cause deleterious consequences both to consumer health and to fresh and fresh-cut produce markets [32]. The feasibility of hyperspectral NIR imaging technique in the spectral range of 1,000 nm to 1,700 nm was investigated for detecting defects on tomatoes. Spectral information obtained from the regions of interest on both defect areas and sound areas were analyzed to determine an optimal waveband ratio that could be used for further image processing to discriminate defect areas from the sound tomato surfaces. Unsupervised multivariate analysis method, such as PCA, was also explored to improve detection accuracy. Threshold values for the optimized features were determined using linear discriminant analysis. Results showed that tomatoes with defects could be differentiated from the sound ones, with an overall accuracy of 94.4%. Tomatoes are sometimes a source of foodborne illness, mainly Salmonella spp. Growth cracks on tomatoes can be a pathway for bacteria, so its detection prior to consumption is important for public health [33]. Multispectral VNIR reflectance imaging techniques were used by J. Dahong et al. to determine optimal wavebands for the classification of defect tomatoes. Hyperspectral reflectance images were collected from samples of naturally cracked tomatoes. To classify the resulting images, the selected wavelength bands were subjected to two-band permutations, and a supervised classification method was used. The results showed that two optimal wavelengths, 713.8 nm and 718.6 nm, could be used to identify cracked spots on tomato surfaces with a correct classification rate of 91.1%. A multispectral fluorescence-based imaging algorithm was developed to detect frass contamination on mature Campari tomatoes [34]. Tomato images were acquired using a hyperspectral fluorescence line-scan imaging system with violet LED excitation, then analyzed for wavelength selection. The fluorescence intensities at five wavelengths, 515 nm, 640 nm, 664 nm, 690 nm, and 724 nm, were used to compute three simple ratio functions to detect frass contamination. The contamination spots were created on the tomato surfaces using four low-concentration frass dilutions. The algorithms detected over 99% of the 0.2 kg/L and 0.1 kg/L frass contamination spots and successfully ­differentiated these spots from tomato skin surfaces, stem scars, and stems. However, differentiation of the 0.05 kg/L and 0.02 kg/L frass contamination spots was more difficult. Adjusting the algorithm to successfully detect 95% of the 0.05 kg/L spots also resulted in false-positive pixel detections occurring on 28% of the tomatoes. The paper of X. Yin et al. determines the canopy spectral reflectance and disease severity of processing tomato bacterial spot disease in field and disease nursery [35]. The canopy spectral reflectance in different varieties and different growth stages were

Applications in Vegetables

217

transformed into first-order differential, second-order differential, and inverse logarithm spectral reflectance. The correlation of the disease severity and primitive spectral reflectance, first-order differential, second-order differential, and inverse logarithm spectral reflectance in order to look for the sensitive band of diseased processing tomato were analyzed. The sensitive spectrum was established for the model of disease severity and test. It had practical value to monitor the large area of processing tomato bacterial spot disease using hyperspectral. As consumers buy with their eyes, color is considered one of the most important quality parameters of food products. Traditionally, this is defined by human inspection or measured using a colorimeter or a spectrophotometer. As the first is subjective and prone to factors such as fatigue, this is not ideal for industrial use. The second only measures a small area of the food product, making it difficult to get a clear overview of the color of the whole sample. To overcome these limitations, HSI has been used in this research to measure the postharvest color of vine tomatoes [36]. Two methods to calculate the color based on hyperspectral images are compared. The first is the use of a direct method to calculate the color from the spectra in terms of CIELAB values, while the second method is a soft modeling approach involving multivariate statistics. The soft modeling method was found to achieve the best results (R L*2 = 0.86; R a*2 = 0.93; Rb*2 = 0.42, R Hue2 = 0.95, RChroma2 = 0.51), but its applicability is limited to the range of products on which the models have been trained. The direct method is more generally applicable, but was found to lack robustness against intensity variations due to the curvature and glossiness of the tomatoes. The objective of the study of A. Rahman et al. was to develop a nondestructive method to evaluate chemical components such as MC, pH, and soluble solid content (SSC) in intact tomatoes by using HSI in the range of 1,000–1,550 nm [37]. The mean spectra of the 95 matured tomato samples were extracted from the hyperspectral images, and multivariate calibration models were built by using partial least squares regression with different preprocessing spectra. The results showed that the regression model developed by PLS regression based on Savitzky–Golay (S–G) first-derivative preprocessed spectra resulted in better performance for MC, pH, and the smoothing preprocessed spectra-based model resulted in better performance for SSC in intact tomatoes compared to models developed by other preprocessing methods, with correlation coefficients (rpred) of 0.81, 0.69, and 0.74 with root mean square error of prediction (RMSEP) of 0.63%, 0.06%, and 0.33% Brix, respectively. The full wavelengths were used to create chemical images by applying regression coefficients resulting from the best PLS regression model.

15.5 CUCUMBER Parameters of importance on cucumbers are chilling injury, defects, different optical parameters, external and internal quality, diseases, and infestation. X. Cheng et al. present a novel method that combines principal component and FLD method [38]. The method is applied to the detection of chilling injury on cucumbers. Based on tests on different types of samples, results show that this new integrated PCA– FLD method outperforms the PCA and FLD methods when they are used separately for classifications. Hyperspectral images of cucumbers were acquired before and during cold storage treatment as well as during subsequent room temperature (RT) storage to explore the potential for the detection of chilling induced damage in whole cucumbers [39].

218

Leo M.L. Nollet, et al.

Region of interest spectral features of chilling injured areas, resulting from cold storage treatments at 0°C or 5°C, showed a reduction in reflectance intensity during multiday post-chilling periods of RT storage. Large spectral differences between good-smooth skins and chilling injured skins occurred in the 700–850 nm VNIR region. A number of data processing methods, including simple spectral band algorithms, second difference, and PCA, were attempted to discriminate the ROI spectra of good cucumber skins from those of chilling injured skins. Results revealed that using either a dual-band ratio algorithm (Q811/756) or a PCA model from a narrow spectral region of 733–848 nm could detect chilling injured skins with a success rate of over 90%. Furthermore, the dual-band algorithm was applied to the analysis of images of cucumbers at different conditions, and the resultant images showed more correct identification of chilling injured spots than other processing methods. It is desirable to detect chilling injury at early stages and/or remove chilling injured cucumbers during sorting and grading [40]. This research was aimed to apply hyperspectral imaging technique, combined with feature selection methods and supervised classification algorithms, to detect chilling injury in cucumbers. Hyperspectral reflectance (500–675 nm) and transmittance (675–1,000 nm) images for normal and chilling injured cucumbers were acquired, using an in-house developed online HSI system. Three feature selection methods including mutual information feature selection (MIFS), max-relevance min-redundancy (MRMR), and sequential forward selection (SFS) were used and compared for optimal wavebands selection. Supervised classifications with naïve Bayes (NB), support vector machine, and k-nearest neighbor were then implemented for the two-class (i.e., normal and chilling) and three-class (i.e., normal, lightly chilling, and severely chilling) classifications based on the spectral and image analysis at selected two-band ratios. It was found that the majority of the optimal wavebands selected by MIFS, MRMR, and SFS for both two-class and three-class classifications were from the spectral transmittance images in the short-NIR region. The SFS feature selection method together with the SVM classifier resulted in the best overall classification accuracy of 100%, and the overall accuracy of 90.5% for the three-class classification, based on the spectral analysis. The classification results based on the textural features (first-order statistics and second-order statistics features) extracted from the optimal two-band ratio images were comparable to those achieved using the spectral features, with the best overall accuracies of 100% and 91.6% for the two-class and the three-class classifications, respectively. Automated detection of defects on freshly harvested pickling cucumbers will help the pickle industry provide higher [41] quality pickle products and reduce potential economic losses. Research was conducted on using a hyperspectral imaging system for detecting defects on pickling cucumbers caused by mechanical stress. A NIR HSI system was used to capture both spatial and spectral information from cucumbers in the spectral region of 900–1,700 nm. Cucumber samples were subjected to two forms of mechanical loading, dropping and rolling, to simulate stress caused by mechanical harvesting. Hyperspectral images were acquired from the cucumbers over time periods of 0, 1, 2, 3, and 6 days after mechanical stress. Hyperspectral image processing methods, including PCA and wavelength selection, were developed to separate normal and mechanically injured cucumbers. Results showed that reflectance from normal or nonbruised cucumbers were consistently higher than that from bruised cucumbers. The spectral region between 950 and 1,350 nm was found to be most effective for bruise detection. The hyperspectral imaging system detected all mechanically injured cucumbers immediately after they were bruised. The overall detection accuracy was 97% within 2 hours of bruising, and it was lower as

Applications in Vegetables

219

time progressed. Lower detection accuracies for the prolonged times after bruising were attributed to the self-healing of the bruised tissue after mechanical injury. Hyperspectral images of cucumbers under a variety of conditions [42] were acquired to explore the potential for the detection of chilling induced damage in whole cucumbers. ROI spectral features of chilling injured areas, resulting from chilling treatment at 0°C, showed the reduction of reflectance intensity over the period at post-chilling room temperature (RT) storage. A large spectral difference between good, smooth skins and chilling-injured skins occurred in the 700–850 nm VNIR region. Both simple band ratio algorithms and PCA were attempted to discriminate the ROI spectra of good cucumber skins from those of chilling injured ones. Results revealed that both the dual-band ratio algorithm (R811 nm/R756 nm) and a PCA model from a narrow spectral region of 733–848 nm can detect chilling-injured skins with a success rate of over 90%. The results also suggested that chilling injury is relatively difficult to detect at the initial post-chilling RT stage, especially during the first 0–2 days in storage, due to insignificant manifestation of chilling induced symptoms. HSI is useful for detecting internal defects of pickling cucumbers. The technique, however, is not yet suitable for high-speed online implementation due to the challenges in analyzing large-scale hyperspectral images. The research of H. Cen et al. [43] aimed to select the optimal wavebands from the hyperspectral image data, so that they can be deployed in either a hyperspectral or multispectral imaging-based inspection system for the automatic detection of internal defects of pickling cucumbers. Hyperspectral reflectance (400–700 nm) and transmittance (700–1,000 nm) images were acquired, using an inhouse developed hyperspectral imaging system running at two conveyor speeds of 85 and 165 mm/s, for 300 “Journey” pickling cucumbers before and after internal damage was induced by mechanical load. Minimum redundancy–maximum relevance (MRMR) was used for optimal wavebands selection, and the loadings of PCA were also applied for qualitatively identifying the important wavebands that are related to the specific features. Discriminant analysis with Mahalanobis distance classifier was performed for the two-class (i.e., normal and defective) and three-class (i.e., normal, slightly defective, and severely defective) classifications using the mean spectra and textural features (energy and variance) from the regions of interest in the spectral images at selected waveband ratios. The classification results based on MRMR wavebands selection were generally better than those from PCA-based classifications. The two-band ratio of 887/837 nm from MRMR gave the best overall classification results with an accuracy of 95.1% and 94.2% at the conveyor speeds of 85 and 165 mm/s, respectively, for the two-class classification. The highest classification accuracies for the three-class classification based on the optimal two-band ratio of 887/837 nm were 82.8% and 81.3% at the conveyor speeds of 85 and 165 mm/s, respectively. The mean spectra-based classification achieved better results than the textural feature-based classification, except in the three-class classification for the higher conveyor speed. The overall classification accuracies for all selected waveband ratios at the low conveyor speed were slightly higher than those at the higher conveyor speed, since the low speed resulted in more scan lines, thus higher spatial resolution hyperspectral images. This paper reports on the development of a HSI prototype for evaluation of external and internal quality of pickling cucumbers [44]. The prototype consisted of a twolane round belt conveyor, two illumination sources (one for reflectance and one for transmittance), and a HSI unit. It had a novel feature of simultaneous imaging under reflectance mode covering the visible region of 400–675 nm and transmittance mode for 675–1,000 nm, coupled with real-time, continuous calibration of reflectance and

220

Leo M.L. Nollet, et al.

transmittance images for each cucumber using reference standards installed on the conveyor. Reflectance information was used for evaluating the external characteristics of cucumbers (i.e., skin color), transmittance for internal defect detection (hollow center), and the combined reflectance and transmittance for predicting flesh firmness. The prototype was tested on “Journey” pickling cucumbers harvested in 2006 and 2007 for predicting skin and flesh color, flesh firmness, and internal defect. Hyperspectral images were processed to extract mean spectra for individual cucumbers, and partial least squares analysis was performed to predict flesh firmness, skin and flesh color, and the presence of internal defect. The prototype performed relatively well in predicting skin color with the coefficient of determination of 0.76 and 0.75 for chroma and hue, respectively; however, it had poor prediction of flesh color and firmness. Transmittance data in the spectral region of 675–1,000 nm provided excellent detection of internal defect for the pickling cucumbers, with the detection accuracy greater than 90%. Hyperspectral imaging under transmittance mode has shown promising results for detecting internal defect in pickling cucumbers, however, the technique still cannot meet the online speed requirement because it needs to acquire and process a large amount of image data [45]. This study was conducted on selecting important wavebands as a basis for developing an online imaging system to detect internal defect in pickling cucumbers. “Journey” pickling cucumbers were subjected to mechanical stress to induce damage in the seed cavity. Hyperspectral transmittance/reflectance images were acquired from normal and defective cucumbers using a prototype hyperspectral reflectance (400–740 nm)/ transmittance (740–1,000 nm) imaging system. Optimal wavelengths were determined by correlation analysis on single, ratio, and difference of two pairs of wavelengths. A global image thresholding method was applied to the selected spectral images to identify defective cucumbers. Images at 740 nm were the best for single waveband classification with an overall accuracy of 87%. For ratios of two wavebands, 925 nm and 940 nm resulted in an overall classification accuracy of 85%, and for differences of two wavebands, images at 745 nm and 850 nm were the best with a classification accuracy of 91%. All the selected wavebands were in the NIR region, which is more effective for internal defect detection compared to the visible region. HSI technology, which can integrate the advantages of spectral detection and image detection [46], meets the need of detecting the cucumber diseases fast and nondestructively. HSI technology is adopted to detect the cucumber downy mildew fast and nondestructively. First, hyperspectral images of cucumber leaves infected downy mildew are acquired by an hyperspectral image acquisition system. Optimum wavelengths are collected by PCA to get the featured images. Then image fusion technology is adopted to combine collected images with the featured images to form new images by pixel level image fusion. Finally, the methods of image enhancement, binarization, corrosion, and dilatation treatments are carried out, so the cucumber downy mildew is detected. The result shows that the accuracy rate of the algorithm for detecting cucumber disease can reach nearly 90%. Fruit fly infestation can be a serious problem in pickling cucumber production [47]. In the United States and many other countries, there is zero tolerance for fruit flies in pickled cucumber products. Currently, processors rely on manual inspection to detect and remove fruit fly-infested cucumbers, which is labor intensive and also prone to error due to human fatigue and the difficulty of visually detecting infestation that is hidden inside the fruit. A HSI system operating in an integrated mode of reflectance and transmittance was used to detect fruit fly-infested pickling cucumbers.

Applications in Vegetables

221

Hyperspectral reflectance (450–740 nm) and transmittance (740–1,000 nm) images were acquired simultaneously for 329 normal (infestation-free) and fruit fly-infested pickling cucumbers of three size classes with mean diameters of 16.8, 22.1, and 27.6 mm, respectively. Mean spectra were extracted from the hyperspectral image of each cucumber and were then corrected for the fruit size effect using a diameter correction equation. Partial least squares discriminant analyses for the reflectance, transmittance, and their combined data were performed for differentiating normal and infested pickling cucumbers. With reflectance mode, the overall classification accuracies for the three size classes and the mixed class were between 82% and 88%, respectively, whereas transmittance achieved better classification results with overall accuracies of 88%–93%. Integration of reflectance and transmittance did not result in noticeable improvements, compared to transmittance mode. The HSI system performed better than manual inspection, which had an overall accuracy of 75% and whose performance decreased significantly for smaller size cucumbers. Mechanical injury often causes hidden internal damage to pickling cucumbers, which lowers the quality of pickled products and can incur economic losses to the processor. A NIR HSI system was developed to capture hyperspectral images from pickling cucumbers in the spectral region of 900–1,700 nm [48]. The system consisted of an imaging spectrograph attached to an InGaAs camera with line-light fiber bundles as an illumination source. Hyperspectral images were taken from the pickling cucumbers at 0–3 and 6 days after they were subjected to dropping or rolling under load which simulated damage caused by mechanical harvesting and handling systems. PCA, band ratio, and band difference were applied in the image processing to segregate bruised cucumbers from normal cucumbers. Bruised tissue had consistently lower reflectance than normal tissue and the former increased over time. Best detection accuracies from the PCA were achieved when a bandwidth of 8.8 nm and the spectral region of 950–1,350 nm were selected. The detection accuracies from the PCA decreased from 95% to 75% over the period of 6 days after bruising, which was attributed to the self-healing of the bruised tissue after mechanical injury. The best band ratio of 988 and 1,085 nm had detection accuracies between 93% and 82%, whereas the best band difference of 1,346 and 1,425 nm had accuracies between 89% and 84%. The general classification performance analysis suggested that the band ratio and difference methods had similar performance, but they were better than the PCA.

15.6 LETTUCE Different parameters are checked by HSI in lettuce. An hyperspectral fluorescence imaging system using ultraviolet-A excitation (320– 400 nm) was used to detect bovine fecal contamination on the abaxial and adaxial surfaces of romaine lettuce and baby spinach leaves [49]. An image processing algorithm to detect the fecal contamination spots was investigated while it correctly identified the clean leaf surfaces. The developed algorithm could successfully detect the fecal contamination spots on the adaxial and abaxial surfaces of romaine lettuce and baby spinach. In the study of J. Sun et al., a fast and nondestructive technology for the prediction of the nitrogen content in lettuce leaves based on HSI technology was developed [50]. First, hyperspectral images of lettuce leaves in the visible and NIR (390–1,050 nm) regions were acquired by the HSI system, and then the corresponding nitrogen content in the lettuce leaves were obtained by Kjeldahl method successively. The binary mask

222

Leo M.L. Nollet, et al.

image was successfully determined by the method of dividing the image of very high reflectance intensity by the image of very low reflectance intensity with a certain threshold, and ROI in the sample of lettuce leaf was determined by removing the regions of noise using the acquired binary mask image. Some information is noisy and redundant. In fact, this leads to the difficulty of meeting the needs for fast and efficient detection of some objects. So, it is very hard to be directly used for on-line industrial application in our daily life. Therefore, effective selection of several characteristic wavelengths is necessary for the hyperspectral images. The initial investigation was carried out by using a PCA to identify a number of potential characteristic wavelengths (662.9 nm, 711.7 nm, 735.0 nm, 934.6 nm) according to the weight coefficient distribution curve of the first three principal component images (PC1, PC2, PC3) under the full wavelengths. Both spectral data and texture data based on a co-occurrence matrix were extracted from the four characteristic wavelength images on the ROI, and the texture data of the first three principal component images were also extracted simultaneously. Then spectral data from four characteristic wavelength images, texture data (from four characteristic wavelength images, from the first three principle component images) and the combined data were utilized to develop different SVR models to predict the nitrogen content in the lettuce leaves, respectively. According to the performance of all the SVR models in the calibration set and the prediction set, the experiment results show that, from the calibration performance index, the model based on spectral data combined texture data from four characteristic wavelength images is the best with a coefficient of determination (R 2 C = 0.996) and the root-mean-square error of calibration (RMSEC) of 0.034. From the prediction performance index, however, the model based merely on spectral data is the best with a coefficient of determination (R 2 P = 0.86) and the RMSEP of 0.22. The study of Sun et al. was carried out to identify nutrient elements (respectively, nitrogen, phosphorus, potassium) in lettuce leaves rapidly and nondestructively using HSI technology [51]. Spectral information of the lettuce samples was collected based on the spectral processing of the RIO region. Savitzky–Golay smoothing (SG) coupled with wavelet transform (WT), respectively, using db4, db6, sym5 as wavelet basis functions, were used to pretreat lettuce leaves spectrum. Successive projections algorithm (SPA) was used to extract the most influential wavelengths. Finally, the identification model was built based on support vector machine (SVM). The results showed that the model based on lettuce leaf spectroscopy pretreated by SG+WT, using db4 as wavelet basis function were best, the prediction accuracy of SVM models of different nutrient elements (respectively, nitrogen, phosphorus, potassium) were 91.84%, 93.75%, 95.83%, which were better than those of other models. Besides, the choice of appropriate wavelet base function will affect the accuracy of classification model. Furthermore, the results also showed that SG+WT+SPA+SVM is an effective method based on HSI technique to identify of nutrient elements (respectively, nitrogen, phosphorus, potassium) in lettuce leaves. Fresh-cut lettuce sold in modified atmosphere packaging (MAP) is a desirable, but highly perishable product. Decay of tissue can start a few days after processing and may be difficult to detect by quick visual observation [52]. A system for early detection of decay and gradual evaluation of its progress is important both for lettuce processing industry and for breeding companies and institutions assessing quality of new cultivars and breeding lines. Simko et al. developed two lettuce decay indices (LEDI) that can be used to detect decay of leaf tissue. One of the indices (LEDI4) is based on three wavelengths identified from HSI, while the second index (LEDICF) is based on chlorophyll fluorescence imaging. In addition to detecting lettuce decay, the indices identified tissue damaged by freezing temperatures. LEDI4 and LEDICF showed almost 97% accuracy in

Applications in Vegetables

223

classifying tissue as being fresh or decayed when tested on red, dark green, green, light green, and yellow leaves. Specificity of the indices decreased when tested on fresh tissue with a very limited amount of chlorophyll that visually appeared to be almost white. Both indices detected lettuce decay without opening plastic MAP bags. Rapid VNIR HSI methods, employing both a single waveband algorithm and multispectral algorithms, were developed in order to discriminate between sound and discolored lettuce [53]. Reflectance spectra for sound and discolored lettuce surfaces were extracted from hyperspectral reflectance images obtained in the 400–1,000 nm wavelength range. The optimal wavebands for discriminating between discolored and sound lettuce surfaces were determined using one-way ANOVA. Multispectral imaging algorithms developed using ratio and subtraction functions resulted in enhanced classification accuracy of above 99.9% for discolored and sound areas on both adaxial and abaxial lettuce surfaces. Ratio imaging (RI) and subtraction imaging (SI) algorithms at wavelengths of 552/701 nm and 557–701 nm, respectively, exhibited better classification performances compared to results obtained for all possible two-waveband combinations. The rapid detection of biological contaminants such as worms in fresh-cut vegetables is necessary to improve the efficiency of visual inspections carried out by workers. Multispectral imaging algorithms were developed using VNIR and NIR HSI techniques to detect worms in fresh-cut lettuce [54]. The optimal wavebands that can detect worms in fresh-cut lettuce were investigated for each type of HSI using one-way ANOVA. Worm-detection imaging algorithms for VNIR and NIR imaging exhibited prediction accuracies of 97.00% (RI547/945) and 100.0% (RI1064/1176, SI1064–1176, RSI-I (1064–1173)/1064, and RSI-II (1064–1176)/(1064 + 1176)), respectively. The two HSI techniques revealed that spectral images with a pixel size of 1 × 1 mm or 2 × 2 mm had the best classification accuracy for worms. The results demonstrate that hyperspectral reflectance imaging techniques have the potential to detect worms in fresh-cut lettuce. In the study of C. Mo et al., an online quality measurement system for detecting foreign substances on fresh-cut lettuce was developed using hyperspectral reflectance imaging [55]. The online detection system with a single hyperspectral camera in the range of 400–1,000 nm was able to detect contaminants on both surfaces of freshcut lettuce. Algorithms were developed for this system to detect contaminants such as slugs and worms. The optimal wavebands for discriminating between contaminants and sound lettuce as well as between contaminants and the conveyor belt were investigated using the one-way ANOVA method. The subtraction imaging (SI) algorithm to classify slugs resulted in a classification accuracy of 97.5%, sensitivity of 98.0%, and specificity of 97.0%. The ratio imaging (RI) algorithm to discriminate worms achieved classification accuracy, sensitivity, and specificity rates of 99.5%, 100.0%, and 99.0%, respectively.

15.7 SPINACH HSI was used on spinach for the analysis of pigments, bacterial contamination, and quality control. HSI covering spectral range of 874–1,734 nm was used to measure spinach leaf pigments content (chlorophyll-a (Chla), chlorophyll-b (Chlb), total chlorophyll (tChl), carotenoids (Car)) under storage at 20°C (Sample set 1) and 4°C (Sample set 2) [56]. A sample set combining the two sample sets was formed, partial least squares models on full spectra obtained acceptable results for all sample sets with correlation coefficient of prediction

224

Leo M.L. Nollet, et al.

(R P) near or over 0.8. Random frog was used to select 20, 20, 20, 22 optimal wavelengths for Chla, Chlb, tChl, and Car from the combined sample set, respectively. PLS models on optimal wavelengths obtained better results than corresponding full spectra PLS models. The visualization of the distribution map of pigments content were acquired by applying the PLS models on pixels within the hyperspectral image. The overall results indicated that HSI could be used for spinach leaf pigments content measurement, providing an alternative for real-world on-line vegetables quality monitor. Hyperspectral fluorescence imaging coupled with multivariate image analysis techniques for the detection of fecal contaminates on spinach leaves (Spinacia oleracea) was evaluated [57]. Violet fluorescence excitation was provided at 405 nm and light emission was recorded from 464 to 800 nm. Partial least square discriminant analysis and wavelength ratio methods were compared for detection accuracy for fecal contamination. Fluorescence emission profiles of spinach leaves were monitored over a 27 days storage period; peak emission blue-shifts were observed over the storage period accompanying a color change from green to green–yellow–brown hue. The PLSDA model developed correctly detected fecal contamination on 100% of relatively fresh green spinach leaves used in this investigation, which also had soil contamination. The PLSDA model had 19% false positives for nonfresh post storage leaves. A wavelength ratio technique using four wavebands (680, 688, 703, and 723 nm) was successful in identifying 100% of fecal contaminates on both fresh and nonfresh leaves. In the study of C. D. Everard et al., three HSI configurations coupled with two multivariate image analysis techniques are compared for the detection of fecal contamination on spinach leaves [58]. Fluorescence imaging in the visible region with ultra violet (UV) and violet excitation sources and reflectance imaging in the visible to NIR regions were investigated. Partial least squares discriminant analysis and two-band ratio analysis techniques were used to compare these HSI configurations. High detection accuracy was found for the two fluorescence HSI configurations compared to the VNIR HSI. Both fluorescence HSI configurations had 100% detection rates for fecal contamination up to 1:10 dilution level and violet HSI had 99% and 87% detection rates for 1:20 and 1:30 levels, respectively. Results indicated that fluorescence imaging with the violet excitation performed superior to HSI with UV excitation for the detection of a range of diluted fecal contamination on leafy greens. About 5% or less false positives were observed for the fluorescence HSI configurations and were associated with yellow hue on the leaves. A rapid method based on HSI for the detection of Escherichia coli contamination in fresh vegetables was developed [59]. E. coli K12 was inoculated into spinach with different initial concentrations. Samples were analyzed using a colony count and a hyperspectroscopic technique. A hyperspectral camera of 400–1,000 nm with a spectral resolution of 5 nm was employed to acquire hyperspectral images of packaged spinach. Reflectance spectra were obtained from various positions on the sample surface and pretreated using Sawitzky–Golay. Chemometrics including PCA and artificial neural network (ANN) were then used to analyze the preprocessed data. The PCA was implemented to remove redundant information of the hyperspectral data. The ANN was trained using Bayesian regularization and was capable of correlating hyperspectral data with number of E. coli. Once trained, the ANN was also used to construct a prediction map of all pixel spectra of an image to display the number of E. coli in the sample. The prediction map allowed a rapid and easy interpretation of the hyperspectral data. Different procedures for monitoring the evolution of leafy vegetables, under plastic covers during cold storage, have been studied [60]. Fifteen spinach leaves were put inside

Applications in Vegetables

225

Petri dishes covered with three different plastic films and stored at 4°C for 21 days. Hyperspectral images were taken during this storage. A radiometric correction is proposed in order to avoid the variation in transmittance of the plastic films during time in the hyperspectral images. Afterwards, three spectral preprocessing procedures (no preprocess, Savitsky–Golay and SNV, combined with PCA) were applied to obtain different models. The corresponding artificial images of scores were studied by means of ANOVA to compare their ability to sense the aging of the leaves. All models were able to monitor the aging through storage. Radiometric correction seemed to work properly and could allow the supervision of shelf-life in leafy vegetables through commercial transparent films. The research of B. Diezma is focused on the application of hyperspectral images for the supervision of quality deterioration in ready to use leafy spinach during storage [61]. Two sets of samples of packed leafy spinach were considered: (a) a first set of samples was stored at 20°C (E-20) in order to accelerate the degradation process, and these samples were measured the day of reception in the laboratory and after 2 days of storage; (b) a second set of samples was kept at 10°C (E-10), and the measurements were taken throughout storage, beginning the day of reception and repeating the acquisition of images 3, 6, and 9 days later. Twenty leaves per test were analyzed. Hyperspectral images were acquired with a push-broom CCD camera equipped with a spectrograph VNIR (400–1,000 nm). Calibration set of spectra was extracted from E-20 samples, containing three classes of degradation: class A (optimal quality), class B, and class C (maximum deterioration). Reference average spectra were defined for each class. Three models, computed on the calibration set, with a decreasing degree of complexity were compared, according to their ability for segregating leaves at different quality stages (fresh, with incipient and nonvisible symptoms of degradation, and degraded): spectral angle mapper distance (SAM), partial least squares discriminant analysis models (PLS-DA), and a nonlinear index (Leafy Vegetable Evolution, LEVE) combining five wavelengths were included among the previously selected by CovSel procedure. In sets E-10 and E-20, artificial images of the membership degree according to the distance of each pixel to the reference classes were computed assigning each pixel to the closest reference class. The three methods were able to show the degradation of the leaves with storage time.

15.8 PEPPERS The objective of the study of Z. Schmilovitch et al. was to develop a nondestructive method to evaluate and to map quality indices in bell pepper [62]. Three cultivars of bell pepper (“Ever Green,” “No. 117,” and “Celica”) were studied during maturation by using HSI in the visible and NIR (550–850 nm) region. Peppers were marked in the flowering stage and 20 samples from each variety were collected weekly, along a growing period of 7 weeks, until full growth. Quality parameters such as total soluble solids, total chlorophyll, carotenoid, and ascorbic acid content were determined and correlated with the spectral data. Images of intact peppers were collected by an acousto-optic tunable filter (AOTF) hyperspectral charged-coupled-device (CCD) camera, in spectral resolution of 5 nm. Spectral information of the hyper cubes was analyzed by chemometric procedures. Partial least squares regression was used for model development. Comparisons were made between the PLS regression analysis of the reflectance spectra (R), and the preprocessed spectra such as the first derivative (D1 R), log(1/R), D1(log(1/R)), and D 2(log(1/R)).

226

Leo M.L. Nollet, et al.

Models were established to predict the quality attributes creating the basis for multiple sampling of a particular fruit or individual peppers from many fruits in the same time. High correlations were obtained by the established models with coefficients of determination of 0.95, 0.95, 0.97, and 0.72 for total soluble solids, total chlorophyll, carotenoid, and ascorbic acid content, respectively.

15.9 ONION Onion internal quality attributes such as dry matter content, SSC, and firmness are important for onion processors and consumers [63]. HSI could be a powerful tool for nondestruction evaluation of the internal quality of onions. However, the elliptical shape of the onion can result in uneven reflection at different positions on onion surface. Diffuse reflectance images were acquired by a line scan hyperspectral imaging system for onion internal quality evaluation after incorporation an elliptical shape correction algorithm. Four steps were followed: (1) Color images were acquired to retrieve geometric information of the onion; (2) Shape correction algorithm was performed on the diffuse reflectance hyperspectral image; (3) Spectral characteristics were extracted from the Region of interests in the corrected images; (4) Partial least square was used to build the correlation model between the internal quality attributes and spectral data. A liquid crystal tunable filter based NIR HSI system was used to assess onion internal quality (firmness, SSC, and dry matter content) [64]. Reflectance, transmittance, and interactance images were collected over the spectral region from 950 to 1,750 nm. The spectra were extracted from regions of interest in the spectral images, which were used to develop calibration models using the partial least square regression. For interactance images, the regions of interest were identified at 1,200 nm wavelength by a gray gradient algorithm. In order to reduce the number of spectral variables and increase the performance of the model, genetic algorithm was used to select optimal wavelengths. Results showed that the reflectance mode performed the best among the three sampling modes. The SSC and dry matter content could be better predicted than the firmness. Sour skin is a common bacterial disease that can affect most onion varieties wounds caused by mechanical injuries in the neck area. Currently, human visual inspection (HVI) is used by most onion-packing houses [65]. Obviously, HVI cannot prevent onions with only internal infections. A NIR HSI system was developed using a liquid crystal tunable filter (LCTF), and it was applied for sour skin detection in Vidalia onions.

15.10 BROCCOLI The use of HSI to (a) quantify and (b) localize total glucosinolates in florets of a single broccoli species has been examined by J. M. Hernández-Hierro et al. [66]. Two different spectral regions (vis-NIR and NIR), a number of spectral pretreatments, and different mask development strategies were studied to develop the quantitative models. These models were then applied to freeze-dried slices of broccoli to identify regions within individual florets which were rich in glucosinolates. The procedure demonstrates potential for the quantitative screening and localization of total glucosinolates in broccoli using the 950–1,650 nm wavelength range. These compounds were mainly located in the external part of florets.

Applications in Vegetables

227

ABBREVIATIONS ANN BPNN CARS DA FFNN FLD FRSTCA GA IPLS Knn LDA MRMR MC MSE MLR MSC MIFS PLS PCA QDA RBFNN ROC ROI RMSEP SG SFS SIMCA SAM SNV SPA SVDD SVM WT

artificial neural network backpropagation neural network competitive adaptive reweighted sampling algorithm discriminant analysis feed forward neural networks Fisher’s linear discriminant fuzzy-rough set model based on the thermal charge algorithm genetic algorithm interval partial least squares K-nearest neighbor linear discriminant analysis max-relevance min-redundancy mean centering mean square error multiple linear regression multiplicative scatter correction mutual information feature selection partial least squares principal component analysis quadratic discriminant analysis radial basis functions neural networks receiver operating characteristic region of interest root-mean-square errors of prediction Savitzky–Golay sequential forward selection soft independent modeling of class analogy spectral angle mapper distance standard normal variate successive projections algorithm support vector data description support vector machine wavelet transform

REFERENCES 1. Z. Zhou, Y. Zang, B. Shen, X. Zhou, X. Luo. Detection of cowpea weevil (Callosobruchus maculatus (F.)) in soybean with hyperspectral spectrometry and a backpropagation neural network. (ICNC), 2010 Sixth International Conference on Natural Computation. 2. M. Huang, X. Wan, M. Zhang, Q. Zhu. Detection of insect-damaged vegetable soybeans using hyperspectral transmittance image. Journal of Food Engineering, 116 (1), 2013, 45–49. 3. Y. Ma, M. Huang, B. Yang, Q. Zhu. Automatic threshold method and optimal wavelength selection for insect-damaged vegetable soybean detection using hyperspectral images. Computers and Electronics in Agriculture, 106, 2014, 102–110.

228

Leo M.L. Nollet, et al.

4. V. Chelladurai, K. Karuppiah, D. S. Jayas, P. G. Fields, N. D. G. White. Detection of Callosobruchus maculatus (F.) infestation in soybean using soft X-ray and NIR hyperspectral imaging techniques. Journal of Stored Products Research, 57, 2014, 43–48. 5. M. Huang, Q. Wang, M. Zhang, Q. Zhu. Prediction of color and moisture content for vegetable soybean during drying using hyperspectral imaging technology. Journal of Food Engineering, 128, 2014, 24–30. 6. S. K. Singh, V. Hoyos-Villegas, J. D. Ray, J. R. Smith, F. B. Fritschi, Quantification of leaf pigments in soybean (Glycine max (L.) Merr.) based on wavelet decomposition of hyperspectral features. Field Crops Research, 149, 2013, 20–32. 7. S. T. Monteiro, Y. Minekawa, Y. Kosugi, T. Akazawa, K. Oda. Prediction of sweetness and amino acid content in soybean crops from hyperspectral imagery. ISPRS Journal of Photogrammetry & Remote Sensing, 62, 2007, 2–12. 8. K. Tan, Y. Chai, W. Song, X. Cao. Detection of isoflavones content in soybean based on hyperspectral imaging technology. Sensors & Transducers, 169 (4), 2014, 55–60. 9. A. A. Gowen, C. P. O’Donnell, M. Taghizadeh, P. J. Cullen, J. M. Frias, G. Downey. Hyperspectral imaging combined with principal component analysis for bruise damage detection on white mushrooms (Agaricus bisporus). Journal of Chemometrics, 22, 2008, 259–267. 10. R. Rojas-Moraleda, N. A. Valous, A. Gowen, C. Esquerre, S. Härtel, L. Salinas, C. O’Donnell. A frame-based ANN for classification of hyperspectral images: assessment of mechanical damage in mushrooms. Neural Comput & Applic, 28(Suppl 1), 2016. doi: 10.1007/s00521-016-2376-7 11. A. A. Gowen, C. P. O’Donnell. Development of algorithms for detection of mechanical injury on white mushrooms (Agaricus bisporus) using hyperspectral imaging. http://proceedings.spiedigitallibrary.org/ on 12/31/2015. 12. A. A. Gowen, M. Taghizadeh, C. P. O’Donnell. Identification of mushrooms subjected to freeze damage using hyperspectral imaging. Journal of Food Engineering, 93, 2009, 7–12. 13. E. Gaston, J. M. Frias, P. J. Cullen,C. P. O’donnell, A. A. Gowen. Prediction of polyphenol oxidase activity using visible near-infrared hyperspectral imaging on mushroom (Agaricus bisporus) caps. Journal of Agricultural and Food Chemistry, 58, 2010, 6226–6233. 14. A. A. Gowen, C. P. O’Donnell, M. Taghizadeh, E. Gaston, A. O’Gorman, P. J. Cullen, J. M. Frias, C. Esquerre, G. Downey. Hyperspectral imaging for the investigation of quality deterioration in sliced mushrooms (Agaricus bisporus) during storage. Sensing and Instrumentation for Food Quality and Safety, 2, 2008, 133–143. 15. A. A. Gowen, C. P. O’Donnell, J. M. Frias, G. Downey. Hyperspectral Imaging for Mushroom (Agaricus Bisporus) Quality Monitoring. 2009 IEEE. 16. K. Mollazade. Non-destructive Identifying Level of Browning Development in Button Mushroom (Agaricus bisporus) Using Hyperspectral Imaging Associated with Chemometrics. 17. M. Taghizadeh, A. Gowen, C. P. O’Donnell. Prediction of white button mushroom (Agaricus bisporus) moisture content using hyperspectral imaging. Sensing and Instrumentation for Food Quality and Safety, 3, 2009, 219–226. 18. M. Taghizadeh, A. Gowen, P. Ward, C. P. O’Donnell. Use of hyperspectral imaging for evaluation of the shelf-life of fresh white button mushrooms (Agaricus bisporus) stored in different packaging films. Innovative Food Science and Emerging Technologies, 11, 2010, 423–431.

Applications in Vegetables

229

19. E. Gaston, J. M. Frías, P. J. Cullen, C. P. O’Donnel, A. A. Gowen. Visible-near infrared hyperspectral imaging for the identification and discrimination of brown blotch disease on mushroom (Agaricus bisporus) caps. Journal of Near Infrared Spectroscopy, 18, 2010, 341–353. 20. L. Bagnasco, M. Zotti, N. Sitta, P. Oliveri. A PCA-based hyperspectral approach to detect infections by mycophilic fungi on dried porcini mushrooms (boletus edulis and allied species). Talanta, 144, 2015, 1225–1230. 21. A. Dacal-Nieto, A. Formella, P. Carrión, E. Vazquez-Fernandez, M. FernándezDelgado. Non-destructive detection of hollow heart in potatoes using hyperspectral imaging. In: Real P., Diaz-Pernil D., Molina-Abril H., Berciano A., Kropatsch W. (eds) Computer Analysis of Images and Patterns. CAIP 2011. Lecture Notes in Computer Science, Springer: Berlin, Heidelberg, vol. 6855, 2017. 22. N. Nguyen Do Trong, M. Tsuta, B. M. Nicolaï, J. De Baerdemaeker, W. Saeys. Prediction of optimal cooking time for boiled potatoes by hyperspectral imaging. Journal of Food Engineering, 105, 2011, 617–624. 23. W. Jiang, J. Fang, S. Wang, Y. Fan. Detection of starch content in potato based on hyperspectral imaging technique. International Journal of Signal Processing, Image Processing and Pattern Recognition, 8 (12), 2015, 49–58. 24. A. M. Rady, D. E. Guyer. Evaluation of sugar content in potatoes using NIR reflectance and wavelength selection techniques. Postharvest Biology and Technology, 103, 2015, 17–26. 25. A. Rady, D. Guyer, R. Lu. Evaluation of sugar content of potatoes using. Hyperspectral Imaging Food Bioprocess Technol, 5 (8), 2015, 995–1010. 26. A. López-Maestresalas, J. C. Keresztes, M. Goodarzi, S. Arazuri, C. Jarén, W. Saeys. Non-destructive detection of blackspot in potatoes by Vis-NIR and SWIR hyperspectral imaging. Food Control, 70, 2016, 299–241. 27. Y. Sun, Y. Liu, H. Yu, A. Xie, X. Li, Y. Yin, X. Duan. Non-destructive prediction of moisture content and freezable water content of purple-fleshed sweet potato slices during drying process using hyperspectral imaging technique. Food Analytical Methods, 10 (5), 2017, 1535–1546. 28. A. Kjær, G. Nielsen, S. Stærke, M. Rahr Clausen, M. Edelenbos, B. Jørgensen. Detection of glycoalkaloids and chlorophyll in potatoes (Solanum tuberosum L.) by hyperspectral imaging. American Journal of Potato Research, 94 (6), 2017, 573–582. 29. G. Polder, G. W. A. M. van der Heijden, H. van der Voet, I. T. Young. Measuring surface distribution of carotenes and chlorophyll in ripening tomatoes using imaging spectrometry. Postharvest Biology and Technology, 34, 2004, 117–129. 30. J. Qin, K. Chao, M. S. Kim. Investigation of Raman chemical imaging for detection of lycopene changes in tomatoes during postharvest ripening. Journal of Food Engineering, 107, 2011, 277–288. 31. J. Xing, M. Ngadi, N. Wang, J. De Baerdemaeker. Wavelength Selection for Surface Defects Detection on Tomatoes by Means of a Hyperspectral Imaging System. 2006 ASABE Annual International Meeting. 32. H. Lee, M. S. Kim, D. Jeong, S. R. Delwiche, K. Chao, B.-K. Cho. Detection of cracks on tomatoes using a hyperspectral near-infrared reflectance imaging system. Sensors, 14 (10), 2014, 18837–18850. 33. J. Danhee, S. K. Moon, L. Hoonsoo, L. Hoyoung Lee, B.-K. Cho. Detection algorithm for cracks on the surface of tomatoes using multispectral Vis/NIR reflectance imagery. Journal of Biosystems Engineering, 38(3), 2013, 199–207.

230

Leo M.L. Nollet, et al.

34. Y. Chun-Chieh, S. K. Moon, P. Millner, K. Chao, C. Byoung-Kwan, C. Moc, L. Hoyoung, D. E. Chan. Development of multispectral imaging algorithm for detection of frass on mature red tomatoes. Postharvest Biology and Technology, 93, 2014, 1–8. 35. Y. Xiaojun, Z. Honghui, Z. Qing, L. Jun, Z. Qingzhan, W. Chuanjian, N. Chuan. The Disease Severity Estimation of Bacterial Spot Disease of Processing Tomato Based on Hyperspectral remote sensing. 23rd International Conference on Geoinformatics. 2015. 36. J. van Roy, J. C. Keresztes, N. Wouters, B. De Ketelaere, W. Saeys. Measuring colour of vine tomatoes using hyperspectral imaging. Postharvest Biology and Technology, 129, 2017, 79–89. 37. A. Rahman, L. M. Kandpal, S. Lohumi, M. S. Kim, H. Lee, C. Mo, B.-K. Cho. Nondestructive estimation of moisture content, pH and soluble solid contents in intact tomatoes using hyperspectral imaging. Applied Science, 7, 2017, 109. 38. X. Cheng, Y. R. Chen, Y. Tao, C. Y. Wang, M. S. Kim, A. M. Lefcourt. A novel integrated PCA and FLD method on hyperspectral image feature extraction for cucumber chilling damage inspection. Transactions of the ASAE, 47 (4), 2004, 1313–1320. 39. Y. Liu, Y.-R. Chen, C. Y. Wang, D. E. Chan, M. S. Kim. Development of hyperspectral imaging technique for the detection of chilling injury in cucumbers. Proceedings Volume 5587, Nondestructive Sensing for Food Safety, Quality, and Natural Resources, (2004). 40. H. Cen R. Lu Q. Zhu, F. Mendoza. Nondestructive detection of chilling injury in cucumber fruit using hyperspectral imaging with feature selection and supervised classification. Postharvest Biology and Technology, 111, 2016, 352–361. 41. D. P. Ariana, R. Lu, D. E. Guyer. Near-infrared hyperspectral reflectance imaging for detection of bruises on pickling cucumbers. Computers and Electronics in Agriculture, 53 (1), 2006, 60–70. 42. Y. Liu, Y.-R. Chen, C. Y. Wang, D. E. Chan, M. S. Kim, C. Y. Wang. Development of a simple algorithm for the detection of chilling injury in cucumbers from visible/ near-infrared hyperspectral imaging. Applied Spectroscopy, 59 (1), 2005, 78–85. 43. H. Cen, R. Lu, D. P. Ariana, F. Mendoza. Hyperspectral imaging-based classification and wavebands selection for internal defect detection of pickling cucumbers. Food Bioprocess Technology, 7, 2014, 1689–1700. 44. R. Lu, D. P. Ariana. Development of a Hyperspectral Imaging System for Online Quality Inspection of Pickling Cucumbers. Food Processing Automation Conference CD-ROM. Proceedings of the 28–29 June 2008 Conference (Providence, Rhode Island) Publication date 28 June 2008. ASABE Publication Number 701P0508cd. 45. D. P. Ariana, R. Lu. Wavebands selection for a hyperspectral reflectance and transmittance imaging system for quality evaluation of pickling cucumbers. 2009 ASABE Annual International Meeting. 46. Y. Tian, L. Zhang. Study on the methods of detecting cucumber downy mildew using hyperspectral imaging technology. Physics Procedia, 33, 2012, 743–750. 47. R. Lu, D. P. Ariana. Detection of fruit fly infestation in pickling cucumbers using a hyperspectral reflectance/transmittance imaging system. Postharvest Biology and Technology, 81, 2013, 44–50. 48. D. P. Ariana, R. Lu, D. E. Guyer. Near-infrared hyperspectral reflectance imaging for detection of bruises on pickling cucumbers. Computers and Electronics in Agriculture, 53 (1), 2006, 60–70. 49. S. Kang, K. Lee, J. Son, M. S. Kim. Detection of fecal contamination on leafy greens by hyperspectral imaging. Procedia—Food Science, 1, 2011, 953–959.

Applications in Vegetables

231

50. J. Sun, X. Jin, H. Mao, X. Wu, H. Gao, W. Zhu, X. Liao. Detecting nitrogen content in lettuce leaves on hyperspectral imaging and multiple regression analysis. Information Technology Journal, 12 (19), 2013, 4845–4851. 51. J. Sun, X. Zhou, X. Wu, X. Zhang, Q. Li, W. Su, X. Lu. Identification of nutrient elements in lettuce leaves using hyperspectral imaging and classification modeling analysis. doi:10.5013/IJSSST.a.17.20.10. 52. I. Simko, J. A. Jimenez-Berni, R. T. Furbank. Detection of decay in fresh-cut lettuce using hyperspectral imaging and chlorophyll fluorescence imaging. Postharvest Biology and Technology, 106, 2015, 44–52. 53. C. Mo, G. Kim, J. Lim, M. S. Kim, H. Cho, B.-K. Cho. Detection of lettuce discoloration using hyperspectral reflectance imaging. Sensors, 15, 2015, 29511–29534. 54. C. Mo, G. Kim, M. S. Kim, J. Lim, S. H. Lee, H. Lee, B.-K. Cho. Discrimination methods for biological contaminants in fresh-cut lettuce based on VNIR and NIR hyperspectral imaging. Infrared Physics & Technology, 85, 2017, 1–12. 55. C. Mo, G. Kim, M. S. Kim, J. Lim, K. Lee, W.-H. Lee, B.-K. Cho. On-line fresh-cut lettuce quality measurement system using hyperspectral. Biosystems engineering, 156, 2017, 38–50. 56. C. Zhang, Q. Wang, F. Liu, Y. He, Y. Xiao. Rapid and non-destructive measurement of spinach pigments content during storage using hyperspectral imaging with chemometrics. Measurement, 97, 2016, 149–155. 57. C. D. Everard, M. S. Kim, H. Cho, C. P. O’Donnell. Hyperspectral fluorescence imaging using violet LEDs as excitation sources for fecal matter contaminate identification on spinach leaves. Food Measure, 10, 2016, 56–63. 58. C. D. Everard, M. S. Kim, H. Lee. A comparison of hyperspectral reflectance and fluorescence imaging techniques for detection of contaminants on spinach leaves. Journal of Food Engineering, 143, 2014, 139–145. 59. U. Siripatrawana, Y. Makinob, Y. Kawagoeb, S. Oshita. Rapid detection of Escherichia coli contamination in packaged fresh spinach using hyperspectral imaging. Talanta, 85, 2011, 276–281. 60. M. A. Lara, L. Lleó, B. Diezma-Iglesias, J. M. Roger, M. Ruiz-Altisent. Monitoring spinach shelf-life with hyperspectral image through packaging films. Journal of Food Engineering, 119, 2013, 353–361. 61. B. Diezma, L. Lleó, J. M. Roger, A. Herrero-Langreo, L. Lunadei, M. RuizAltisenta. Examination of the quality of spinach leaves using hyperspectral imaging. Postharvest Biology and Technology, 85, 2013, 8–17. 62. Z. Schmilovitch, T. Ignat, V. Alchanatis, J. Gatker, V. Ostrovsky, J. Felföldi. Hyper­ spectral imaging of intact bell peppers. Biosystems engineering, 117, 2014, 83–93. 63. H. Wang, C. Li, M. Wang, Onion quality assessment using diffuse reflectance hyperspectral images with a shape correction algorithm. 2011 ASABE Annual International Meeting. 64. H. Wang, C. Li, M. Wang. Onion Quality Assessment Using a Near Infrared Hyperspectral Imaging System. 2012 ASABE Annual International Meeting. 65. W. Wang, C. Li, C. Thai, R. Gitaitis, E. W. Tollner. Sour Skin Detection in Vidalia Onions Using Hyperspectral Imaging. Ext-Res Report 2008–2009 Final. 66. J. M. Hernández-Hierro, C. Esquerre, J. Valverde, S. Villacreces, K. Reilly, M.  Gaffney, M. L. González-Miret, F. J. Heredia, C. P. O’Donnell, G. Downey. Preliminary study on the use of near infrared hyperspectral imaging for quantitation and localisation of total glucosinolates in freeze-dried broccoli. Journal of Food Engineering, 126, 2014, 107–112.

Chapter

16

Applications in Medicinal Herbs and Pharmaceuticals Leo M.L. Nollet University College Ghent

Hong-Ju He and Hui Wang Henan Institute of Science and Technology (HIST)

CONTENTS 16.1 Medicinal Herbs and Teas 233 16.2 Pharmaceuticals 237 References 237

16.1  MEDICINAL HERBS AND TEAS Herbal medicine is a plant or plant part. Medicinal herbs are in use for thousands of years and are renowned for their effectiveness in many diseases. Hyperspectral imaging (HSI) as a rapid, no-destructive technique has been used for quality control in medicinal herbs. In this chapter, based on the HSI technique, medicinal qualities of herbs are estimated. Sceletium tortuosum is the most sought after species of the genus Sceletium and is commonly included in commercial products for the treatment of psychiatric conditions and neurodegenerative diseases. However, this species exhibits several morphological and phytochemical similarities to Sceletium crassicaule. The aim of the investigation of E. A. Shikanga et al. [1] was to use ultrahigh-performance liquid chromatography (UHPLC) and hyperspectral imaging, in combination with chemometrics, to distinguish between S. tortuosum and S. crassicaule and to accurately predict the identity of specimens of both species. Chromatographic profiles of S. tortuosum and S. crassicaule specimens were obtained using UPLC with photodiode array detection. A sisuChema near infrared hyperspectral imaging camera was used for acquiring images of the specimens, and the data were processed using chemometric computations. Chromatographic data of the specimens revealed that both species produce the psychoactive alkaloids that are used as quality control biomarkers. Principal component

233

234

Leo M.L. Nollet et al.

analysis (PCA) of the hyperspectral image of reference specimens for the two species yielded two distinct clusters, the one representing S. tortuosum and the other representing S. crassicaule. A partial least squares discriminant analysis model correctly predicted the identity of an external dataset consisting of S. tortuosum or S. crassicaule samples with high accuracy (>94%). Illicium verum (Chinese star anise) dried fruit is popularly used as a remedy to treat infant colic [2]. However, instances of life-threatening adverse events in infants have been recorded after use, in some cases due to substitution and/or adulteration of I. verum with Illicium anisatum (Japanese star anise), which is toxic. Short wave infrared (SWIR) hyperspectral imaging and image analysis as a rapid quality control method have the potential to distinguish between I. anisatum and I. verum whole dried fruit. Images were acquired using a sisuChema SWIR hyperspectral pushbroom imaging system with a spectral range of 920–2,514 nm. PCA was applied to the images to reduce the high dimensionality of the data, to remove unwanted background, and to visualize the data. A classification model with four principal components and partial least squares discriminant analysis (PLS-DA) was developed. The model was subsequently used to accurately predict the identity of I. anisatum (98.42%) and I. verum (97.85%) introduced into the model as an external dataset. Echinacea species are popularly included in various formulations to treat upper respiratory tract infections [3]. Due to their close taxonomic alliance, it is difficult to distinguish between the three Echinacea species and incidences of incorrectly labeled commercial products have been reported. Hyperspectral images of root and leaf material of authentic Echinacea species (E. angustifolia, E. pallida, and E. purpurea) were acquired using a sisuChema shortwave infrared (SWIR) hyperspectral pushbroom imaging system with a spectral range of 920–2,514 nm. PCA plots showed a clear distinction between the root and leaf samples of the three Echinacea species and further differentiated the roots of different species. A classification model with a high coefficient of determination was constructed to predict the identity of the species included in commercial products. The majority of products (12 of 20) were convincingly predicted as containing E. purpurea or E. angustifolia, or both. On the other hand, the use of ultra-performance liquid chromatography-mass spectrometry (UPLC-MS) in the differentiation of the species presented a challenge due to chemical similarities between the solvent extracts. Harpagophytum procumbens (Burch.) DC. ex Meisn. subsp. Procumbens is an important African medicinal plant growing in the Kalahari region of southern Africa. This species, together with its close taxonomically Harpagophytum zeyheri are collectively referred to as Devil’s Claw and are used interchangeably for the treatment of inflammation-related disorders [4]. Although the two taxa are botanically and chemically similar, H. zeyheri contains lower levels of harpagoside and these two species have not been proven to exhibit equipotent pharmacological activity. Differentiation between the two species was achieved using single point mid-infrared spectroscopy in combination with chemometric data analysis. The orthogonal projections to latent structures discriminant analysis (OPLS-DA) model had good predictive ability, as illustrated by the model statistics: R 2 X (cum predictive + orthogonal) = 0.86 and Q 2 (cum) = 0.63. Short wave infrared (SWIR) hyperspectral imaging could distinguish between the two species with acceptable model statistics: R 2 X and R 2Y of 0.99 and 0.78, respectively. This study demonstrated that both MIR single point spectroscopy and SWIR hyperspectral imaging coupled with chemometric modeling are reliable and rapid methods to determine the authenticity of Harpagophytum spp.

Applications in Medicinal Herbs and Pharmaceuticals

235

Stephania tetrandra (“hang fang ji”) and Aristolochia fangchi (“guang fang ji”) are two different plant species used in Traditional Chinese Medicine. Both are commonly referred to as “fang ji,” and S. tetrandra is mistakenly substituted and adulterated with the nephrotoxic [5] A. fangchi as they have several morphological similarities. A. fangchi contains aristolochic acid, a carcinogen that causes urothelial carcinoma as well as aristolochic acid nephropathy. In Belgium, 128 cases of AAN were reported, while in China, a further 116 cases with end-stage renal disease were noted. Hyperspectral imaging in combination with partial least squares discriminant analysis (PLS-DA) is suggested as an effective method to distinguish between S. tetrandra and A. fangchi root powder. Hyperspectral images were obtained in the wavelength region of 920–2,514 nm. Reduction of the dimensionality of the data was done by selecting the discrimination information range (964–1,774 nm). A discrimination model with a coefficient of determination (R 2) of 0.9 and a root mean square error of prediction (RMSEP) of 0.23 was created. The constructed model successfully identified A. fangchi and S. tetrandra samples inserted into the model as an external validation set. In addition, adulteration detection was investigated by preparing incremental adulteration mixtures of S. tetrandra with A. fangchi (10–90%). Hyperspectral imaging showed the ability to accurately predict adulteration as low as 10%. Scutellaria lateriflora (skullcap) is a medicinal herb that has a long history of use in the treatment of ailments such as insomnia and anxiety [6]. Commercial herbal formulations claiming to contain S. laterifolia herba have flooded the consumer markets. However, due to intentional or unintentional adulteration, cases of hepatotoxicity have been reported. Possible adulteration with the potentially hepatotoxic Teucrium spp., T. canadense and T. chamaedrys, has been reported. In this study, hyperspectral imaging in combination with multivariate image analysis methods was used to differentiate the raw materials of S. laterifolia, T. canadense, and T. chamaedrys in a nondestructive manner. Furthermore, the ability to detect adulteration of raw materials using the developed multivariate models was also investigated. Chemical images were captured using a shortwave infrared pushbroom imaging system in the wavelength range of 920–2,514 nm. PCA was applied to the images to investigate chemical differences between the species. Partial least squares discriminant analysis was used to model pre-assigned class images, and the classification model predicted the levels of adulteration in spiked raw materials. UHPLC-MS as an independent analytical technique was used to confirm chemical differences between the three species. The ability of hyperspectral chemical imaging as a nondestructive technique in the differentiation of the three species was achieved with three distinct clusters in the score scatter plot. A 92.3% variation in modeled data using PC1 and PC2 was correlated to chemical differences between the three species. Near infrared signals in the regions 1,924 nm and 2,092 nm (positive P1), 1,993 nm and 2,186 nm (negative P1), 1,918 nm, 2,092 nm, and 2,266 nm (positive P2), as well as 1,993 nm and 2,303 nm (negative P2) were identified as containing discriminating information using the loadings line plots. Chemical imaging of spiked samples showed spatial orientation of contaminants within the powdered samples, and percentage adulteration was accurately predicted at levels ≥ 40% adulteration based on pixel abundance. The potential of hyperspectral imaging as a rapid quality control method for herbal tea blends from rooibos (Aspalathus linearis), honeybush (Cyclopia intermedia), buchu (Agathosma betulina), and cancer bush (Sutherlandia frutescens) was investigated [7]. Hyperspectral images of raw materials and intact tea bags were acquired using a sisuChema shortwave infrared (SWIR) hyperspectral pushbroom imaging system (920–2,514 nm). PCA plots showed clear discrimination between raw materials. Partial least squares

236

TABLE 16.1  Applications of HSI Technique in Various Herbs Herbal Medicine

External Quality

Mode

Spectral Range

Sceletium tortuosum and Sceletium crassicaule Japanese star anise

Quality control

Diffuse reflectance

920–2,514 nm

Neurotoxic

920–2,514 nm

Optimal Wavelengths

PLS-DA, PCA, MSC PCA, PLS-DA, MSC

Accuracy

Reference

>94%

1

Accurately is 98.42% for I. anisatum and 97.85% for I. verum

2

920–2,514 nm 920–2,514 nm

PCA OPLS-DA

Q2 = 0.779 R2X = 0.99 R2Y = 0.78

3 4

Quality control

920–2,514 nm

OPLS-DA

R2 = 0.9

5

Adulteration

920–2,514 nm

PCA

920–2,514 nm

PCA, PLS-DA

Echinacea Harpagophytum procumbens and H. zeyheri Stephania tetrandra and Aristolochia fangchi Scutellaria lateriflora

Quality control Quality control

Herbal tea blends

Quality control

Reflectance

Reflectance

6 Q = 92.7% 2

7

Leo M.L. Nollet et al.

1,235–1,342, 1,762–1,881, 2,049–2,173 nm for I. anisatum 1,392–1,537, 1,662–1,681, 1,899–2,024, 2,248–2,272 nm for I. verum

Data Analysis

Applications in Medicinal Herbs and Pharmaceuticals

237

discriminant analysis (PLS-DA) models correctly predicted the raw material constituents of each blend and accurately determined the relative proportions. The results were corroborated independently using UHPLC-MS (Table 16.1).

16.2 PHARMACEUTICALS Only a few examples of hyperspectral analysis of pharmaceuticals are given in the next paragraphs. The sensitivity and spatial resolution of hyperspectral imaging instruments were tested using pharmaceutical applications by Hamilton and Lodder [8]. The first experiment tested the hypothesis that a near-IR tunable diode-based remote sensing system is capable of monitoring degradation of hard gelatin capsules at a relatively long distance (0.5 km). Spectra from the capsules were used to differentiate among capsules exposed to an atmosphere containing 150-ppb formaldehyde for 0, 2, 4, and 8 hours. The second experiment tested the hypothesis that near-infrared (IR) imaging spectrometry of tablets permits the identification and composition of multiple individual tablets to be determined simultaneously. A near-IR camera was used to collect thousands of spectra simultaneously from a field of blister-packaged tablets. The number of tablets that a typical near-IR camera can currently analyze simultaneously was estimated to be approximately 1,300. The bootstrap error-adjusted single-sample technique chemometric-imaging algorithm was used to draw probability-density contour plots that revealed tablet composition. The single capsule analysis provides an indication of how far apart the sample and instrumentation can be and still maintain adequate S/N, while the multiple-sample imaging experiment gives an indication of how many samples can be analyzed simultaneously while maintaining an adequate S/N and pixel coverage on each sample. Near infrared hyperspectral images (HSI-NIR) of tablets with different expiration dates were employed to evaluate the degradation of captopril into captopril disulfide in different layers, on the top and on the bottom surfaces of the tablets [9]. Multivariate curve resolution (MCR) models were used to extract the concentration distribution maps from the hyperspectral images. Afterward, multivariate image techniques were applied to the concentration distribution maps (CDMs), to extract features and build models relating the main characteristics of the images to their corresponding manufacturing dates. Resolution methods followed by extracting features were able to estimate the tablet manufacture date with a prediction error of 120 days. The model developed could be useful to evaluate whether a sample shows a degradation pattern consistent with the date of manufacturing or to detect abnormal behaviors in the natural degradation process of the sample (Table 16.1).

REFERENCES 1. E. A. Shikanga, A. M. Viljoen, I. Vermaak, S. Combrinck. A novel approach in herbal quality control using hyperspectral imaging: Discriminating between Sceletium tortuosum and Sceletium crassicaule. Phytochemical Analysis 24(6), 2013, 550–555. 2. I. Vermaak, A. Viljoen, S. W. Lindström. Hyperspectral imaging in the quality control of herbal medicines—The case of neurotoxic Japanese star anise. Journal of Pharmaceutical and Biomedical Analysis 75, 2013, 207–213.

238

Leo M.L. Nollet et al.

3. M. Sandasi, I. Vermaak, W. Chen, A. M. Viljoen. Hyperspectral imaging and chemometric modeling of Echinacea—A novel approach in the quality control of herbal medicines. Molecules 19, 2014, 13104–13121. 4. N. Mncwangi, I. Vermaak, A. M. Viljoen. Mid-infrared spectroscopy and short wave infrared hyperspectral imaging—A novel approach in the qualitative assessment of Harpagophytum procumbens and H. zeyheri (Devil’s Claw). Phytochemistry Letters 7, 2014, 143–149. 5. S. Tankeu, I. Vermaak, W. Chen, M. Sandasi, A. M. Viljoen. Differentiation between two “fang ji” herbal medicines, Stephania tetrandra and the nephrotoxic Aristolochia fangchi, using hyperspectral imaging. Phytochemistry 122, 2016, 213–222. 6. M. Sandasi, I. Vermaak, W. Chen, A. M. Viljoen. Skullcap and germander: preventing potential toxicity through the application of hyperspectral imaging and multivariate image analysis as a novel quality control method. Planta Medica 80(15), 2014, 1329–1339. 7. M. Djokam, M. Sandasi, W. Chen, A. Viljoen, I. Vermaak. Hyperspectral imaging as a rapid quality control method for herbal tea blends. Applied Sciences 7, 2017, 268–284. 8. S. J. Hamilton, R. A. Lodder. Hyperspectral imaging technology for pharmaceutical analysis. SPIE Proc. BiOS 2002. 9. L. de Moura França, M. F. Pimentel, S. da Silva Simões, S. Grangeiro Jr., J. M. Prats-Montalbán e. Alberto Ferrer NIR hyperspectral imaging to evaluate degradation in captopril commercial tablets. European Journal of Pharmaceutics and Biopharmaceutics 104, 2016, 180–188.

Chapter

17

Hyperspectral Imaging in Dairy Products Analysis Basil K. Munjanja University of Pretoria

CONTENTS 17.1 Introduction 239 17.2 Cheese Analysis 240 17.3 Authentication of Dairy Products 240 17.4 Conclusions and Future Trends 241 References 242

17.1 INTRODUCTION Hyperspectral imaging (HSI) is an emerging analytical technique that combines imaging and spectroscopy to obtain both the spatial and spectral information of an object (Gowen et al., 2007). By acquiring the spectral information of the object, HSI can determine both the internal and external attributes of food (Liu, Pu, and Sun, 2017). An added advantage of the technique is that it can be used for analysis of heterogeneous samples because the spectra are collected at each pixel (Manley, 2014). However, the disadvantages of the technique include high cost when wavelengths up to 2,500 nm are required (Manley, 2014). In order to fully understand how this works, it is necessary to discuss the ­fundamentals of the technique. The hyperspectral imaging system consists of a light source to interact with the food sample, a light dispersion device, an imaging unit, and the computer hardware and software to make the decision (Feng and Sun, 2012). Hyperspectral images are generated in three main ways namely: point to point, spectral scanning, and line by line spatial scanning (ElMasry and Nakauchi, 2016). It is also possible to acquire images in reflectance, transmittance, and interactance modes, giving a three dimensional form that is referred to as a hypercube (ElMasry and Nakauchi, 2016). To analyze the images, the steps followed include reflectance calibration, pre-processing, classification, and image ­processing, respectively (Gowen et al., 2007). Over the past years, numerous technological advances have been made in the instrumentation. The majority of publications have been made on other food products such as meat (Lee, Kim, Lee, et al., 2018), fruits, and vegetables (Teena et al., 2014; Liu, Zeng, and Sun, 2015), and cereal products (Erkinbaev, Henderson, and Paliwal, 2017). However, very few publications have been reported on dairy products because of the

239

240

Basil K. Munjanja

homogeneity of liquid and powdered milk samples, and the difficulty to analyze cheese due to the heat generated by the light source and the longer acquisition time (ElMasry et al., 2012). Nevertheless, hyperspectral imaging has high applicability in monitoring the uniformity of the major ingredients of dairy products because of its ability to create distribution maps of quality parameters during processing. This chapter reviews the applications of HSI in dairy products analysis. Emphasis will be on cheese analysis and authentication of dairy products.

17.2  CHEESE ANALYSIS A study was carried out to classify 13 cheese samples containing various amounts of protein, fat, and carbohydrates using hyperspectral NIR Imaging system in the range of 960–1,662 nm. A good prediction model of the ingredients was observed by trimming of bad pixel outliers, and the technique gave detailed spatial resolution (Burger and Geladi, 2006a). In another study, the potential of multispectral imagery coupled to chemometrics was explored in the classification of varieties of blue cheese (Kulmyrzaev, Bertrand, and Dufour, 2008). The image acquisition was done in the ultraviolet, visible, and near-infrared regions. The spectral functions of the image texture based on Fourier spectrum were extracted from the raw multivariate images using an image processing tool and decomposition of the covariance matrices was done. Principal component analysis (PCA) and partial least squares discriminant analysis (PLSDA) of the spectrum functions showed good discrimination of blue cheese. Similar results were obtained in another study in which multispectral imaging coupled to PLSDA and PLS-R were used to discriminate four French cheese brands (Jacquot et al., 2015). A distinct advantage of the technique was that it could distinguish blue cheese even though they looked similar. In a related study, HSI and PLS regression models were used to predict the concentrations of different constituents in 13cheese samples. Very low bias was obtained for both protein (−0.6%) and fat (−0.2%). A distinct advantage of the technique was that it gave better characterization of organic and biological materials in the samples (Burger and Geladi, 2006b). Cheese ripening has been studied using various techniques such as fluorescence spectroscopy (Karoui, Dufour, and Baerdemaeker, 2007), near and mid infrared spectroscopy (Woodcock et al., 2008), and magnetic resonance imaging (MRI) (Huc et al., 2014). However, HSI is a promising technique in this area, as it has also been used to monitor the effect of transglutaminase in semi-hard cheese during ripening. HSI was used to differentiate according to fat level and ripening status. A linear correlation was observed between the hardness and the HSI reflectance at the significant wavelength of enzyme treatment. Furthermore, the results obtained with HSI were comparable to those obtained using texture profile analysis (Darnay et al., 2017).

17.3  AUTHENTICATION OF DAIRY PRODUCTS Near infrared (NIR) hyperspectral imaging coupled to partial least squares regression (PLSR) was used to detect melamine in milk powders in the spectral range of ­990–1,700 nm up to very low concentrations of 0.02% (Lim et al., 2016). In a different but related study, the same group demonstrated the potential of NIR hyperspectral imaging and spectral similarity analyses for discrimination of melamine adulteration in

HSI in Dairy Products Analysis

241

milk powders. The spectrum of each pixel in the sample images was compared to the pure melamine spectrum by spectral angle measure (SAM), spectral correlation measure (SCM), and Euclidian distance measure (EDM), with comparable results (Fu et al., 2014). The same technique was used to semiquantify melamine in powdered milk using the CLS imaging approach, and processing the extracted spectra with SNV. It was observed that the detection limit was less than that obtained using LC-MS, and ELISA detection (Huang et al., 2014). Recently, NIR hyperspectral imaging and simple band rationing have been explored as a tool for quantifying melamine in milk powders and non-fat milk (Huang et al., 2016). The band ratio images were essential for establishing a single threshold for deciding whether a pixel is rich or deficient in melamine. In addition, the resultant binary images can be used to visualize the spatial distribution of melamine adulterant regions in the investigated milk powders. NIR imaging methods have also been successfully used in distribution assessment and quantification of counterfeit melamine in powdered milk using data abstraction methods such as PCA, classical least squares regression (CLS), and alternative least squares (ALS) (Huang et al., 2015). It was observed that CLS could quantify the melamine at low concentration ranges of 0–0.5% w/w. In addition, its accuracy was lower than that of ALS. Another chemometric method that has been successfully utilized in the detection and quantification of melamine is multivariate curve resolution (MCR) (Forchetti and Poppi, 2017). The advantage of this technique was that no priori information about the presence of adulterants was necessary. Recently, the applicability of hyperspectral imaging has also been extended to predict the starch content in adulterated cheese, with high levels of predictability (Barreto et al., 2018). Raman chemical imaging is another technique that has also been used to detect adulteration in milk powder in the wave number range of 103–2,881 cm−1 (Qin et al., 2017). Image classifications were conducted using a thresholding method applied to single band fluorescence corrected images at unique Raman peaks selected for melamine 673 cm−1 and urea 1,009 cm−1. Limits of detection for both adulterants were estimated in the order of 50 ppm. The group also went on to apply the technique to detect four adulterants in dry milk in the concentration range of 0.1–5.0% for each adulterant. The spectra from the pure components were extracted using self-modeling mixture analysis (SMA) for all the adulterants. Raman chemical images were then created using contributions from SMA, and thus reflecting their real concentrations (Qin, Chao, and Kim, 2013). Recently, an advancement that has been made in Raman chemical imaging is the use of a mercury cadmium telluride (MCT)-based shortwave infrared (SWIR) hyperspectral imaging system coupled to an algorithm to quantitatively detect melamine in milk powder over the region of 100–2,500 nm. Using analysis of variance (ANOVA) and partial least squares regression (PLS-R), the models for detection were built, and it was found that melamine could be detected at levels as low as 50ppm (Lee, Kim, Lohumi, et al., 2018).

17.4  CONCLUSIONS AND FUTURE TRENDS While HSI can be celebrated because of its unique properties, it is a pity that very few publications have been reported on its application in dairy products. To-date, most of the publications reported with this technique are on authentication of dairy products, and a few more on cheese analysis. Recently, a trend that has been observed is the application of Raman chemical imaging in dairy products analysis. It is therefore hoped that

242

Basil K. Munjanja

with significant advances being made in the instrumentation, more potential applications of the technique will include milk homogenization, blending, and classification of dairy products among others. Nonetheless, if the unique capabilities of this technique are exploited, it is hoped that significant breakthroughs will be realized in this field.

REFERENCES Barreto, A, JP Cruz-Tirado, R Siche, and R Quevedo. 2018. “Determination of Starch Content in Adulterated Fresh Cheese Using Hyperspectral Imaging.” Food Bioscience 21: 14–19.doi:10.1016/j.fbio. 2017.10.009. Burger, J, and P Geladi.2006a. “Hyperspectral NIR Image Regression Part II: Dataset Preprocessing Diagnostics.” Journal of Chemometrics 20: 106–19. Burger, J, and P Geladi. 2006b. “Hyperspectral NIR Imaging for Calibration and Prediction: A Comparison between Image and Spectrometer Data for Studying Organic and Biological Samples.”Analyst 131: 1152–60. Darnay, L, F Kralik, G Oros, A Koncz, and F Firtha. 2017. “Monitoring the Effect of Transglutaminase in Semi-Hard Cheese during Ripening by Hyperspectral Imaging.” Journal of Food Engineering 196: 123–29. ElMasry, GM, M Kamruzzaman, D Sun, and P Allen. 2012. “Principles and Applications of Hyperspectral Imaging in Quality Evaluation of Agro-Food Products: A Review.” Critical Reviews in Food Science and Nutrition 52: 999–1023. ElMasry, GM, and S Nakauchi. 2016. “Image Analysis Operations Applied to Hyperspec­ tral Images for Non-Invasive Sensing of Food Quality A Comprehensive Review.” Biosystems Engineering 142: 53–82.doi:10.1016/j.biosystemseng.2015.11.009. Erkinbaev, C, K Henderson, and J Paliwal. 2017. “Discrimination of Gluten-Free Oats from Contaminants Using near Infrared Hyperspectral Imaging Technique.” Food Control 80: 197–203.doi:10.1016/J.FOODCONT.2017.04.036. Feng, Y, and D Sun. 2012. “Application of Hyperspectral Imaging in Food Safety Inspection and Control: A Review.” Critical Reviews in Food Science and Nutritionutrition 52: 1039–58. doi:10.1080/10408398.2011.651542. Forchetti, DAP, and RJ Poppi. 2017. “Use of NIR Hyperspectral Imaging and Multivariate Curve Resolution (MCR) for Detection and Quantification of Adulterants in Milk Powder.” LWT - Food Science and Technology 76: 337–43.doi:10.1016/j. lwt.2016.06.046. Fu, X, MS Kim, K Chao, J Qin, J Lim, H Lee, A Garrido-Varo, D Pérez-Marín, and Y Ying. 2014. “Detection of Melamine in Milk Powders Based on NIR Hyperspectral Imaging and Spectral Similarity Analyses.” Journal of Food Engineering 124: 97–104.doi:10.1016/j.jfoodeng.2013.09.023. Gowen, AA, CP O’Donnell, PJ Cullen, G Downey, and JM Frias. 2007. “Hyperspectral Imaging an Emerging Process Analytical Tool for Food Quality and Safety Control.” Trends in Food Science & Technology 18: 590–98. Huang, M, MS Kim, SR Delwiche, K Chao, J Qin, C Mo, C Esquerre, and Q Zhu. 2016. “Quantitative Analysis of Melamine in Milk Powders Using Near Infrared Hyperspectral Imaging and Band Ratio.” Journal of Food Engineering 181: 10–19. doi:10.1016/j.jfoodeng.2016.02.017. Huang, Y, S Min, J Duan, L Wu, and Q Li. 2014. “Identification of Additive Components in Powdered Milk by NIR Imaging Methods.” Food Chemistry 145: 278–83. doi:10.1016/j.foodchem.2013.06.116.

HSI in Dairy Products Analysis

243

Huang, Y, K Tian, S Min, Y Xiong, and G Du. 2015. “Distribution Assessment and Quantification of Counterfeit Melamine in Powdered Milk by NIR Imaging Methods.” Food Chemistry 177: 174–81. doi:10.1016/j.foodchem.2015.01.029. Huc, D, N Roland, D Grenier, S Challois, C Michon, and F Mariette. 2014. “Influence of Salt Content on Eye Growth in Semi-Hard Cheeses Studied Using Magnetic Resonance Imaging and CO2 Production Measurements.” International Dairy Journal 35: 157–65. doi:10.1016/J.IDAIRYJ.2013.11.010. Jacquot, S, R Karoui, K Abbas, A Lebecque, C Bord, and A Ait-Kaddour. 2015. “Potential of Multispectral Imager to Characterize Anisotropic French PDO Cheeses: A Feasibility Study.” International Journal of Food Properties 18: 213–30. doi:10.10 80/10942912.2013.828746. Karoui,R, É Dufour, and J Baerdemaeker. 2007. “Monitoring the Molecular Changes by Front Face Fluorescence Spectroscopy throughout Ripening of a Semi-Hard Cheese.” Food Chemistry 104: 409–20. doi:10.1016/j.foodchem.2006.09.020. Kulmyrzaev, A, D Bertrand, and E Dufour. 2008. “Characterization of Different Blue Cheeses Using a Custom-Design Multispectral Imager.” Dairy Science and Technology 88:537–48. Lee, H, MS Kim, W-H Lee, and B-K Cho. 2018. “Determination of the Total Volatile Basic Nitrogen (TVB-N) Content in Pork Meat Using Hyperspectral Fluorescence Imaging.” Sensors and Actuators B: Chemical 259: 532–39. doi:10.1016/J.SNB.2017.12.102. Lee, H, MS Kim, S Lohumi, and B Cho. 2018. “Detection of Melamine in Milk Powder Using MCT based Shortwave Infrared Hyperspectral Imaging System.” Food Additives and Contaminants: Part A 35: 1027–1037. doi:10.1080/19440049.2018.1469050. Lim, J, G Kim, C Mo, MS Kim, K Chao, J Qin, X Fu, I Baek, and B Cho. 2016. “Detection of Melamine in Milk Powders Using near-Infrared Hyperspectral Imaging Combined with Regression Coefficient of Partial Least Square Regression Model.”Talanta 151: 183–91. doi:10.1016/j.talanta.2016.01.035. Liu, D, X Zeng, and D Sun. 2015. “Recent Developments and Applications of Hyperspectral Imaging for Quality Evaluation of Agricultural Products: A Review.” Critical Reviews in Food Science and Nutrition 55: 1744–57. doi:10.1080/10408398.2013.777020. Liu, Y, H Pu, and D Sun. 2017. “Hyperspectral Imaging Technique for Evaluating Food Quality and Safety during Various Processes: A Review of Recent Applications.” Trends in Food Science & Technology 69: 25–35. doi:10.1016/j.tifs.2017.08.013. Manley, M. 2014. “Near-Infrared Spectroscopy and Hyperspectral Imaging: NonDestructive Analysis of Biological Materials.” Chemical Society Reviews 43: 8200–8214. Qin, J, K Chao, and MS Kim. 2013. “Simultaneous Detection of Multiple Adulterants in Dry Milk Using Macro-Scale Raman Chemical Imaging.” Food Chemistry 138: 998–1007. doi:10.1016/j.foodchem.2012.10.115. Qin, J, MS Kim, K Chao, S Dhakal, H Lee, B Cho, and C Mo. 2017. “Detection and Quantification of Adulterants in Milk Powder Using a High-Throughput Raman Chemical Imaging Technique.” Food Additives and Contaminants: Part A 34: ­152–61. doi:10.1080/19440049.2016.1263880. Teena, MA, A Manickavasagan, L Ravikanth, and DS Jayas. 2014. “Near Infrared (NIR) Hyperspectral Imaging to Classify Fungal Infected Date Fruits.” Journal of Stored Products Research 59: 306–13. doi:10.1016/J.JSPR.2014.09.005. Woodcock, T, CC Fagan, CP O’Donnell, and G Downey. 2008. “Application of Near and Mid-Infrared Spectroscopy to Determine Cheese Quality and Authenticity.” Food and Bioprocess Technology 1:117–29.

Chapter

18

Hyperspectral Imaging Application in Quality and Safety of Beverages N.C. Basantia Cavinkare Pvt. Ltd

CONTENTS 18.1 Introduction 245 18.2 Process for Hyper Spectral Imaging Analysis of Food 246 18.3 Application of HSI in Safety and Quality Beverages 246 18.3.1 A lcoholic Beverages 246 18.3.1.1 B eer 247 18.3.1.2 Wine 248 18.3.1.3 W hisky 249 18.3.2 Nonalcoholic Beverages 249 18.3.2.1 Coffee 249 18.3.2.2 Tea 249 18.3.2.3 Fruit-Based Beverages 251 18.4 Conclusion 256 References 256

18.1 INTRODUCTION As a global issue, food safety is receiving increasing attention in both developed and developing countries. Food safety problems are frequently confronting in our daily life, and a number of food safety incidents are taking place. Any occurrence of food safety problem always brings substantial influences to the society. Therefore, various regulatory bodies worldwide are taking strict measures to implement effective practices and policies for the surveillance of the food industry and to develop consumer’s confidence and trust for the food industry. The food quality and safety evaluation has become more important and needs a more comprehensive assessment. The quality and food safety can be evaluated in many different methods and techniques such as traditional sensory evaluation and chemical and microbiological analysis. The sensory evaluation is the oldest method and still used every day and relatively fast, but the reliability varies with food item group and by person, and the results are subjective. The chemical analysis is a technique that often provides reliable results but it has some limitations in terms of need of complex multipurpose processing, expertise, cost, need of chemicals, time-consuming, destructive

245

246

N.C. Basantia

type, and not suitable for online product management. The microbiological methods are although objective and reliable but are slow and more tedious and expensive. Therefore, the need for a fast and nondestructive method was realized a couple of decades ago. As an emerging technology, hyperspectral imaging (HSI) has been successfully employed in food safety inspection and control. HSI technology of food and agricultural products has been evolving rapidly in the last 15 years owing to tremendous interest from both academic and industrial field. Because of inherent merit of this technique, it has put into application in number of fields such as agricultural, pharmaceutical, and material science. Application of HSI in food safety and quality includes detection of contaminants, identification of defects, and quantification of constituents. Recently, the technology has become more and more popular in food quality control to meet the consumer demands and challenge of market segment and legal restriction. The most prominent commercial application of HSI remains in the areas of agricultural raw material, food ingredients, and finished food products. This includes a wide range of commodities, i.e., meat, fruits and vegetables, dairy products, cereals, beverages, and tea. Beverage is an important category of food which has paramount importance in the diet and may have high prices and large commercialization. This category of food includes liquid foods such as alcoholic beverages (red wines, brandy, whisky, and beer) or nonalcoholic beverages (juices, fruit vinegars, coffee, and tea). Increasing demand of these liquid foods has made them vulnerable to economic adulteration during processing and chain supply. Adulteration of alcoholic drink is often common due to products’ high prices and large commercialization. Determination of authenticity of the beverages, identification of origin of beverages, and the quality of raw materials such as grapes used for wine, barley and malt for beer, coffee beans used for roasted coffee, and fruits used for fruit beverages is an important aspect of quality control. The majority of reported research in food science and technology concerns quality control of vegetables, fruit, grain, meat, and poultry products. There have been, to date, very few reported applications of HSI in Beverages. However, a wide range of quality and safety testing practices in beverage industry could be complemented and potentially improved with HSI. In this chapter, the potential applications of HSI are quality monitoring of beverages specifically in selection of raw materials, process monitoring, ­compositional analysis, prediction of functional properties, and authentication.

18.2  PROCESS FOR HYPER SPECTRAL IMAGING ANALYSIS OF FOOD The flowchart given in Figure 18.1 briefly describe the steps involved in processing of HSI.

18.3  APPLICATION OF HSI IN SAFETY AND QUALITY BEVERAGES 18.3.1 Alcoholic Beverages The alcoholic beverages are named and classified based on their source and alcoholic content: i. Beer ii. Wines iii. Whiskey

HSI in Quality and Safety of Beverage

247

Sample

Acquision of hyperspectral image

Image calibraon

Spectral data extracon and preprocessing PCA, PLS, DA, PLSDA

Spectral Data Analysis (Chemometrics) Dimensionality reducon and wavelength selecon

Image post processing and paern recognion Final result

Classificaon, idenficaon and/or visualisaon

FIGURE 18.1  Flow chart for hyperspectral imaging processing.

18.3.1.1 Beer Beer is made by a process of fermentation. A liquid mix called wort is prepared by combining yeast and malted cereals such as corn, rye, wheat, or barley. The quality or taste of beer depends on the type or source of wort used. Beer is an alcoholic beverage of intense consumption and can be found almost everywhere. It has been produced and consumed in many countries especially in Europe and Asia. Commercially, the most common beer in Brazil is pale lager, having an alcohol content of 4% by volume. It has a light-yellow color and low content of fermentable carbohydrates. This type beer is from 100% barley malt. Among many attributes of beer, one easily characterized is its color. Until today, the classification of beer by color is done during production process through a European wide standard known as European Brewing Convention (EBC). According to EBC, this pale lager beer must contain less than 20 EBC units. The amber color of beer is due to pigments known as melanoidin and caramel added to the malt. Therefore, the type and quality of raw material plays an important role to get a product with typical taste and color as per consumer requirement. In order to select raw materials, there is a need to select a suitable method which is rapid, accurate, objective, and cost-effective. HSI has been found suitable and applied for the classification of grains including maize, wheat, barley, and oat. A NIR HSI system (900–1,700nm) was developed in a mathematical

248

N.C. Basantia

modeling framework to identify pregerminated barley at an early stage in order to segregate the barley kernels into low or high quality (Arngren et al., 2011). A low classification error of 3% proved feasibility of the developed system for describing the degree of pre-germination of single barley kernel. Williams et al. (2009) described the application of InGaAs and HgCdTe detector based NIR HSI (960–1,162nm) techniques in distinguishing between hard, intermediate, and soft maize kernels. ANIR HSI system (1,006–1,650) was developed to classify oat and groat kernels (Serranti et al., 2013). The discussed methods are summarized in Table18.1. Another HSI system (960–1,700 nm) was developed to differentiate wheat classes grown in western Canada. Different classification models are used in the system. Some research work has been conducted for the detection of damaged and contaminated grains (Singh et al, 2009). Williams et al. (2012) developed a NIR HSI system (1,000–2,498 nm) to track changes in whole maize kernel contaminated with fungi. PLS regression model was established to assess the changes over time. The results indicated the possibility of early detection of fungal contamination and activity. An NIR HSI system in the wavelength range of 1,000–1,600 nm was developed for the detection of insect-damaged wheat kernels (Singh et al., 2009, 2010). Applications such as genotype classification, mycotoxin detection, quantitative analysis of moisture, protein, and β-glucan are important for the production of good quality beer (Manley et al., 2011). 18.3.1.2 Wine Wines are mostly made from grapes and much less from peaches, plums, and apricots. The soil in which the grapes are grown and the weather conditions in the growing season determine the quality and taste of grapes which in turn affect the taste and quality of wine. Hence, there is a need to select a suitable type and variety of fruits to get the suitable quality and taste of wine. To select a suitable type and variety of fruit, a reliable, fast, and non-destructive type of method is needed. In the wine sector, it is really important to know the critical parameters and attributes of grapes, and it is necessary to get the information quickly and accurately. Among other grape varieties, maturity and sugar content are typically analyzed in order to determine the grape quality, to set a price for the grape, and to classify grapes for a range of commercially produced wines. HSI has been applied to characterize grape skin in the past. However, several studies of HSI have appeared with the aim of predicting different parameters in grapes. There are some features that make grape skin ideal for hyperspectral analysis. It is not necessary to destroy the berry. Whole grapes, which contain some important secondary metabolites such as anthocyanin or flavonoids, are used in order to develop hyperspectral analysis for grape skin. The thinness of the grape skin allows the HSI also to measure the pulp metabolites, and grape skin shows essential physiological changes in grapes (i.e., veraison and ripeness). HSI has been applied to grape skin with the goal of predicting the technological maturity of grapes and the total phenolic compounds, total anthocyanins, and extractable polyphenols of grape skin. Chen et al. (2015) demonstrated the capability of HSI in predicting anthocyanin content changes in wine grapes during ripening. They recorded hyperspectral images in the spectral range of 900–1,700 nm. The anthocyanin content was measured by pH differential method. A quantitative model was developed using partial least square regression (PLSR) or support vector regression (SVR) for calculating the anthocyanin content. Among these, the best model was obtained using SVR, which yielded coefficient of validation (P-R 2) of 0.9414 and root mean square errors of prediction (RMSEP) of 0.0046 higher than the PLSR model.

HSI in Quality and Safety of Beverage

249

18.3.1.3 Whisky Whisky is made by distilling fermented juice of cereal grains such as corn, barley, and rye. Well-known Scotch whiskies were originally made in Scotland, and the word Scotch has become almost synonymous with the whisky of good quality. Scotch is a high-quality spirit drink exclusively produced in Scotland in a manner specified by the law. One distinct feature of certain scotch whiskies is their smoky characteristic. It is produced by exposing malted barley during the kilning process to the smoke of burning peat phenolic compounds, where the smoke adheres to the surface of grain and are carried through the process into the spirit. The phenol levels are used as a marker to the degree of smokiness in whisky. Phenol levels are varied depending on desired smokiness. It is important to control the phenol level to avoid blending of external spirits to obtain the required aroma. Determination of phenol levels directly in the malted barley before processing has become beneficial to the whisky industry. The techniques used to determine these levels of phenols, such as High Performance Liquid Chromatography (HPLC), are timeconsuming and require distillation of malt prior to analysis. Tschannerl et al. (2017) explored near infrared–short wave infrared (NIR-SWIR) HSI to detect phenolic compounds in the smoke directly on the malted barley. Some spectral differences between peated and unpeated barley can be detected with NIR-SWIR-HSI. However, it was not clear if these differences were from chemical absorption of phenol, physical scattering, or from color differences. There is more work needed for phenol detection via HSI.

18.3.2 Nonalcoholic Beverages 18.3.2.1 Coffee Coffee is one of the most popular beverages in the world (Duarte et al., 2005). Due to national and regional variation in growing conditions and cultivation of Arabica and Robusta coffee varieties, there is a considerable diversity in potential sources of green coffee beans. HSI has been used to discriminate sample of green beans of Arabica and Robusta with over 97% of classification accuracy. Casal et al. (2000) characterized Arabica and Robusta roasted coffees from several geographical origins for their contents in caffeine, trigonelline, and nicotinic acid by HPLC method. However, the HPLC method is time-consuming and needs sample preparation. The two coffee varieties were clearly separated by their triginelline and caffeine content. HSI has been applied to discriminate the different classes of coffee samples (Calvini et al., 2015). Four different Chinese domestic coffee beans were identified rapidly by near infrared HSI covering the spectral range of 874–1,734 nm (Yi-dan et al., 2015). 18.3.2.2 Tea Tea is the most consumed beverage worldwide except for water. In 2012, the total consumption of tea was shown to comprise of 85% black tea and 15% green tea with Oolong and white tea making up the remainder. Tea is produced from the young shoots of the plant Camelia sinesis (L.) (Willson and Clifford, 1992). Depending on the techniques used in the production and processing, tea can be divided into four basic types, namely, green tea, black tea, Oolong tea, and white tea (Liu and Yue, 2005). In recent years, consumers have become more health conscious, both in terms of prevention and treatment of disease which has led to an escalation in the use and development of plantbased health products including herbal tea. Herbal tea is a non-caffeine-containing hot

250

N.C. Basantia

beverage prepared through infusion of any plant other than leaves of Camellia sinensis. The quality of tea is mainly reflected in its smell and taste, which are generated by volatile and nonvolatile organic compounds present in tea (Chen et al., 2008; Dutta et al., 2003; Liang et al., 2003).The quality of tea can be influenced at two different stages during the growing period of the tea plant and at the tea processing stage (Obanda et al., 1997). After careful picking and selecting the fresh tea leaves, these are dried, rolled, curled, and twisted for the production of green tea or withered, curled, fermented, and dried for production of black tea. The biochemical properties of fresh tea leaves in growing stage are more difficult to control (Wright et al., 2000). The concentration of foliar chemical compounds has an important impact on tea infusions flavor, smell, and other factors that make up the tea quality (Chen et al., 2006; Teranishi et al., 1999; Yamamoto et al., 1997). Therefore, traditionally tea quality was judged through sensory evaluation and the leaf color, size, and shape were also graded. Together with the flavor score, they quantify the overall quality (Stone and Sidel, 2004; Togari et al., 1995). Fresh tea leaves contain caffeine, tea polyphenols, tea polysaccharides, and nutrients such as proteins, amino acids, and vitamins. Generally, four chemical components, free amino acids, total tea polyphenols, soluble sugars, and caffeine are considered as indicators of tea quality. Significant relationship has been identified between certain foliar chemical variables and tea sensing preferences (Liang et al., 2008; Obanda et al., 1997). Of hundreds of chemical compounds found in tea, amino acids, and polyphenols are generally considered to be the two main factors determining its quality (Thomas et al., 2009; Yamamoto et al., 1997). The contents of amino acids are a major factor in determining the freshness and mellowness of green tea. An unique amino acid in tea, named theanine, produces relaxation effects (Chu et al., 1999). Tea polyphenols, 70% of which is catechin, influence the smell and astringent taste of tea and polyphenols account for 20–35% in weight of dry matter of tea. In addition to total tea polyphenols, free amino acids, soluble sugars, and caffeine are also generally regarded as the major indicator of tea quality. Using hyperspectral data from the tea canopy level, an artificial neural network in combination with a successive projections algorithm was applied to estimate foliar biochemical concentration of tea (Kokaly et al., 2009). The successive projections algorithm was applied to select the optimal wavelength bands from canopy level hyperspectral data. Thus, integrated approach has led to satisfactory prediction accuracy. The best trained neural network resulted in a coefficient of determination (r 2) between predicted and observed concentration of 0.82, 0.76, and 0.76 for tea polyphenols, free amino acids, and soluble sugars, respectively. Zhouet al. (2017) described the application of HSI technology coupled with four kinds of classification methods to detect and grade five green teas, such as Biluochun, Jing Shan, Long Jing, Que She, and San Bei Xiang. Besides the grading, quantitative analysis has also been used on the spectra to predict two lipid soluble pigments that is β-carotene and lutein. The results showed that support vector machine (SVM) obtained relatively better results on separating three grade of each green tea with classification accuracy higher than 93%. This study introduces a simple method for green tea classification and grading and accomplishes also the quantitative determination of two pigments such as β-carotene and lutein. This methodology is potential for online detection and grading of green tea processing. Ning et al. (2017) identified five tea categories by visible and near infrared HSI combined with classification pattern recognition. Visible and NIR HSI combined with LibSVM has the capability of classifying tea categories rapidly and nondestructively.

HSI in Quality and Safety of Beverage

251

18.3.2.3 Fruit-Based Beverages Fruit juices or fruit-based beverages have paramount importance in human diet. Consumption of fruit juices and fruit-based beverages is rising continuously due to consumer preference for healthy eating habits. Nowadays, the consumer expectation is much more demanding than the past. The consumer demands quality products in terms of taste, flavor, and free from preservatives, pathogenic organisms, and chemical contaminants. Therefore, there is a need for the manufacturer to adopt such a quality control process, which could control starting from reception of raw material, processing, transportation, storage, and till consumption of the product. The fresh fruit or fruit pulp is the major raw material for processing of fruit juice and fruit-based beverages. The important attributes of fruit which affect the taste are soluble solid concentration (SSC), sugar content, acidity, and physical attributes which reflect the quality firmness or maturity of fruit, defects as fruit bruising which affect the quality fecal contamination, microbiological contamination, and chemical contamination. Next to sensory evaluation, chemical, microbiological, and chromatographic and spectroscopic methods are available. These have limitations in terms of being subjective, time-consuming, requiring chemicals, expertise and being destructive. HSI could provide abundant information related to fruit quality. Many ­studies have been conducted in this field in the last 2 decades as depicted in Table 18.1. One quality attribute of fruit that has been assessed using HSI is a soluble solid content. Peng and Lu (2008) designed a reflectance system to detect apple firmness and total SSC using stable object stage and optical fiber. Focusing lenses were used to illuminate samples as a spotlight source and 2-D hyperspectral images were collected. The light source used in this study was delivered by a circular beam of 1.5 mm, which scanned the fruit on 1.6 mm of the incident center. Ten MLD functions were proposed to fit the spectral scattering profile, and the best one was chosen for predicting fruit firmness and SSC using MLR. Mendoza et al. (2011) employed integrated scattering image in the range of ­480–1,016 nm to predict the firmness and SSC in apples. The results indicated 6% of standard error in prediction of firmness and 3% of SEP for SSC. Large latent variables in the prediction model indicated the necessary of a more robust prediction model for firmness and SSC of apple. Leiva-Valenzuela et al. (2013) used VIS/NIR HSI (500–1,000 nm) to determine the soluble solid content in blue berries reaching a prediction accuracy of 0.87 and 0.79 for the firmness and SSC, respectively. El Masry et al. (2007) used HSI in the VIS/NIR region (400–1,000 nm) to predict the soluble solid content in strawberry. Another quality attribute fruit evaluated using HSI is mealiness. Huang and Lu (2010) examined the relationship between reflectance hyperspectral line images and apple mealiness. The spectral scattering profile at individual wavelength of apples undergoing different time images are obtained and correlated to different mealiness levels. The mealiness of apples was determined by the hardness and juiciness. Its correlation with hyperspectral scattering profile was predicted using PLS. The best classification accuracy was obtained in the classification of mealy and nonmealy with an accuracy of 75%. This study demonstrated that hyperspectral scattering technique was potentially useful for nondestructive detection of apple mealiness and that further research was required to improve classification accuracy especially the discrimination of less severe mealy apple. The presence of defects in fruits usually indicates possible invasion and dwelling of pathogenic microbes in the flawed portion of the commodities appearing as symptoms such as bruises or lesions (Kim et al., 2001, Mehl et al., 2001). These defects can undergo a fast deterioration process, thus pose a great threat to human beings especially when they are

Sl. No

Type of Beverage Product

252

TABLE 18.1  Applications of Hyperspectral Imaging in Beverage Quality and Safety Material

Quality Attribute

Mode

Spectral Range

Modeling

Reference Arngren et al. (2011)

Williams et al. (2009)

Alcoholic beverages

Barley

Classification as low or high

Scattering

900–1,700 nm

2

Maize

Hard or soft kernel

Scattering

960–1,662 nm

Oat and groat Kernels Wheat

Classification

Scattering

1,006–1,650 nm

PLS-DA

Serranti et al. (2013)

Classification

Reflectance

960–1,700 nm

LDA,QDA,ANN

Mahesh et al. (2008)

Wheat

Insect damaged

Reflectance

700–1,100 nm

LDA, QDA, MDC

Singh et al. (2010)

Grape

900–1,700 nm

PLSR, SVR

Chen et al. (2015)

Coffee beans

—

874–1,734 nm

PLS-DA, KNN, SVM, ELM

Yi-dan et al. (2015)

8

Tea

Green tea

Anthocyanin content Identification and classification Grading

Reflectance

7

Alcoholic beverages Alcoholic beverages Alcoholic beverages Alcoholic beverages Alcoholic beverages Coffee

Maximum likely hood multinomial regression classifier PLS-DA

9

Tea

—

589,635,670, and783 nm

LDA, Lib SVM, ELM

Ning et al. (2017)

10

Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages

Tea (green, white, Classification yellow, black and Oolong) Apple SSC or sugar content Apple SSC or sugar content Apple Firmness

Reflectance

480–1,016 nm

PLS

Shan et al. (2011)

Reflectance

500–1,000 nm

PLS

Reflectance

500–1,000 nm

PLS

Apple

Reflectance

400–1,000 nm

GLCM, SVM

Mendoza et al. (2011) Mendoza et al. (2011) Guo et al. (2013)

3 4 5 6

11 12 13

Discrimination

—

(Continued)

N.C. Basantia

1

TABLE 18.1  (Continued) Applications of Hyperspectral Imaging in Beverage Quality and Safety Sl. No 14 15 16

18 19 20 21 22 23 24 25 26

Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit-based beverages Fruit base beverages Fruit base beverages

Material

Quality Attribute

Mode

Spectral Range

Modeling

Reference

Apple

Bruises

—

400–2,500 nm

LDA, SVM, SIMCA

Oranges

Qualitative

400–1,100 nm

PCA, BR, TH

Baranwoski et al. (2012) Li et al. (2011)

Kiwi fruits

SSC/sugar content

Reflectance

650–1,100 nm

PLC

Apple

Mealiness

Reflectance

900–1,700 nm

PLSDA

Apple

Fluorescence

500–1,000 nm

—

Apple

Maturity & quality evaluation Defect & feces

Strawberry

Martinsen and Schaare (1998) Huang and Lu (2010) Noh et al. (2007)

Feces

Reflectance & 400–1,000 nm fluorescence Fluorescence

Band ratio and Kim et al. (2007) thresh holding PCA Band ratio and PCA Vargas et al. (2004)

Citrus

Rottenness

Reflectance

460–1,020 nm

ANN, CART

Citrus

Canker

Reflectance

450–930 nm

SID

Gomez-Sanchis et al. (2008) Qin et al. (2009)

Peach

Firmness

Scattering

500–1,000 nm

—

Lu and Peng (2006)

Cherry

Pit detection

Transmittance 450–1,000 nm

—

Qin and Lu (2005)

Orange

Pesticide residue methomyl Pesticide residue dichlorvos

Reflectance

PCA

Xue et al. (2008)

PLS

Li et al. (2010)

Orange

Reflectance

625–725nm

253

(Continued)

HSI in Quality and Safety of Beverage

17

Type of Beverage Product

254

TABLE 18.1  (Continued) Applications of Hyperspectral Imaging in Beverage Quality and Safety Sl. No 27

Type of Beverage Product

Quality Attribute

Mode

Spectral Range

Modeling

Reference

Scotch whisky

Prediction of smokiness Aflatoxin

Reflectance

900–1,700 nm

SNV

Reflectance and fluorescence Reflectance

400–878 nm

Tschannerl et al. (2017) Zhu et al. (2016)

920–2,514 nm

PCA, PLS-DA

Djokam et al. (2017)

408–1,008 nm

LDA

Nansen et al. (2016)

28

Alcoholic beverages Alcoholic

Corn Kernel

29

Tea

Herbal tea bag

30

Coffee

Ground coffee

Identification of type of raw material and their proportion in the blend Consistency in terms of roasting classes

Reflectance mode

Note: PLS-DA, partial least square discriminant analysis; LDA, linear Discriminant Analysis; ANN, artificial Neutral Network; PLSR, partial least square regression; KNN, K-nearest neighbor; SVM, support vector machine; PLS, partial least square regression; SIMCA, soft independent modeling of class analogy; PCA, principal component analysis; BR, band ratio; TH, threshold; QDA, quadratic discriminant analysis; SVR, support vector regression; MDC, mahalanobis discriminant classifier; ELM, extreme learning machine; GLCM, gray level co-occurrence matrix; CART, classification and regression trees; SID, spectral information divergence; Lib-SVM, library support vector machine.

N.C. Basantia

Material

HSI in Quality and Safety of Beverage

255

mistakenly used for producing cider and beverages without pasteurization or sterilization. Lu (2003) employed a HSI system in the 900–1,700 nm range to study its potential for predicting bruises during by integrating principal component analysis (PCA) and minimizing noise fraction (MNF) transform. The authors found that for red delicious apple, the highest classification accuracy was gained for 1 day after bruises occurrence. The result trends suggested that the variation among cultivars affect the detection greatly. Xing et al., (2007) compared the PLSDA and PCA model for classification. They found PLSDA had more potential as compared to PCA, since the prediction map produced was free from the effects of sample surface curvatures. El Masry et al. (2008) found that HSI could sense bruises as early as 1h after bruising. Xing and DeBaerdemaeker (2005) developed a successful scheme to discriminate one bruise portion from another. In terms of bruise inspection, some other fruits such as strawberry (Nagata et al., 2006) was studied. In these studies, PCA models were frequently used to extract features or to reduce data size. However, band difference was frequently suggested for future application because of its simplicity and equivalent performance to PCA. In apples, bitter pit lesion (Nicolaï et al., 2006) and open wounds rot and brown decayed spots on apples were determined (Lu et al., 2003). Ariana et al. (2006) built an ANN model to categorize apples into either two groups (normal and defect) or multi classes (normal, bitter pit, black rot, decay soft scald, and superficial scald tissues). Fecal contamination on fruits such as apples may result in the introduction of pathogenic microorganisms (e.g., Escherchia coli, O157:H7) into unpasteurized apple juice or cider which may be a potential health threat to public (Cody et al., 1999).Therefore, they should be refused for further processing. HSI offers a solution through nondestructive measurement of faeces on apples. However, during application one should consider the challenges such as both appearance and component variances among different cultivars and individuals of same cultivars due to different environment and growth stage. These variations are of great concern because they may affect the successful recognition of feces on apples. To eliminate such influence, different wavelength ranges and models were tried. Kim et al. (2001) showed that the image in the far red band (specifically 800 nm) depicted a little or no difference in sun exposed and shaded surface of Red Delicious apples. However in this approach, the classification accuracy or sensitivity could not be guaranteed since only thick feces could be identified easily in the image. Thin layer appears problematic. Other options may include employment of chemometric algorithms and multivariate data analysis. PCA, application of uniform power transformation, and second difference can help the performance (Lefcourt et al., 2005a, b; Lefcourt and Kim, 2006; Mehl et al., 2004). Liu et al. (2007) obtained the best classification by using a two band ratio (R725/R811) where the quotient of difference using three bands yielded less contrast than two band ratio methods. A recently developed simple algorithm based on peak shifts and shoulder curves was employed when a multispectral fluorescence imaging system was investigated in the red and far-red region (Yang et al., 2011,2012).In addition to the above mentioned methods on apples, HSI was used to detect faeces in strawberries (Vargas et al., 2004). Qin et al (2009) examined the relationship between hyperspectral area images and citrus canker an electron multiplying charge coupled device (EMCCD) which is a type of CCD with high photosensitivity. Later, the hyperspectral images were processed and classified to differentiate canker lesion from normal and other peeled disease conditions including greasy spot, insect damage, melanose scab, and wind scar. The analysis method was a SID-based classification and the overall classification accuracy was 96%. Pesticide residues in food are one of the problems which leads to chronic disease such as cardiovascular disease, diabetes, and cancer (Bolognesi and Marasso, 2000). Presently, the methods available are chromatographic based such as GC-MS or LC-MS-MS which

256

N.C. Basantia

are time-consuming, needing chemicals, and sophisticated equipment, expertise, and being not cost-effective. Therefore, application of HSI was tried by various researchers. Xue et al. (2008) studied the feasibility detecting methomyl on navel oranges by HSI. PCA used to extract features or reduce data size. After applying PCA to the images acquired 4 principal components contributed 99.18% of the total information. They found that the third PC gave the best discrimination result. Using the same system and different lighting system on the same product, quantitative analysis of concentration of dichlorvos was studied through spectroscopic and chemometric analysis (Li et al., 2010).

18.4 CONCLUSION HSI is an emerging platform for food quality and safety analysis in processing of alcoholic and nonalcoholic beverages in terms of selection and classification of raw materials based on their quality and safety parameters, to monitor the quality during processing, to predict the taste and quality of finished product based on properties of raw material and authenticity of finished product. In addition to the prominent advantages of HSI being a precise, rapid, chemical free, non-invasive, and non-destructive method, it could provide internal spectral information of samples while detecting spatial signals which are related to physical and chemical features in a large amount of raw materials and beverages. That information will offer exciting new possibilities for further research. It has flexibility in choosing any region of interest (ROI) in the image even after image acquisition. When an object or ROI in the object presents very obvious spectral characteristic that region could be selected and its spectrum could be saved in a spectral library. Although HSI with chemometrics frees researchers from laborious measurements and burdensome computation during food quality assessment, HSI has not been much used for online detection which is restricted by its massive data volume, different prediction results of spectra mode and expensive equipment. From the analyst point of view, one of the main analytical drawbacks of the HSI technique is that it is an indirect method and needs standardization, calibration, and model transfer procedures. Although there is a load of existing challenges, the emergence of new technologies and new devices is bringing a huge potential to this field. The improved acquisition speed and simplified operation of newly developed spectrographs, an HSI system combined with implementation of effective chemometric methods such as PLS and LS-SVM, and small scale HSI systems could make the idea of online detection possible. As now there is a paradigm shift towards onsite testing, cloud based applications, data transfer through smartphone HSI could have play an important role. In future, a smartphone will be developed with image acquisition power Powerful algorithms and a cloud-based wide library for onsite rapid prescreening of quality and safety of raw material and finished product of beverages, and wireless data transfer to relevant stakeholders will be a reality.

REFERENCES Ariana, D., Guyer, D. E., and Shrestha, B. (2006). Integrating multispectral reflectance and fluorescence imaging for defect detection on apples. Computers and Electronics in Agriculture, 50: 148–161. Arngren, M., Hansen, P. W., Eriksen, B., Larsen, J., and Larsen, R. (2011). Analysis of pregerminated barley using hyperspectral image analysis. Journal of Agricultural and Food Chemistry, 59: 11385–11394.

HSI in Quality and Safety of Beverage

257

Baranwoski, P., Mazurek, W., Wozniak, J., and Majewska, U. (2012). Detection of early bruises in apples using hyperspectral data and thermal imaging. Journal of Food Engineering, 110: 345–355. Bolognesi, C., and Morasso, G. (2000). Genotoxicity of pesticides: Potential risk for consumers. Trends in Food Science & Technology, 11: 182–187. Calvini, R., Ulrici, A., and Amigo, J. M. (2015). Practical comparison of sparse methods for classification of Arabica and Robusta coffee species using near infrared hyperspectral imaging. Chemometrics and Intelligent Laboratory Systems, 146: 503–511. Casal, S., Oliveira, M., Alves, M., and Ferreira, M. (2000). Discriminate analysis of roasted coffee varieties for trigonelline, nicotinic acid, and caffeine content. Journal of Agricultural and Food Chemistry, 48(8): 3420–3424. Chen, Q., Zhao, J., Huang, X., Zhang, H., and Liu, M. (2006). Simultaneous determination of total polyphenols and caffeine contents of green tea by near-infrared reflectance spectroscopy. Microchemical Journal, 83(1): 42–47. Chen, Q., Zhao, J., and Vittayapadung, S. (2008). Identification of the green tea grade level using electronic tongue and pattern recognition. Food Research International, 41(5): 500–504. Chen, S., Zhang, F., Ning, J., Liu, X., Zhang, Z., and Yang, S. (2015). Predicting the anthocyanin content of wine grapes by NIR hyperspectral imaging. Food Chemistry, 172(1): 788–793. Chu, D. C., Okubo, T., Nagato, Y., and Yokogoshi, H. (1999). L-theanine—A unique amino acid of green tea and its relaxation effect in humans. Trends in Food Science and Technology, 10(6–7): 199–204. Cody, S. H., Glynn, M. K., Farrar, J. A., Cairns, K. L., Griffin, P. M., Kobayashi, J., Fyfe, M., Hoffman, R., King, A. S., and Lewis, J. H. (1999). An outbreak of Escherichia coli O157:H7 infection from unpasteurized commercial apple juice. Annals of Internal Medicine, 130: 202–209. Djokam, M., Sandasi, M., Chen, W., Viljoen, A., and Vermaak, I. (2017). Hyperspectral imaging as a rapid quality control method for herbal tea blends. Applied Sciences, 268(7): 1–16. Duarte, S. M. d. S., Abreu, C. M. P. d., Menezes, H. C. d., Santos, M. H. d., Gouvêa, C. M. C. P. (2005). Effect of processing and roasting on the antioxidant activity of coffee brews. Food Science and Technology (Campinas), 25: 387–393. Dutta, R., Kashwan, K. R., Bhuyan, M., Hines, E. L., and Gardner, J. W. (2003). Electronic nose based tea quality standardization. Neural Networks, 16(5): 847–853. El Masry, G., Wang, N., Elsayed, A., Ngadi, M. (2007). Hyperspectral imaging for nondestructive determination of some quality attributes for strawberry. Journal of Food Engineering, 81: 98–107. El Masry, G., Wang, N., Vigneault, C., Qiao, J., and ElSayed, A. (2008). Early detection of apple bruises on different background colors using hyperspectral imaging. LWTFood Science and Technology, 41: 337–345. Guo, Z. M., Huang, W. Q., Chen, L. P., Zhao, C. J., Peng, Y. K. (2013). Geographical classification of apple based on hyperspectral imaging. In proceedings of the sensing for agriculture and food quality and safety V, Baltimore, MD, USA 30 April–1 May 2013 SPIE-INT Soc. Optical Engineering: Baltimore MD, USA, 2013: p. 8. Gomez-Sanchis, J., Molto, E., Camps-Valls, G., Gomez-Chova, L., Aleixos, N., and Blasco, J. (2008). Automatic correction of effects of light source on spherical objects. An application to the analysis of hyperspectra images of citrus fruits. Journal of Food Engineering, 85: 191–200.

258

N.C. Basantia

Huang, H., and Lu, L. (2010). Apple mealiness detection using hyperspectral scattering technique. Postharvest Biology and Technology, 58: 168–175. Kim, M., Chao, K., Chen, Y., Chan, D., and MEHL, P. (2001). Hyperspectral imaging system for food safety: Detection of fecal contamination on apples. In: Proceedings of SPIE. 4206: 174–184. Chen, Y. R., and Tu, S.-I. (Eds.), Society of Photo-Optical Instrumentation Engineers, Boston, MA. Kim, M., Chen, Y., Cho, B., Chao, K., Yang, C., Lefcourt, A., and Chan, D. (2007). Hyperspectral reflectance and fluorescence line scan imaging for online defect and fecal contamination inspection of apples. Sensing and Instrumentation for Food Quality and Safety, 1: 151–159. Kokaly, R. F., Asner, G. P., Ollinger, S. V., Martin, M. E., and Wessman, C. A. (2009). Characterizing canopy biochemistry from imaging spectroscopy and its application to ecosystem studies. Lefcout, A. M., and Kim, M. S. (2006). Technique for normalizing intensity histograms of images when the approximate size of the target is known: Detection of feces on apples using fluorescence imaging. Computers and Electronics in Agriculture, 50: 135–147. Lefcourt, A. M., Kim, M. S., and Chen, Y.-R. (2005a). Detection of fecal contamination on apples with nanosecond-scale time resolved imaging of laser-induced fluorescence. Applied Optics, 44: 1160–1170. Lefcourt, A. M., Kim, M. S., and Chen, Y.-R. (2005b). A transportable fluorescence imagining system for detecting fecal contaminants. Computers and Electronics in Agriculture, 48: 63–74. Leiva-Valenzuela, G. A., Renfu, L., Aguilera, J. M. (2013). Prediction of firmness and soluble solids content of blueberries using hyperspectral reflectance imaging. Journal of Food Engineering, 115: 91–98. Li, J., Rao, X., Ying, Y. (2011). Detection of common defects on oranges using hyperspectral reflectance imaging. Computers and Electronics in Agriculture, 78, 38–48. Li, J., Xue, L., Liu, M., Wang, X., and Luo, C. (2010). Hyperspectral imaging technology for determination of dichlorvos residue on the surface of navel orange. Chinese Optics Letters, 8: 1050–1052. Liang, Y., Lu, J., Zhang, L., Wu, S., and Wu, Y. (2003). Estimation of black tea quality by analysis of chemical composition and colour difference of tea infusions. Food Chemistry, 80(2): 283–290. Liang, Y. R., Ye, Q., Jin, J., Liang, H., Lu, J. L., Du, Y. Y., and Dong, J. J. (2008). Chemical and instrumental assessment of green tea sensory preference. International Journal of Food Properties, 11(2): 258–272. Liu, T., and Yue, L. (2005). Chinese Tea, China Intercontinental Press, Beijing, 112 p. Liu, Y., Chen, Y.-R., Kim, M. S., Chan, D. E., and Lefcourt, A. M. (2007). Development of simple algorithms for the detection of fecal contaminants on apples from visible/near infrared hyperspectral reflectance imaging. Journal Food Engineering, 81: 412–418. Lu, R.(2003). Detection of bruises on apples using near-infrared hyperspectral imaging. Transactions of the ASABE, 46: 523–530. Lu, R. F., and Peng, Y. K. (2006). Assessing peach firmness by multi-spectral scattering. Journal of Near Infrared Spectroscopy, 13(1): 27–35. Mahesh, S., Manickavasagana, A., Jayasa, D. S., Paliwala, J., Whiteb, N. D. G. (2008). Feasibility of near infrared hyperspectral imaging to differentiate Canadian wheat classes. Biosystems Engineering, 101: 50–57.

HSI in Quality and Safety of Beverage

259

Manley, M., Du Toit, G., Geladi, P. (2011). Tracking diffusion of conditioning water in single wheat kernels of different hardness by near infrared hyperspectral imaging. Analytica Chimica Acta, 686: 64–75. Martinsen, P., and Schaare, P. (1998). Measuring soluble solids distribution in Kiwi fruit using near-infrared imaging spectroscopy. Postharvest Biology and Technology, 14: 271–281. Mehl, P., Chao, K., Kim, M., and Chen, Y. (2001). Detection of contamination on selected apple cultivars using reflectance hyperspectral and multispectral analysis. In: Photonic Detection and Intervention Technologies for Safe Food Proceedings of SPIE. 4206: 201–213. Chen, Y.-R., and Tu, S.-I. (Eds.), Boston, MA. Mehl, P. M., Chen, Y.-R., Kim, M. S., and Chan, D. E. (2004). Development of hyperspectral imaging technique for the detection of apple surface defects and contaminations. Journal of Food Engineering, 61: 67–81. Mendoza, F., Lu, R., Ariana, D., Cen, H., Bailey, B. (2011). Integrated spectral and image analysis of hyperspectral scattering data for prediction of apple fruit firmness and soluble solid content. Postharvest Biology and Technology, 62: 149–160. Nagata, M., Tallada, J. G., and Kobayashi, T. (2006). Bruise detection using NIR hyperspectral imaging for strawberry (Fragaria × ananassa Duch.). Environment Control in Biology, 44: 133–142. Nansen, C., Singh, K., Mian, A., and Allison, B. J. (2016). Using hyperspectral imaging to characterize consistency of coffee brands and their respective roasting classes. Journal of Food Engineering, 190: 1–16. Nicolaï, B. M., Lötze, E., Peirs, A., Scheerlinck, N., and Theron, K. I. (2006). Nondestructive measurement of bitter pit in apple fruit using NIR hyperspectral imaging. Postharvest Biology and Technology, 40: 1–6. Ning, J., Sun, J., Li, S., Sheng, M., and Zhang, Z. (2017). Classification of five Chinese tea categories with different fermentation degrees using visible and near infrared hyperspectral imaging. International Journal of Food Properties, 46(1): 1–8. Noh, H., Peng, Y., and Lu, R. (2007). Integration of hyperspectral reflectance and fluorescence imaging for assessing apple maturity. Transactions of the ASAE, 50: 963–971. Obanda, M., Owuor, P. O., and Taylor, S. J. (1997). Flavanol composition and caffeine content of green leaf as quality potential indicators of Kenyan black teas. Journal of the Science of Food and Agriculture, 74(2): 209–215. Peng, Y., and Lu, R. (2008). Analysis of spatially resolved hyperspectral scattering images for assessing apple fruit firmness and soluble solids content. Postharvest Biology and Technology, 48: 52–62. Qin, J., Burks, T. F., Ritenour, M. A., Bonn, W. G. (2009). Detection of citrus canker using hyperspectral reflectance imaging with spectral information divergence. Journal of Food Engineering, 93: 183–191. Qin, J., and Lu, R. (2005). Detection of pits in tart cherries by hyperspectral transmission imaging. Transactions of the ASAE, 48: 1963–1970. Serranti, S., Cesare, D., Marini, F., Bonifazi, G. (2013). Classification of oat and groat kernels using NIR hyperspectral imaging. Talanta, 103: 276–284. Shan, J., Peng, Y., Wang, W., Li, Y., Wu, J., and Zhang, L. (2011). Simultaneous detection of external and internal quality parameters of apples using hyperspectral technology. Trans. CSAM, 42: 140–144. Singh, C. B., Jayas, D. S., Paliwal, J., White, N. D. G. (2009). Detection of insect-­damaged wheat kernels using near-infrared hyperspectral imaging. Journal of Stored Products Research, 45: 151–158.

260

N.C. Basantia

Singh, C. B, Jayas, D. S., Paliwal, J., White, N. D. G. (2010). Detection of midge-­damaged wheat kernels using short-wave near-infrared hyperspectral and digital color imaging. Biosystems Engineering, 105: 380–387. Stone, H., and Sidel, J. L. (2004). Sensory Evaluation Practices, Elsevier Academic press, London, 408 p. Teranishi, R., Wick, E. L., and HornsteinI, R. (Eds.). (1999). Flavour Chemistry: Thirty Years of Progress, Springer, New York. Thomas, J., Mandal, A. K. A., Kumar, R. R., and Chordia, A. (2009). Role of biologically active amino acid formulations on quality and crop productivity of tea (Camellia sp.). International Journal of Agricultural Research, 4(7): 228–236. Togari, N., Kobayashi, A., and Aishima, T. (1995). Relating sensory properties of tea aroma to gas chromatographic data by chemometric calibration methods. Food Research International, 28(5): 485–493. Tschannerl, J., Ren, J., Jack, F., Marshall, S., and Zaho, H. (2017). Employing NIR SWIR hyperspectral imaging to predict the smokiness of scotch whisky. OCM. 3rd International Conference on Optical Characterization of Materials. Karlsruhe, Germany. KIT Scientific Publishing. Eds. J. Beyerer, F. Puente Leon, T. Längle. Vargas, A., Kim, M., Tao, Y., Lefcourt, A., and Chen, Y. (2004). Safety inspection of cantaloupes and strawberries using multispectral fluorescence imaging techniques 2004 ASAE Annual meeting Ottawa Can. Williams, P., Geladi, P., Fox, G., Manley, M. (2009). Maize kernel hardness classification by near infrared (NIR) hyperspectral imaging and multivariate data analysis. Analytica Chimica Acta, 653: 121–130. Williams, P. J., Geladi, P., Britz, T. J., Manley, M. (2012). Investigation of fungal development in maize kernels using NIR hyperspectral imaging and multivariate data analysis. Journal of Cereal Science, 55: 272–278. Willson, K. C., and Clifford, M. N. (1992). Tea: Cultivation to Consumption, Chapman & Hall, London, 769 p. Wright, L. P., Mphangwe, N. I. K., Nyirenda, H. E., and Apostolides, Z. (2000). Analysis of caffeine and flavan-3-ol composition in the fresh leaf of Camellia sinensis for predicting the quality of the black tea produced in Central and Southern Africa. Journal of the Science of Food and Agriculture, 80(13): 1823–1830. Xing, J., and De Baerdemaeker, J. (2005). Bruise detection on ‘Jonagold’ apples using hyperspectral imaging. Postharvest Biology and Technology, 37: 152–162. Xing, J., Saeys, W., and De Baerdemaeker, J. (2007). Combination of chemometric tools and image processing for bruise detection on apples. Computers and Electronics in Agriculture, 56: 1–13. Xue, L., Li, J., and Liu, M. (2008). Detecting pesticide residue on navel orange surface by using hyperspectral imaging. Acta Optica Sinica, 28: 2277–2280. Yamamoto, T., Juneja, L. R., Chu, D. C., and Kim, M. (1997). Chemistry and Applications of Green Tea, CRC Press, Boca Raton, 176 p. Yang, C.-C., Kim, M., Kang, S., Tao, T., Chao, K., Lefcourt, A., and Chan, D. (2011). The development of a simple multispectral algorithm for detection of fecal contamination on apples using a hyperspectral line-scan imaging system. Sensing and Instrumentation for Food Quality and Safety, 5: 10–18. Yang, C.-C., Kim, M. S., Kang, S., Cho, B.-K., Chao, K., Lefcourt, A. M., and Chan, D. E. (2012). Red to far-red multispectral fluorescence image fusion for detection of fecal contamination on apples. Journal of Food Engineering, 108: 312–319.

HSI in Quality and Safety of Beverage

261

Yi-dan, B., Na, C., Yong, H., Fei, L., Chu, Z., and Wen-Wen, K. (2015). Rapid identification of coffee bean variety by near infrared hyperspectral imaging technology, Optics and Precision Engineering, 23(2): 349–355. Zhou, R., Li, X., and He, Y. (2017). Grading of green tea and quantitative determination of beta carotene and lutein based on hyperspectral imaging. American Society and Biological Engineers. Zhu, F., Yao, H., Hruska, Z., Kincaid, R., Brown, R., Bhatnagar, D., and Cleveland, T. (2016). Integration of fluorescence and reflectance visible near infrared (VNIR) hyperspectral images for detection of aflatoxins in corn kernels. Trans. ASABE, 59: 785–794.

Chapter

19

Raman Hyperspectral Imaging Application in Food Additives’ Quality and Safety Rajesh Kumar R. Singh Prem Henna Private Limited

N.C. Basantia Cavinkare Research centre

CONTENTS 19.1 Introduction 263 19.2  Raman Hyperspectral Imaging 264 19.2.1 Instrumentation 265 19.2.2 System Calibration 267 19.2.2.1 Spectral Calibration 267 19.2.2.2 Spatial Calibration 268 19.2.3 Image Acquisition and Spectral Extraction 268 19.3  Characterization of Food Additives 269 19.4  Impurity Profile of Food Additives 270 19.5  Nanoparticle Food Additives 271 19.6 Conclusion 273 References 274

19.1 INTRODUCTION Food additives are defined as chemical substances deliberately added to foods directly or indirectly in known quantities for the purposes of assisting in the processing of foods, preservation of foods, or in improving the flavor, texture, or appearance of foods (Daniel, 2007). They help to increase the shelf life of the food by maintaining product ­consistency, wholesomeness, and freshness. Food additives can be used directly or indirectly. Direct additives are those are intentionally added to foods for a specific purpose while indirect additives are those to which the food is exposed during processing, packaging, or storing (Boca and Smoley, 1993). For example, the low-calorie sweetener aspartame, which is used in beverages, puddings, yoghurt, chewing gum, and other foods, is considered a direct additive. Many direct additives are identified on

263

264

Rajesh Kumar R. Singh and N.C. Basantia

the ingredient label of the product. Food additives are the substances not commonly regarded or used as food, but by addition affect the keeping quality, texture, consistency, taste, color, alkalinity or acidity, or to serve any other technological function in relation to food, and including processing aids. With the advent of processed foods in the second half of the 20th century, many more additives have been introduced of both natural and artificial origin (Boca and Smoley, 1993). Food additives in use today can be divided roughly into three main types: cosmetics, preservatives, and processing aids (Tuormaa, 1994). The growth in the use of food additives has increased enormously in the past 30 years. They must be added in regulated quantities, specified concentration, and should be within the acceptable daily intakes (ADIs). Food additives can have some devastating effects on the consumer when they have been added above the ADI value or specified concentration, and due to substandard quality in terms of purity, type, and quantity of impurity present and size of particle (i.e., nanoparticle). Therefore, there is a need for chemical characterization for risk assessment, surveys and in regulatory monitoring activities. Suitable analytical methods are necessary to define the nature, including isomeric composition and chemical purity: chemical characterization is also necessary for the preparation of specifications for the identity and purity of food additives. A number of analytical techniques are available in literature and specified by regulatory bodies such as chemical, biochemical, chromatographic and spectroscopic methods. However, all the techniques available are time consuming, require chemicals, and are of destructive type. Therefore, there is a need of nondestructive, fast and chemical free method for identification and quantification of impurities and their identification and quantification in food matrix.

19.2  RAMAN HYPERSPECTRAL IMAGING Raman hyperspectral imaging is an emerging platform technology for analysis of food additives. Raman hyperspectral imaging is synonymous with Raman imaging, Raman chemical imaging, and Raman molecular imaging. This technology is a fusion method that combines Raman spectral technology and hyperspectral imaging technology encompassing the advantage of both methods. This technology is widely applied to the detection of food ingredients and additives (Qin et al., 2010; Qin et al., 2014). A chief motivation for applying Raman imaging in material characterization is that most of the material is spatially heterogeneous in composition and structure. There is a fundamental need and desire to measure material properties in two or three spatial dimensions to fully understand material identity, compositional distribution, and conformational distribution. Raman hyperspectral imaging addresses these needs by providing an efficient intuitive means of visualizing the two- and three-dimensional architectures of materials in a nondestructive and noninvasive manner. Raman hyperspectral imaging is basically based on the Raman effect. The Raman effect is an important physical phenomenon that is useful for composition analysis and surface inspection. When a sample is exposed to a monochromatic light beam with high energy, such as a laser, the incident light is scattered after photons interact with sample molecules. The scattered light consists of both elastic scattering and inelastic scattering. The elastically scattered light is Rayleigh scattering, which is the majority of the scattering and has the same frequency as the incident radiation. The inelastically scattered light, a small fraction of the total scattered light, is Raman scattering

Raman Hyperspectral Imaging

265

(Smith and Dent, 2005). The frequency (or energy) of the Raman scattering is shifted from that of the incident light by the vibration energy that is increased or decreased from the ­photon–molecule interaction. Molecular information can thus be obtained through analysis of the frequency shift (or energy change) of the scattered light. Raman spectroscopy measures the oscillation of atoms in molecules. The observation of the vibrational transitions yields information about the molecular vibrational energy levels which in turn relates to molecular confirmation, structure, intermolecular interactions, and chemical bonding. Raman spectroscopy is suitable for measuring solid, aqueous, and gaseous samples, and it can detect subtle changes of various chemicals. Some advantages of Raman spectroscopy include highly specific detection, nondestructive measurement, easy sample preparation, lack of water interference, and detection capability through glass or polymer packages. 19.2.1 Instrumentation Raman imaging instrumentation advancements over the years have paralleled progression in Raman spectroscopy instrumentation. Raman imaging has benefited from advancements in laser technology that provides a highly monochromatic source, multilayer dielectric and holographic diachronic filters that effectively remove Raleigh scattered laser light, imaging spectrometers that isolate wavelengths of interest, highly sensitive charged couple device (CCD) imaging detectors, and powerful PCs having sufficient data storage capacities and processing speeds capable of handling the large data files obtained in Raman imaging experiment. Other developments in Raman imaging instrumentation include increased automation, improved ease of use and enhanced hardware robustness and stability. Due to these instrumental improvements, there has been dramatic increase in use and acceptance for a whole host of industrial, agricultural, forensic, and pharmaceutical and applications in food (Nelson and Treado, 1997). Over the past three decades, numerous instrumental designs for performing Raman imaging have been developed and commercialized. These instruments can be broadly classified as scanning (i.e., mapping) approaches or wide field (i.e. global) source illumination approaches i. Scanning Method a. Point by point scan b. Line scanning ii. Wide field Imaging a. Liquid Crystal Tunable Filter (LCTF) b. Fiber Array Spectral Translators (FAST) Scanning: Most of the Raman imaging, since the development of Raman microprobe, has been performed using scanning based instrumentation. Wide-field Imaging: A variety of instrumental advancements that have occurred over the past decades fall under this category. Such advancements include fiber array assemblies, tunable laser, and fixed or rotating dielectric filters, acoustooptic tunable filters (AOTFs), and liquid crystal tunable filters (LCTFs). In addition, wide-field Raman imaging techniques such as coherent antistrokes Raman scattering (CARS) and Stimulated Raman also falls under this category.

266

Rajesh Kumar R. Singh and N.C. Basantia

Figure 19.1 show the schematic of a Raman hyperspectral imaging system which consists of (Lee et al., 2017) i. CCD camera ii. Spectrograph iii. Laser module iv. Beam splitter v. Moveable stage. CCD camera: It is necessary to use a high-performance CCD camera for imaging because the Raman signal involves a small fraction of the photons scattered by excitation (approximately 1 in 10 million). The CCD has an area array of 1024– 1024 pixels with a spectral response greater than 90% at 800 nm. The cooling temperature is set to 65°C to reduce the dark noise of the CCD. The dynamic range of the camera sensor is 16 bits, and the camera is fixed along. The dynamic range of the camera sensor is 16 bits, and the camera is fixed along the vertical direction with the use of a ball-type regulator. Laser source: This system consists of two spatially combined 785-nm laser heads mounted on a water-cooled cold plate, a laser-diode power supply, a system chiller, a collimating lens, and a clean-up filter. The laser beams generated by the two spatially combined laser heads pass through the collimating lens, resulting in a line laser. In the study, the laser head is positioned along the horizontal direction.

FIGURE 19.1  Raman hyperspectral imaging system.

Raman Hyperspectral Imaging

267

A 785-nm laser clean-up filter was setting front of the collimating lens to improve the signal-to-noise ratio of the laser system. The maximum output of the system is 30 W. A Raman imaging spectrograph is used to disperse Raman scattering from the sample light, with the spectrograph being attached to the CCD camera with an A C-mount lens (Schneider Optics, XENOPLAN 1.4/23 mm compact). 785-nm long-pass filters: Two filters are positioned between the lens and the spectrograph to eliminate Rayleigh scattering from the sample. The software to control the camera and step motor is developed with MATLAB ® (version 7.14; Mathworks, Natick, MA, USA). The software can adjust the variables of the camera (cooling temperature, exposure time, and pre-amplifier gain and accumulation) and motor (size of step, number of scans) for optimum Raman signal acquisition. 19.2.2 System Calibration A Raman spectrum is generally presented as a shift in energy from that of the excitation source. Although Raman shift is essentially a relative unit, the spectral dimension, i.e., x-axis of a Raman spectrum is traditionally expressed as wave number in cm−1 instead of cm−1. Spectral calibration for the Raman chemical imaging system is intended to define the wave numbers for the pixels along the spectral dimension. Spectrally established light sources are spectral calibration lamps, lasers, fluorescent lamps, and broadband lamps. 19.2.2.1 Spectral Calibration For spectral calibration of relative Raman shift, it is more convenient to use an excitation source with fixed spectral output and chemicals with known relative wave number shifts. A guide for Raman shift standards for spectrometer calibration has been established by ASTM International (ASTM Standards, 2007). This guide covers Raman shift wave numbers of eight liquid and solid chemicals measured using Fourier transform or dispersive Raman spectrometers with high spectral resolutions. The eight materials are naphthalene, 1,4-bis(2-methylstyryl)benzene, sulfur, 50/50 (v/v) toluene/ acetonitrile, 4-acetamidophenol, benzonitrile, cyclohexane, and polystyrene, and they cover a wide  wave number range of 85–3327 cm−1. The well-known chemical components for Raman calibration, naphthalene, 4-cetamidophenol and polystyrene, are used for spectral calibration of the Raman hyperspectral imaging system. These chemicals are scanned using the Raman hyperspectral imaging system, and the corresponding Raman spectrum is extracted from the Raman image data. Five and seven peaks are identified in the Raman spectra of polystyrene and naphthalene, respectively, collectively covering a wavenumber range between 513.8 and 1602.3 cm−1. Linear, quadratic, and cubic regression functions are used to establish the relationship between the known wavenumbers of the 12 identified Raman peaks and the corresponding pixel indices along the horizontal dimension of the CCD. Performances of the three regression models are evaluated by predicting the 12 known wavenumbers using the pixel indices. Root mean squared errors (RMSE) are calculated between the true wavenumbers and the prediction values of 8.49, 2.46, and 2.44 cm−1 for the linear, quadratic, and cubic functions, respectively. Based on the spectral calibration results using the quadratic model, the Raman chemical imaging system is found to cover a wavenumber range of 102.2–2538.1 cm−1. Wavelength and wavenumber can be converted to each other.

268

Rajesh Kumar R. Singh and N.C. Basantia

19.2.2.2 Spatial Calibration Figure 19.2 shows a single band image obtained by the Raman chemical imaging system for a standard resolution test chart. The 785 nm laser is used to illuminate the glass chart placed on a piece of square white paper. The diameter of the smallest dots in the central area of the test chart is 0.25 mm, and distance between these dots is 0.50 mm. The outermost large dots are positioned within a 50-mm square. A step size of 0.1 mm was used to scan both the x and y directions. The 0.25 mm dots can be clearly discerned in the image owing to the small step sizes used to scan the chart. The step sizes for the two scan directions determine the spatial resolutions of the Raman images: the smaller the step sizes, the higher the spatial resolutions. No image distortions are observed due to the point scanning method used to acquire spatial information from the target. In practice, the spatial resolution of the image and the time used to finish the scan are always a pair of trade off parameters and need to be determined case by case to meet specific requirements for different applications. 19.2.3 Image Acquisition and Spectral Extraction In order to obtain the optimal Raman scattering signal, samples are positioned such that to be excited uniformly by the line laser. The moving distance is set to acquire Raman images with the same spatial resolution. The Raman image is obtained without binning for pixels along the horizontal and vertical directions. In order to obtain an optimal

FIGURE 19.2  Image of a resolution test chart acquired by the Raman chemical imaging system with a step size of 0.1 mm for both the x and y scan directions.

Raman Hyperspectral Imaging

269

Raman signal without damaging the samples, the laser power and CCD exposure time needs to be set. A dark image acquired with laser off is also acquired with a cap covering the lens or probe to subtract from the original Raman hyperspectral image data. After image acquisition and calibration (subtraction of the dark image), the region of interest (ROI) is extracted using the Raman hyperspectral image data.

19.3  CHARACTERIZATION OF FOOD ADDITIVES Chemical characterization plays a critical role in risk assessment. Risk characterization brings together the results of hazard characterization with the estimated human exposure. According to the report of the WHO Scientific Group on Procedures for Investigating Intentional and Unintentional Food Additives, 1967, adequate specifications for identity and purity should be available before initiation of toxicological work (WHO, 1967). Specifications of identity and purity are necessary products of JECFA safety evaluations for food additives. A food additive may be a single chemical substance, a manufactured chemical mixture, or a natural product. Complete information on chemical composition—including description, methods of manufacture and raw materials, and analyses for impurities is equally important for each type of additive (FAO/WHO, 2005, 2006). However, implementation of the requirement for chemical composition data may vary depending on the type of substance. For commercially manufactured complex mixtures, such as mono and diglycerides, information is needed on the range of substances produced, with emphasis on descriptions of manufacturing processes, supported by analytical data on the components of the different commercial products. Natural products present particularly difficult problems due to their biological variability and because the chemical constituents are too numerous for regular analytical determinations. For additives derived from natural products, it is vital that the sources and methods of manufacture are defined precisely. Chemical composition data should include analyses for general chemical characteristics, such as proximate analyses for protein, fat, moisture, carbohydrate, and mineral content. Analyses should be undertaken for specific toxic impurities carried over from raw materials or chemicals used in the manufacture of the product. Presently various chemical, biochemical, gravimetric, titrimetric, spectrophotometric, IR spectra, or chromatographic methods are being used for identification and quantification of food additives. These methods are time consuming, need chemicals, are of destructive type, and are affected by impurities and interferences. In recent years, Raman hyperspectral imaging has become an emerging technique that integrates conventional imaging and spectroscopy to combine spatial and spectral information from a sample (Gendrin et al., 2008). The use of vibrational spectroscopy such as Raman is particularly appreciated within the pharmaceutical research and development environment. However, few studies have been carried out on application of Raman hyperspectral imaging in food additives. The use of Raman spectroscopy for the detection of trace crystallinity (Widjaja et al., 2011) and the determination of active content within pharmaceutical capsules (Matousek et al., 2007) are of great interest for the development and the quality control of a formulation. Confocal Raman Microscopy (CRM) is a well-established and widely used spectroscopic method for the investigation of the chemical composition of a sample. In pharmaceutical research, CRM can be used to probe the distribution of components within formulations, to characterize the homogeneity of pharmaceutical samples, to determine the state of drug substances and excipients, and to characterize contaminants and foreign

270

Rajesh Kumar R. Singh and N.C. Basantia

particulates (Smith et al., 2015). The information obtained by CRM is also useful for drug substance design, for the development of solid and liquid formulations, as a tool for process analytics and for patent infringement and counterfeit.

19.4  IMPURITY PROFILE OF FOOD ADDITIVES A variety of impurities are known to occur in many organic chemicals. In most instances, these impurities occur as traces or are innocuous compounds, biologically inactive and of little technologically significant for the quality of chemicals. Chemicals used as drugs, analytical reagents, certain organic solvents, standards, or chemicals used for special purpose need normally to meet specific criteria for their purity. Chemicals used in the food industries such as food additives are of no exception. While the importance of eliminating impurities from drugs has been well acknowledged, the possibility of unwarranted side effects from the traces of impurities in food additives is only recently recognized. Impurities in synthetic food additives or agricultural chemicals are usually by products of the synthetic process and may arise from side reaction of the main process or from incomplete reactions or from the reaction of contaminants already present in the starting materials. It is important to give greater consideration to these detrimental impurities. The actual and potential impurities in food additives are most likely drugs during the synthesis, purification, and storage. Impurities are associated with raw materials could also contribute to the impurity profile. These impurities may be organic, inorganic, residual solvent, or degradation product type. Additionally, some impurities can cause toxicological problems. The presence of these unwanted chemicals, even in small amounts, may influence the efficacy and safety as discussed in pharmaceutical products. The presence of traces of certain impurities in food additives used for various biological tests for assessment of toxicity gives erroneous impression of the toxicity of the food additives (Stavric et al., 1979). Impurity profiling (i.e., the identity as well as the quantity of impurity in the pharmaceuticals) is now receiving critical attention from regulatory authorities in pharmaceutical products. In food additives, it is now recognized for its importance. The large number of food additives, the different raw materials used for same additive and the low concentrations present a significant analytical challenge for the detection, quantitation, and characterization of the compounds. The procedure of impurity profiling begins with detection of impurities using thin-layer chromatography (TLC), high performance liquid chromatography (HPLC), gas chromatography (GC), and identification through impurity standards. In case of unsuccessful identification with impurity standards, impurity starts with UV spectra followed by structure of impurity with NMR spectra data. If the UV spectra are not sufficient, then mass spectrum of the impurity is taken into account. However, the solubility, volatility, and thermal stability of the impurity cause problems in chromatographic separation. As an alternative, derivatization reaction is widely used in GC/MS analysis. Therefore, there is a dire need of a nondestructive, confirmatory method for detection, identification and quantification of impurities in food additives based on the experiences of pharmaceutical products. Raman chemical imaging (R-CI) and its specialized versions such as hyperspectral stimulated Raman scattering microscopy and coherent anti-Stokes Raman scattering microscopy have become important imaging techniques for quantitative analysis. However, in spite of these improvements, some typical disadvantages of Raman spectroscopy, such

Raman Hyperspectral Imaging

271

as low sensitivity and long image acquisition time, still limit the applicability of R-CI. Taking advantage of the powerful signal enhancing behavior of metal (primarily silver and gold) substrates, Surface enhanced Raman Spectroscopy (SERS) can offer a solution for the mentioned difficulties. Firkala et al. (2015) reported the application of surface enhanced Raman chemical imaging (SER-CI) as a nondestructive quantitative analytical method for the investigation of model pharmaceutical formulations containing the active pharmaceutical ingredient (API) in low concentrations (0.5–2%). Recently, Raman spectroscopy has been combined with optical microscopy, giving rise to Raman hyperspectral imaging (Netchacovitch et al., 2015). The use of the latter has expanded in the pharmaceutical and biomedical fields since microspectroscopic techniques present several advantages by combining the acquisition of spatial and spectral information from a sample. Raman hyperspectral imaging allows characterizing the sample in terms of chemical (API and excipients) identification and content for example. Impurities are, most of the time, present in a very low dosage in the pharmaceutical products, consequently preventing their detection using Raman imaging. Surface-enhanced Raman chemical imaging (SER-CI) was therefore used for the quantitative detection of an impurity. Bleye et al. (2014) detected quantitatively 4-aminophenol (4-AP), the main impurity of paracetamol, using Surface-enhanced Raman chemical imaging (SER-CI). This technique potentially could be used for identification and quantification of impurities in food additives. The potential of Raman hyperspectral imaging technique needs to be explored for its application in food additives for quantification of impurities at lower concentration.

19.5  NANOPARTICLE FOOD ADDITIVES The advent of nanotechnology, that involves manufacture and use of materials in the size range of up to 100 nanometers, has opened a way for a multibillion dollar global industry in recent years. The applications of nanotechnology in the food sector are relatively new, but they are predicted to grow rapidly in the coming years. This is because food industry has always been looking out for new technologies to improve the nutritional value, shelf life, and traceability of food products and to provide new tastes, flavors, textures, etc. A number of new processes and materials derived from nanotechnology can provide answers to such needs. The main developments in nanofood area have so far been aimed at altering the texture of food components, encapsulating food components or additives, developing new tastes and sensations, controlling the release of flavors, and/or increasing the bioavailability of nutritional components. It is clear from a number of reports, reviews, patents, and company products that nanotechnology applications have started to make an impact on different aspects of the food and associated industries (Chen et al., 2006). The currently available information suggests that both organic and inorganic nano-materials have been used as food additives. The organic food additive lycopene (BASF) is a carotenoid nanomaterial with particle size in the range of 100 nm. The materials for coating of confectionary products (Patent application by Mars Inc.) include the permitted additives silicon dioxide (SiO2 , E551), magnesium oxide (MgO, E530), and titanium dioxide (TiO2 , E171), which are preferably insoluble. The coating is applied using a continuous process as a thin amorphous film of 50 nm or less. The Bioral (TM) nanocochleate delivery system contains phosphatidylserine carrier that can be as small as 50 nm in diameter, comprising a crystalline

272

Rajesh Kumar R. Singh and N.C. Basantia

latticework with an anhydrous interior. The diameter of nanomicelle-based carrier system Novasol® (marketed by Aquanova® Germany) for introduction of antioxidants in food and beverage products is approximately 30 nm. More details on other nanostructured food ingredients are currently not available, but it is anticipated that the average particle size will be up to 100 nm (to fulfill the current definition of a nanomaterial). It is known that materials manufactured at nanoscale may have substantially different physicochemical and biological properties from their conventional forms. This is because conventional physicochemical rules are not as well understood at the nanometer scale. Depending on the physicochemical nature of the material, quantum effects may have a much greater influence on the properties of a nanomaterial compared to larger particles. Also, on a weight per weight basis, nanomaterials have much larger surface areas compared to say, microparticles (10–6 m). Very few studies have been carried out so far into the toxicology of nanoparticles, and much of the published research relates to inhalation exposure. The potential effects of nanoparticles through the gastrointestinal route are largely unknown. The application of nanotechnology in food has, therefore, led to concerns that ingestion of nanoparticles may pose unforeseen health or environmental hazards. Such concerns have arisen from a growing body of scientific evidence which indicates that free nanoparticles can cross cellular barriers and that exposure to some engineered nanoparticles can lead to increased production of ox radicals and consequently oxidative damage to the cell. However, despite the potential of some nanoparticles for every authorized food additive included in the positive list, a specification must be laid down that contains the criteria on the purity and defines the origin of the food additive, and the verification of such criteria. The most relevant aspect in relation to the use of nanoscale food additives is perhaps in the re-evaluation of safety assessment. To ensure that food additives once permitted are kept under continuous observation and re-evaluation wherever necessary. Nanomaterials are increasingly prevalent throughout industry, manufacturing, and biomedical research. The need for tools and techniques that aid in the identification, localization, and characterization of nanoscale materials in food samples is on the rise. As nanomaterials are increasingly used in a variety of industries and applications, there is a need for nanoscale imaging and characterization methods that are more rapid, affordable, and convenient than traditional modalities, such as electron microscopy (Roth et al., 2015). To visualize nanoparticle (NP) interactions with cells, tissues, and living systems, many optical techniques have been employed, including differential interference contrast (DIC) microscopy (Sun et al., 2008) and evanescent field-based approaches, such as total internal reflection (TIR) or near-field scanning optical microscopy (NSOM) (Anselme et al., 2010). However, these are high-end analytical approaches, out of reach of most nonspecialist labs (Weinkauf et al., 2009). Electron microscopy, including transmission electron microscopy (TEM), has also been used to study NP interaction with cells (Roth et al., 2015). High angle annular dark field (HAADF) scanning transmission electron microscopy has been used to study the interaction of NPs with viruses (Elechiguerra et al., 2005). Confocal microscopy is another popular technique used to study NP-cell interactions, (Carlson et al., 2008). Hyperspectral imaging system complemented with Enhanced dark field microscopy may provide a solution to this bottleneck by enabling rapid and less expensive characterization of nanoparticles in histological samples. Badireddy et al. (2012) introduced a novel methodology based on hyperspectral imagery with enhanced dark field microscopy for detection, characterization, and analysis of engineered nanoparticles in both

Raman Hyperspectral Imaging

273

ultrapure water and in complex waters, such as simulated-wetland ecosystem water and wastewater. Hyperspectral imagery with enhanced dark field microscopy can be a promising tool for detection and characterization of engineered nanoparticles in environmental systems, facilitating studies on fate and transformation of these particles in various types of water samples. Hyperspectral imaging microscopy (HSI-M) possesses several advantages that make the technique very compelling for NP analysis. Perhaps the most important one is the simplicity of sample preparation. Many common techniques used for NP characterization easily introduce artifacts through the necessary manipulation and sample preparation (Baer et al., 2008; Tiede et al., 2008). This is because the behavior of NPs is very sensitive to changes in their environment (e.g., pH, ionic strength, ligands that suppress or enhance agglomeration) but also because particles may be altered by high energy radiation such as X-rays (Baer et al., 2005) or electron beams (Koh et al., 2009). In HSI-M, samples can be directly placed on glass slides and analyzed without any further pre-treatment. It is known that many nanomaterials have physicochemical properties that differ from physicochemical properties of the same material in bulk size. It is therefore important to characterize nanoparticles used as food additives and to evaluate their toxicity. Ahlinder (2015) studied metal oxide and carbon-based nanoparticles in living cells using Raman spectroscopy. Raman spectroscopy is a method that facilitates a nondestructive analysis without using any fluorescent labels or any other specific sample preparation. It is possible to collect Raman images, i.e., images where each pixel corresponds to a Raman spectrum, and to use the spectral information to detect nanoparticles, and to identify organelles in cells. Although hyperspectral imaging complemented with microscopy or Raman spectroscopy has been used to characterize and identification nanoparticles in soil, environment, water, and living cells, it has the potential to be explored in food additives such as synthetic amorphous silica and migration added nanoparticles in food packaging material.

19.6 CONCLUSION Food additives play a vital role in food industries. Additives has been used for many years to preserve, flavor, blend, thicken, and color foods and have played an important and essential role in reducing serious nutritional deficiencies through supplementation of nutritional additives. But the various adverse effects associated with them due to the quality of the additives in terms of purity, type and quantity of impurities, contaminants, reaction of impurities and degradation products with food components and reaction with the cellular components of the body which leads to health risk. Therefore, proper maintenance of the quality of food additives and reliable risk assessment is essential. National and international regulatory bodies are making laws and specifications more and more stringent. At the same time more and more new food additives and formulations with better functionality are rushing in to the market. Therefore, it is becoming a challenge for analytical scientists and regulators to develop a suitable technique to identify, quantify, and characterize the additive accurately, fast at its native state and identify and restrict the use of unapproved food additives in food. Raman hyperspectral imaging as such or complemented with other techniques such as dark field microscope, surface enhancing resolution (SER), and various advancements could address the present as well as upcoming challenges food additive analysis and characterization. The upcoming challenges are such as characterization and risk assessment of engineered nanoparticles, migration

274

Rajesh Kumar R. Singh and N.C. Basantia

of engineered nanoparticle additives in packaging material, differentiation between genetically modified and nonmodified additives, natural and synthetic food additives, and identification and quantification of unpredictable impurities. While Raman spectroscopy complemented with hyperspectral imaging has long been established as proven ­technology for the harsh environments of military and remote sensing applications, its use in pharmaceutical manufacturing operations has been expanded considerably in the past few years and initiation has been taken in the food sector. Undoubtedly, it will be the chosen technology for the food additive sector. The advancements in commercially available equipment and the exploring of this technology in the food additive area will make it capable to address the above-mentioned challenges.

REFERENCES Ahlinder, L. (2015). Raman spectroscopy and hyperspectral analysis of living cells exposed to nanoparticles. Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 1257. 72 pp. Uppsala: Acta Universitatis Upsaliensis. ISBN 978-91-554-9250-2. Anselme, K., et al. (2010). The interaction of cells and bacteria with surfaces structured at the nanometer scale. Acta Biomate. 6(10), 3824–3846. doi:10.1016/j. actbio.2010.04.001. ASTM Standards. (2007). E1840‐96: Standard guide for Raman shift standards for spectrometer calibration. West Conshohocken, PA: ASTM. Badireddy, A. R., et al. (2012). Detection, characterization, and abundance of engineered nanoparticles in complex waters by hyperspectral imagery with enhanced dark field microscopy. Environmental Science & Technology. 46(18), 10081–10088. doi:10.1021/es204140s. Baer, D. R., Engelhard, M. H., Gaspar, D. J., Matson, D. W., Pecher, K. H., Williams, J.  R., Wang, C. M. (2005). Challenges in applying surface analysis methods to nanoparticles and nanostructured materials. Journal of Surface Analysis. 12(2), 101. Baer, D. R., et al. (2008). Characterization challenges for nanomaterials. Surface and Interface Analysis. 40(3–4), 529–537. Bleye, C. D., Sacré, P. Y., Dumont, E., Netchacovitch, L., Chaveza, P.-F., Piel, G., Lebrun, P., Hubert, P., Ziemons, E. (2014). Development of a quantitative approach using surface-enhanced Raman chemical imaging: First step for the determination of an impurity in a pharmaceutical model. Journal of Pharmaceutical and Biomedical Analysis. 90, 111–118. Boca, F. L., Smoley, C. K. (eds). (1993). U.S. Food and Drug Administration. Everything Added to Food in the United States. CRC Press, Inc., New York. 171 p. Carlson, C., et al. (2008). Unique cellular interaction of silver nanoparticles: Sizedependent generation of reactive oxygen species. Journal of Physical Chemistry A. 112(43), 13608–13619. doi:10.1021/jp712087m. Chen, H., Weiss, J., Shahidi, F. (2006). Nanotechnology in nutraceuticals and functional foods. Food Technology. 60(3), 30–36. Daniel, M. (2007). Reactions to Food Additives and Preservatives. Elechiguerra, J. L., et al. (2005). Interaction of silver nanoparticles with HIV-1. Journal of Nanobiotechnology. 3(6), 1–10. doi:10.1186/1477-3155-3-6. FAO/WHO. (2005). Codex Alimentarius Commission Procedural Manual, 15th Ed., Food and Agriculture Organization of the United Nations, Rome.

Raman Hyperspectral Imaging

275

FAO/WHO. (2006). Manual on development and use of FAO and WHO specifications for pesticides. March 2006 revision of the First Edition (available at www.fao.org/ ag/AGP/AGPP/Pesticid/Default.htm). Firkala, T., Farkas, A., Vajna, B., Nagy, Z. K., Pokol, G., Marosi, G., Szilaggi, M. (2015). Journal of Pharmaceutical and Biomedical Analysis. 107, 318–324. Gendrin, C., Roggo, Y., Collet, C. (2008). Pharmaceutical applications of vibrational chemical imaging and chemometrics: A review. Journal of Pharmaceutical and Biomedical Analysis. 48, 533–553. Koh, A. L., et al. (2009). Electron energy-loss spectroscopy (EELS) of surface plasmons in single silver nanoparticles and dimers: influence of beam damage and mapping of dark modes. ACS Nano. 3(10), 3015–3022. Lee, H., Moon, S. Kim, l., Qin, J., Song, Y. R., Oh, C. S., and Cho, B. K. (2017). Raman hyperspectral imaging for detection of watermelon seeds infected with Acidovorax citrulli. Sensors. 17, 2188. Matousek, P., Parker, A. W. (2007). Non-invasive probing of pharmaceutical capsules using transmission Raman spectroscopy. Journal of Raman Spectroscopy. 38, 563–567. Nelson, M. P., Treado, P. J. (1997). Raman imaging instrumentation, Raman, Infrared and Near Infrared Chemical Imaging, Edited by Sasic, S. and Ozaki, Y. Wiley, UK, Chapter-2. Netchacovitch, L., Thiry, J., De Bleye, C., Chavez, P.-F., Krier, F., Sacré, P.-Y., Evrard, B., Hubert, P., Ziemons, E. (2015). Vibrational spectroscopy and microspectroscopy analyzing qualitatively and quantitatively pharmaceutical hot melt extrudates. Journal of Pharmaceutical and Biomedical Analysis. 113, 21–33. Qin, J., Chao, K., Kim, M. S. (2010). Raman chemical imaging system for food safety and quality inspection. Trans ASABE. 53 (6), 1873–1882. Qin, J., Chao, K., Kim, M. S. (2014). A line-scan hyperspectral system for ­high-throughput Raman chemical imaging. Applied Spectroscopy. 68 (6), 692–695. Roth, G. A., Sosa Peña, M. d., Neu-Baker, N. M., Tahiliani, S., Brenner, S. A. (2015). Identification of metal oxide nanoparticles in histological samples by enhanced dark field microscopy and hyperspectral mapping. Journal of Visualized Experiments. (106), e53317:1-9. doi:10.3791/53317. Smith, G. P. S., McGoverin, C. M., Fraser, S. J., Gordon, K. C. (2015). Raman imaging of drug delivery systems. Advanced Drug Delivery Reviews. 89, 1–21. Smith, W. E., Dent, G. (2005). Modern Raman Spectroscopy: A Practical Approach. Chichester, UK: John Wiley and Sons. Stavric, B., Stoltz, D. R., Klassen, R. (1979). Trace Organic Analysis: A new frontier in Analytical Chemistry, National Bureau of Standards Special Publication 519, Proceedings of 9th material Research Symposium, April 10–13, 1978, Gaithersburg. Sun, W., et al. (2008). Endocytosis of a single mesoporous silica nanoparticle into a human lung cancer cell observed by differential interference contrast microscopy. Analytical and Bioanalytical Chemistry. 391(6), 2119–2125. doi:10.1007/s00216-008-2162-1. Tiede, K., et al. (2008). Detection and characterization of engineered nanoparticles in food and the environment. Food Additives and Contaminants. 25(7), 795–821. Tuormaa, T. E. (1994). The adverse effects of food additives on health: A review of the literature with special emphasis on childhood hyperactivity. Journal of Orthomolecular Medicine. 9(4), 225–243. Weinkauf, H., et al. (2009). Enhanced dark field microscopy for rapid artifact-free detection of nanoparticle binding to Candida albicans cells and hyphae. Biotechnology. 4(6), 871–879. doi:10.1002/biot.200800358.

276

Rajesh Kumar R. Singh and N.C. Basantia

WHO. (1967). Procedures for investigating intentional and unintentional food additives. Report of a WHO. Widjaja, E., Kanaujia, P., Lau, G., Ng, W. K., Garland, M., Saal, C., Hanefeld, A., Fischbach, M., Maio, M., Tan, R. B. H. (2011). Detection of trace crystallinity in an amorphous system using Raman microscopy and chemometrics analysis. European Journal of Pharmaceutical Sciences. 42, 45–54.

Index A Abstract factor analysis (AFA), 93 Acceptable daily intake (ADI), 264 Acousto-optic tunable filter (AOTF), 10, 11, 225, 265 Active pharmaceutical ingredient (API), 271 Agaricus bisporus, see Mushroom Airborne Visible/Infrared Imaging Spectrophotometer (AVIRIS), 8 Alcoholic beverages, 246–249, 256 ALS, see Alternative least squares (ALS) Alternative least squares (ALS), 241 Ambient light, 24–25, 30–31 Amino acids, 209, 250 Analysis of variance (ANOVA), 207, 223, 225, 241 Angle Measure Technique (AMT), 103 Anisakis, 160 ANN, see Artificial neural network (ANN) Anthocyanin, 248 AOTF, see Acousto-optic tunable filter (AOTF) API, see Active pharmaceutical ingredient (API) Arabica coffee, 12, 249 Aristolochia fangchi, 235, 236 Artificial neural network (ANN), 12, 65, 67, 111, 115, 126–131, 209, 224, 250 architectures/topology multilayer feed-forward neural networks, 128–129 radial basis function neural networks, 129–130 self-organizing maps, 130–131 definition of, 126 food quality control applications in, 126, 131–133 consumer demands, 246 neuron models McCulloch–Pitts model, 127 perceptron model, 127–128 objectives, 132, 134 script for, 150–151 AVIRIS, see Airborne Visible/Infrared Imaging Spectrophotometer (AVIRIS)

B Back propagation neural network (BPNN), 116, 121, 207, 208, 215 Bayesian framework segmentation method, 38–39 Beer–Lambert law, 13 Bell pepper, 225 Binary partition tree (BPT), 123 Biological neural networks, 127 Biplot, principal component analysis, 89 Broccoli, 226 Brushing procedure, pixel-based approach, 100 C Camellia sinensis, 249, 250 Canopy spectral reflectance, 216–217 CARS algorithm, see Competitive adaptive reweighted sampling algorithm (CARS) algorithm CDMs, see Concentration distribution maps (CDMs) Charge-coupled device (CCD), 8, 77, 184, 225, 265, 266 Cheese analysis, 240 Cheese ripening techniques, 240 Chemical analysis, of food, 4 Chemometrics, 6, 8, 12–14, 45 Chilling injury, on cucumbers, 217–219 Classical least squares regression (CLS), 241 Competitive adaptive reweighted sampling algorithm (CARS) algorithm, 165, 169, 181–183 Concentration distribution maps (CDMs), 237 Confocal microscopy, 272 Confocal Raman Microscopy (CRM), 269, 270 Conventional culture techniques, 5 Conventional imaging technology, 7 Conventional spectroscopy, 4, 14 Cucumber, 217–221 Cuticle cracks, 216 D Dairy products analysis, hyperspectral imaging in, 239–240 authentication of, 240–241 cheese analysis, 240

277

278

Index

Dark reference materials, 23, 26–27 Data extraction and treatment image feature extraction gray level co-occurrence matrix, 51–56 run length matrices, 50–51 spectral extraction, 46–48 treatment, 48–50 structure of, 45, 46 Daubechies8 (db8), 111 Decision tree (DT), 123–124 Differential interference contrast (DIC) microscopy, 272 2-D matrix, see Two-dimensional (2-D) matrix Docosahexaenoic acid (DHA), 160 DT, see Decision tree (DT) E EBC, see European Brewing Convention (EBC) Echinacea angustifolia, 234 Echinacea pallida, 234 Echinacea purpurea, 234 Electronic spectroscopy, 6 Electron microscopy, 272 Electron multiplying charge coupled device (EMCCD), 255 End member threshold selection method, 40 Enterobacteriaceae, 111, 169 Environment for visualizing images (ENVI), 46, 74–75 European Brewing Convention (EBC), 247 Evolutionary cellular automata-based segmentation method, 39 F Fish blood concentration images, 165, 166 chemical compositions of, 165–168 fillets, 165, 166 freshness detection in, 160–164 frozen-thawed, 120 inspection microbial spoilage, 169 nematodes, 168–169 overview of, 159–160 physical properties of, 164–165 and seafood, 160, 169 thiobarbituric acid values, 166 Fisher’s linear discriminant (FLD), 115 Food additives, 263, 264 characterization of, 269–270 impurity profile, 270–271 nanoparticle, 271–273 Raman hyperspectral imaging in, 269

Food adulteration detection in lamb meat, 183–185 in minced beef, 187–188 in minced lamb, 185–187 in minced meat, 185 Food safety and quality alcoholic beverages beer, 247–248 whisky, 249 wine, 248 beverage, applications of, 252–254, 256 nonalcoholic beverages coffee, 249 fruit-based, 251–256 tea, 249–250 Frame-based classification approach, 210 Freshness detection, in fish, 160–164 Frozen-thawed (F-T) fish, 120 Fruit(s) linear discriminant analysis, 116–117 overview of, 195–197 partial least squares regression, 111–112 quality of, 197–199 support vector machines, 120 types of, 199–201 Fruit fly infestation, 220 F-T fish, see Frozen-thawed (F-T) fish G Gabor filters, 50, 55–56 Global spectrum method, 49 Glycine max L., see Soybean Gray-level co-occurrence matrix (GLCM), 50–56, 103, 111, 120, 121 H HAADF, see High angle annular dark field (HAADF) Harpagophytum procumbens, 234 Harpagophytum zeyheri, 234 HCI, see Hyperspectral chemical imaging (HCI) HgCdTe, see Mercury cadmium telluride (HgCdTe) Hierarchical segmentation method, 38 High angle annular dark field (HAADF), 272 High Performance Liquid Chromatography (HPLC), 249 HIT, see Hyperspectral imaging technique (HIT) HPLC, see High Performance Liquid Chromatography (HPLC) HSI-M, see Hyperspectral imaging microscopy (HSI-M)

Index HSI-NIR, see Near infrared hyperspectral images (HSI-NIR) Hypercube, 8–9, 61, 197 Hyperspectral chemical imaging (HCI), 21–22 Hyperspectral data, unfolding and refolding, 93–94 Hyperspectral imaging (HSI), 39–42, 95, 96, 111, 115–117, 132, 143, 160, 163–170, 196–199, 202, 208–210 Hyperspectral imaging microscopy (HSI-M), 273 Hyperspectral imaging technique (HIT), 22–23, 126 advantages, 14 analysis, 12–14 components and set-up, 9–11 definition of, 3 disadvantages, 15 history of, 8 principles of, 8–9 script for, 143–147 I Illicium anisatum, 234 Illicium verum, 234 Image acquisition, 268–269 Image-based approach, 94–98 Image feature extraction gray level co-occurrence matrix, 51–56 run length matrices, 50–51 Image segmentation supervised, 37–39 unsupervised, 39–41 Image texture algorithms, 103 Impurity, food additives, 270–271 Indium gallium arsenide (InGaAs), 10, 23 J Jet Propulsion Laboratory, 3 K Kernel principal component analysis (KPCA), 121 Kjeldahl method, 5 k-means clustering method, 41 K-nearest neighbor (KNN), 121 segmentation method, 39 Kubelka–Munk (K–M) units, 48 L Lamb meat, 74, 76, 177, 183–185 Laser source, 266 Latent variables (LVs), 64, 67 LCTF, see Liquid crystal tunable filter (LCTF)

279

LDA, see Linear discriminant analysis (LDA) Leaf pigments, 208 Least squares support vector machine (LS-SVM), 67, 78, 111, 119, 120, 167 Lettuce decay indices (LEDI), 221–223 Linear discriminant analysis (LDA) definition of, 115 fruits and vegetables, 116–117 wheat, 115–116 Lipid oxidation detection, 165 Liquid crystal tunable filter (LCTF), 10, 22 Loading matrix, 88 LS-SVM, see Least squares support vector machine (LS-SVM) LVs, see Latent variables (LVs) Lycopene, 215 Lycopersicon esculentum, see Tomato M McCulloch–Pitts model, 127 Mahalanobis distance (MD), 23, 26–27, 219 MAP, see Modified atmosphere packaging (MAP) Mathematical data treatment, 25–26 MATLAB®, 74, 75, 207, 267 MBM, see Meat and bone meal (MBM) MCR, see Multivariate curve resolution (MCR) MD, see Mahalanobis distance (MD) Meat and bone meal (MBM), 21 ambient light, 24–25, 30–31 dark reference materials, 23, 26–27 hyperspectral imaging equipment, 22–23 mathematical data treatment, 25–26 partial least squares regression, 109–110 sample presentation, 25, 31 scanning frequency, for dark and white references, 24, 28–30 Meat-processing industry, 175, 176 Mechanical injury, 210, 219, 221 Medicinal herbs applications, 236 Melamine adulteration, 240–241 partial least squares regression, 112 Mercury cadmium telluride (HgCdTe), 10 MIA techniques, see Multivariate image analysis (MIA) techniques Microbial spoilage inspection, 169 Minimum noise fraction (MNF), 116, 120 Minimum redundancy–maximum relevance (MRMR), 219 Minimum spanning forest method, 39 MLF-NNs, see Multilayer feed-forward neural networks (MLF-NNs)

280

Index

M. longissimus dorsi (LD), 75, 124 MLP, see Multilayer perceptron (MLP) MLR, see Multiple linear regression (MLR) MNF, see Minimum noise fraction (MNF) Modified atmosphere packaging (MAP), 222, 223 Moisture content, 181–184 M. psoas major (PM), 75 MRMR, see Minimum redundancy–maximum relevance (MRMR) MSC, see Multiplicative scatter correction (MSC) Multilayer feed-forward neural networks (MLF-NNs), 128–129 Multilayer perceptron (MLP), 132 Multiple linear regression (MLR), 66, 111, 183, 188, 209, 251 Multiplicative scatter correction (MSC), 13, 49 Multispectral fluorescence-based imaging algorithm, 216, 223 Multispectral imaging, 11–12 Multivariate analysis applications of, 75–79 chemometric terms, definitions of, 66 classification map development, 70–72 model, 65, 68 hyperspectral images, 62, 63–65 image processing, 69–70 prediction map, 71, 73–74 regression model, 66–69 software for, 74–75 Multivariate calibration model, 63, 64, 67, 76 Multivariate curve resolution (MCR), 237, 241 Multivariate data, definition of, 66 Multivariate image analysis (MIA) techniques, 70 Multivariate image processing, 69–70 Multivariate regression model, 66–67 Multivariate system, 86 Multiway PLS (N-PLS) algorithm, 111 Mushroom, 116, 209–213 N Nanomaterials, 271, 272, 273 Nanoparticle (NP), 271–273 National Aeronautics and Space Administration Jet Propulsion Laboratory (NASA/JPL), 8 Near infrared hyperspectral images (HSINIR), 237 Near-infrared spectroscopy (NIRS), 4–6, 14, 15, 21, 35–36, 77, 115, 164, 185, 196, 198, 208, 226, 240, 241

Nematodes inspection, 168–169 Nilaparvata lugens, 121 NIPALS algorithm, see Nonlinear iterative partial least squares (NIPALS) algorithm NIR HSI system, 247, 248 NIR hyperspectral imaging, 121 NIRS, see Near-infrared spectroscopy (NIRS) NIR-SWIR-HSI, 249 Nonalcoholic beverages, 249–256 Nonlinear iterative partial least squares (NIPALS) algorithm, 90–92 N-PLS algorithm, see Multiway PLS (N-PLS) algorithm O Object-based approach, 94, 98, 99 Octave, open access software, 132–134 artificial neural network modeling, 140–143 data extraction, 138–140 dimensionality reduction, 139–140 image acquisition, 135–136 methodology, 134, 135 preprocessing stage, 136–138 texture profile analysis, 134–135 ω-3 fatty acids, 160 Onion, 226 Oolong tea, 249 Organic food additive lycopene (BASF), 271 Orthogonal projections to latent structures discriminant analysis (OPLS-DA), 234 Otsu’s method, 47 P Partial least squares-discriminant analysis (PLS-DA), 14, 77, 109, 121, 199, 208, 210, 212, 214, 224, 234–237, 240, 251, 255 Partial least squares regression (PLSR), 66, 67, 75, 76–78, 119–120, 184, 187, 188, 214, 240, 248 bacteria, 110–111 definition of, 109 fruits, 111–112 meat, 109–110 seeds and melamine, 112 PCA, see Principal component analysis (PCA) PCA-based image texture analysis, 102–104 PCA-FLD method, 117 PCR, see Principal component regression (PCR) PCS, see Principal component spectra (PCS)

Index Peppers, 225–226 Perceptron model, 127–128 Performance metrics, 41 Pickled cucumber, 218, 219 Pixel-based approach, 98–102 Pixel-based segmentation, see Supervised segmentation PLS-DA, see Partial least squares-discriminant analysis (PLS-DA) PNN, see Probabilistic neuronal network (PNN) Point scan approach, 11 Polynomial curve-fitting method, 215 Polyphenol(s), 248, 250 Polyphenol oxidase (PPO), 211 Polyunsaturated fatty acids (PUFAs), 160, 165 Potato, 213–215 Prediction map, 71, 73–74 Pre-processing methods, for hyperspectral images, 36–37 Pre-treatment, definition of, 66 Principal component (PC), 66–67, 85–88 Principal component analysis (PCA), 13, 65, 70, 75, 77, 115, 120, 140, 184, 209, 233–235, 240, 255, 256 application, script for, 149–150 compression method, 92–93 exploratory methods, 85–86 hyperspectral data, unfolding and refolding, 93–94 image-based approach, 94–98 nonlinear iterative partial least squares algorithm, 90–92 object-based approach, 98, 99 pixel-based approach, 98–102 projection method biplot, 89 bivariate space, 87 linear combinations, 89 loading matrix, 88 orthogonal space, 86–87 score and loading plot, 89 score matrix, 87–88 singular value decomposition method, 90, 92 texture analysis, 102–103 Principal component regression (PCR), 66–67, 93 Principal component spectra (PCS), 121 Probabilistic neuronal network (PNN), 132 Profiles extraction, script for, 148–149 Pseudomonas, 110, 169 Pseudomonas tolaasii, 212

281

PUFAs, see Polyunsaturated fatty acids (PUFAs) Push-broom hyperspectral imaging system, 10 Q Qualitative/quantitative modeling, definition of, 66 Quality index method (QIM), 163 Quality index scores (QIS), 163 R Radial basis function neural networks (RBF-NNs), 129–130 Raman calibration, 267 Raman chemical imaging (R-CI), 241, 270, 271 Raman effect, 264 Raman hyperspectral imaging system, 264–266 image acquisition, 268–269 instrumentation, 265–267 spatial calibration, 268 spectral calibration, 267 extraction, 268–269 Raman imaging instrumentation, 265–267 Raman imaging spectrograph, 267 Raman scattering, 264–265, 267 Raman spectroscopy, 265, 267, 271, 273 Random forest (RF), 112, 121 Rapid and nondestructive testing technique, 120, 189 Ratio of prediction to deviation (RPD), 69 Rayleigh scattering, 264, 267 RBF-NNs, see Radial basis function neural networks (RBF-NNs) R-CI, see Raman chemical imaging (R-CI) Receiver operator characteristics (ROC), 68 Red–Green–Blue (RGB), 186 Red meat, 63 color parameters in, 179–180 multivariate analysis in, 75 overview of, 175–177 prediction maps, 184 quality, 177–179 water holding capacity in, 180–188 Reference data, definition of, 66 Reflectance calibration, 13 Region of interest (ROI), 46, 110, 120, 138, 139, 202, 219, 256, 269 RF, see Random forest (RF) RGB, see Red–Green–Blue (RGB) RGB image generation, 148 RMS, see Root mean square (RMS)

282

Index

RMSE, see Root mean squared errors (RMSE) RMSEC, see Root mean squared errors for calibration (RMSEC) RMSECV, see Root mean square error of cross-validation (RMSECV) RMSEP, see Root mean square error of prediction (RMSEP) Robusta coffee, 249 ROC, see Receiver operator characteristics (ROC) ROI, see Region of interest (ROI) Root mean square (RMS), 24, 25 Root mean squared errors (RMSE), 267 Root mean squared errors for calibration (RMSEC), 68 Root mean squared errors of prediction estimated by cross-validation (RMSECV), 68, 69 Root mean square error of cross-validation (RMSECV), 212, 213 Root mean square error of prediction (RMSEP), 235, 248 RPD, see Ratio of prediction to deviation (RPD) Run length matrices, 50–51 S Salmonella, 216 Salmon fillets, 119, 165, 168, 169 Salt content, 167–168 Savitzky–Golay (SG) derivatives, 13 method, 50 Scanning based instrumentation, 265 Scanning frequency, for dark and white references, 24, 28–30 Scanning method, 268 Sceletium crassicaule, 233, 234 Sceletium tortuosum, 233, 234 Score matrix, 87–88 Script for HSI reading, 143–144 Scutellaria lateriflora, 235 SE, see Structuring element (SE) Seafood products of, 160, 169 support vector machines, 119–120 SEC, see Standard error of calibration (SEC) Seeds partial least squares regression, 112 support vector machines, 120–121 Self-modeling mixture analysis (SMA), 241 Self-organizing maps (SOMs), 130–131 Semitendinosus (ST), 75

Sensory evaluation, 245, 250 SEP, see Standard error of prediction (SEP) Sepedonium infections, 212 SER-CI, see Surface enhanced Raman chemical imaging (SER-CI) SERS, see Surface enhanced Raman Spectroscopy (SERS) SG, see Savitzky–Golay (SG) Short wave infrared (SWIR), 123, 234, 235, 241 Signal-to-noise (S/N) ratio, 22 SIMCA, see Spectral imaging and soft independent modeling of class analogy (SIMCA) Singular value decomposition (SVD), 90, 92 skullcap, see Scutellaria lateriflora Slice shear force (SSF), 110 SMA, see Self-modeling mixture analysis (SMA) Smoked salmon, 167–168 SNV, see Standard normal variate (SNV) Soft independent modeling of class analogy (SIMCA), 93, 121 Soluble solid content (SSC), 111, 196, 198, 199, 217, 251 SOMs, see Self-organizing maps (SOMs) SORS, see Spatially offset Raman spectroscopy (SORS) Soybean, 207–209 Spatial calibration, 268 Spatially offset Raman spectroscopy (SORS), 215 Spatial pre-processing, definition of, 66 Spatial–spectral graph method, 41 Spectral extraction, 268–269 Spectral extraction and treatment, 46–50 Spectral imaging, 7 Spectral imaging and soft independent modeling of class analogy (SIMCA), 168 Spectral pre-processing definition of, 66 multivariate modeling, 64 SpectrononPro software, 136 Spectroscopic measurements, 6 Spinach, 223–225 SSC, see Soluble solid content (SSC) SSF, see Slice shear force (SSF) ST, see Semitendinosus (ST) Standard error of calibration (SEC), 68 Standard error of prediction (SEP), 68, 69 Standard normal variate (SNV), 13, 49 Stephania tetrandra, 235

Index Structuring element (SE), 38 Superpixel segmentation method, 41 Supervised learning, definition of, 66 Supervised segmentation, 37–39 Support vector machine (SVM), 39, 67, 119, 250 fruits, 120 seafood, 119–120 seeds, 120–121 Support vector regression (SVR), 222, 248 Surface enhanced Raman chemical imaging (SER-CI), 271 Surface enhanced Raman Spectroscopy (SERS), 271 SVD, see Singular value decomposition (SVD) SVM, see Support vector machine (SVM) SVR, see Support vector regression (SVR) SWIR, see Short wave infrared (SWIR) System calibration, 267 T Teucrium canadense, 235 Teucrium chamaedrys, 235 Texture profile analysis (TPA), 134–135 TGA, see Total glycoalkaloid concentrations (TGA) Thiobarbituric acid (TBA) values, 166, 167 Three-dimensional (3D) data, 36, 176 hypercube, 61, 197 matrix, 135, 136 of pizza, 78 space, 37 Time series-hyperspectral imaging (TS-HSI), 111 Tomato, 215–217 Total glycoalkaloid concentrations (TGA), 215 Total viable count (TVC), 111, 169 Total volatile basic nitrogen (TVB-N), 167 TPA, see Texture profile analysis (TPA) Tree-based learning algorithms, 123 Triticum aestivum, 103 Triticum durum, 103 TS-HSI, see Time series-hyperspectral imaging (TS-HSI) TVB-N, see Total volatile basic nitrogen (TVB-N) TVC, see Total viable count (TVC) Two-dimensional (2-D) matrix, 70

283

U Ultrahigh-performance liquid chromatography (UHPLC), 233, 235 Ultraviolet (UV) region, 5 Unsupervised learning, definition of, 66 Unsupervised segmentation, 39–41 Ustilaginoidea virens, 121 V Vacuum-packed chilled smoked salmon, 39, 163 Vegetable(s) broccoli, 226 cucumber, 217–221 lettuce, 221–223 linear discriminant analysis, 116–117 mushroom, 209–213 onion, 226 peppers, 225–226 potato, 213–215 soybean, 207–209 spinach, 223–225 tomato, 215–217 Vibrational spectroscopy methods, 5, 6 Visible near-infrared (VNIR) region, 77–79, 119, 120, 123 spectroscopy, 77, 123, 164, 178–180, 198, 213 VIS/NIR HSI, 199, 251 W Warner–Bratzler shear force (WBSF), 76–77, 119, 164, 184 Water holding capacity (WHC), 110, 165, 178, 180, 181 in adulteration detection lamb meat, 183–185 minced beef, 187–188 minced lamb, 185–187 minced meat, 185 Watershed method, 40 Wavelet texture analysis, 115 Wheat, linear discriminant analysis, 115–116 Whisk-broom approach, 11 White button mushroom, 209–213 Whitefish fillets, 165 Wide-field imaging, 265 Winning neuron (WN), 131 Wort, 247

Taylor & Francis eBooks www.taylorfrancis.com A single destination for eBooks from Taylor & Francis with increased functionality and an improved user experience to meet the needs of our customers. 90,000+ eBooks of award-winning academic content in Humanities, Social Science, Science, Technology, Engineering, and Medical written by a global network of editors and authors.

TAYLOR & FRANCIS EBOOKS OFFERS: A streamlined experience for our library customers

A single point of discovery for all of our eBook content

Improved search and discovery of content at both book and chapter level

REQUEST A FREE TRIAL [email protected]