Selective Maintenance Modelling and Optimization: Basic Methods and Some Recent Advances 3031173228, 9783031173226

Table of contents :
Preface
Contents
1 Introduction
1.1 Overview of Maintenance Optimization
1.1.1 Paradigms of Maintenance Optimization
1.1.2 System Degradation Characteristics
1.1.3 Maintenance Efficiencies
1.1.4 Inspection Strategies
1.1.5 Multi-component Systems
1.1.6 Maintenance Objectives
1.1.7 Optimization Algorithms
1.2 Selective Maintenance
1.2.1 System Modelling
1.2.2 Efficiency of Maintenance Actions
1.2.3 Constraints of Maintenance Resources
1.2.4 Mission Characteristics and Operating Environment
1.2.5 Solution Algorithms
References
2 Basic Selective Maintenance Model
2.1 Introduction
2.2 Problem Statements and Model Assumptions
2.2.1 Problem Statements
2.2.2 Model Assumptions
2.3 Decision Variables
2.4 Probability of a System Successfully Completing a Mission
2.4.1 Survival Probability of a Component
2.4.2 Typical Lifetime Distribution
2.4.3 Probability of a System Successfully Completing the Next Mission
2.5 Selective Maintenance Modelling
2.5.1 Constraints of Selective Maintenance Problems
2.5.2 Optimization Models of Selective Maintenance Problems
2.6 Illustrative Example
2.7 Closure
References
3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance
3.1 Introduction
3.2 Problem Statements and Model Assumptions
3.3 Imperfect Maintenance and Its Cost
3.4 Probability of a System Successfully Completing a Mission
3.5 Selective Maintenance Modelling
3.6 Illustrative Examples
3.6.1 A Three-Component System
3.6.2 A Multi-state Coal Transportation System
3.7 Closure
References
4 Selective Maintenance for Multi-state Systems with Loading Strategy
4.1 Introduction
4.2 Problem Statements and Model Assumptions
4.3 Imperfect Maintenance
4.3.1 Failure Rate with Load Distribution
4.3.2 Imperfect Maintenance Modelling
4.4 Probability of a System Successfully Completing a Mission
4.5 Selective Maintenance Modelling
4.6 Illustrative Example
4.7 Closure
References
5 Selective Maintenance under Stochastic Time Durations of Breaks and Maintenance Actions
5.1 Introduction
5.2 Problem Statements and Model Assumptions
5.3 Probability of a System Successfully Completing a Mission
5.3.1 Probability Distribution of the Number of Completed Maintenance Actions
5.3.2 Saddlepoint Approximation
5.3.3 Probability of a System Successfully Completing the Next Mission
5.4 Selective Maintenance Optimization
5.4.1 Selective Maintenance Optimization Modelling
5.4.2 Tailored Ant Colony Optimization Algorithm
5.5 Illustrative Examples
5.5.1 A Four-Component System
5.5.2 A Multi-state Coal Transportation System
5.6 Closure
References
6 Robust Selective Maintenance under Imperfect Observations
6.1 Introduction
6.2 Problem Statements and Model Assumptions
6.3 Imperfect Maintenance Model
6.4 Survival Probability of a Component under Imperfect Observations
6.4.1 State and Effective Age under Imperfect Observations
6.4.2 State and Effective Age after Maintenance
6.4.3 Survival Probability of a Component
6.5 Probability of a System Successfully Completing a Mission
6.6 Robust Selective Maintenance Modelling
6.7 Illustrative Examples
6.7.1 A Five-Component System
6.7.2 A Coal Transportation System
6.8 Closure
References
7 Selective Maintenance and Inspection Optimization for Partially Observable Systems
7.1 Introduction
7.2 Problem Statements and Model Assumptions
7.2.1 Problem Statement
7.2.2 Imperfect Maintenance Model
7.2.3 Imperfect Inspection Model
7.3 Joint Selective Maintenance and Inspection Optimization
7.3.1 Probability of a System Successfully Completing a Mission
7.3.2 Mixed Observability Markov Decision Process
7.3.3 Dynamic Programming Algorithm
7.3.4 Deep Reinforcement Learning Algorithm
7.4 Illustrative Examples
7.4.1 A Five-Component System
7.4.2 A Multi-state Coal Transportation System
7.5 Closure
References
8 Selective Maintenance for Systems Operating Multiple Consecutive Missions
8.1 Introduction
8.2 Problem Statements and Model Assumptions
8.3 Imperfect Maintenance Model
8.4 Survival Probability of a Component
8.5 Probability of a System Successfully Completing Missions
8.5.1 Probability of a Component Successfully Completing Future Missions
8.5.2 Probability of a System Successfully Completing Future Missions
8.6 Selective Maintenance Optimization
8.6.1 Selective Maintenance Optimization Model
8.6.2 Customized Simulated Annealing-Based Genetic Algorithm
8.7 Illustrative Examples
8.7.1 A Five-Component System
8.7.2 A Coal Transportation System
8.8 Closure
References
9 Dynamic Selective Maintenance for Multi-state Systems Operating Multiple Consecutive Missions
9.1 Introduction
9.2 Problem Statements and Model Assumptions
9.3 Imperfect Maintenance Model
9.4 Dynamic Selective Maintenance Modelling
9.4.1 States and Effective Ages of Components at the End of a Mission
9.4.2 Probability of System Successfully Completing a Mission
9.4.3 Markov Decision Process Formulation
9.5 Customized Deep Reinforcement Learning Method
9.5.1 Actor-Critic Framework
9.5.2 Agent Training: Experience Replay and Target Network
9.6 Illustrative Examples
9.6.1 A Four-Component System
9.6.2 A Multi-state Coal Transportation System
9.7 Closure
References
Appendix Appendix Parameters for the Multi-state Coal Transportation System in Chapter 7

Citation preview

Springer Series in Reliability Engineering

Yu Liu Hong-Zhong Huang Tao Jiang

Selective Maintenance Modelling and Optimization Basic Methods and Some Recent Advances

Springer Series in Reliability Engineering Series Editor Hoang Pham, Department of Industrial and Systems Engineering, Rutgers University, Piscataway, NJ, USA

Today’s modern systems have become increasingly complex to design and build, while the demand for reliability and cost effective development continues. Reliability is one of the most important attributes in all these systems, including aerospace applications, real-time control, medical applications, defense systems, human decision-making, and home-security products. Growing international competition has increased the need for all designers, managers, practitioners, scientists and engineers to ensure a level of reliability of their product before release at the lowest cost. The interest in reliability has been growing in recent years and this trend will continue during the next decade and beyond. The Springer Series in Reliability Engineering publishes books, monographs and edited volumes in important areas of current theoretical research development in reliability and in areas that attempt to bridge the gap between theory and application in areas of interest to practitioners in industry, laboratories, business, and government. Now with 100 volumes! **Indexed in Scopus and EI Compendex** Interested authors should contact the series editor, Hoang Pham, Department of Industrial and Systems Engineering, Rutgers University, Piscataway, NJ 08854, USA. Email: [email protected], or Anthony Doyle, Executive Editor, Springer, London. Email: [email protected].

Yu Liu · Hong-Zhong Huang · Tao Jiang

Selective Maintenance Modelling and Optimization Basic Methods and Some Recent Advances

Yu Liu Center for System Reliability and Safety University of Electronic Science and Technology of China Chengdu, Sichuan, China

Hong-Zhong Huang Center for System Reliability and Safety University of Electronic Science and Technology of China Chengdu, Sichuan, China

Tao Jiang Center for System Reliability and Safety University of Electronic Science and Technology of China Chengdu, Sichuan, China

ISSN 1614-7839 ISSN 2196-999X (electronic) Springer Series in Reliability Engineering ISBN 978-3-031-17322-6 ISBN 978-3-031-17323-3 (eBook) https://doi.org/10.1007/978-3-031-17323-3 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

Reliability has become a crucial characteristic of advance engineered systems as systems with poor reliability suffer from a great amount of lifecycle cost and potential risk of failures. Maintenance, involving both corrective and preventive actions, is an effective way to retain a system in or restore it to an acceptable operating condition, and it has been extensively implemented in industrial applications. Examples of maintenance activities for engineered systems are oil change of rotating systems, rotor balance of mechanical systems, shaft/coupling alignment, filter replacement, corroded components coating, and so forth. It is noted that inappropriate maintenance scheme may not guarantee the reduction of operation cost and the fulfillment of reliability target, and systems are over-maintained or under-maintained during its operation stage. The maintenance models and optimization have been, therefore, intensively studied in the past decade with the purpose of minimizing maintenance cost and/or maximizing reliability or availability of a specific system. The paradigm of maintenance strategy has shifted from corrective maintenance to preventive maintenance, and then to condition-based maintenance and predictive maintenance nowadays. Selective maintenance optimization, as a specific condition-based maintenance problem, was firstly presented by Rice, Cassady, and Nachlas in the 7th Industrial Engineering Research Conference. In selective maintenance optimization, a system intends to perform successive assigned missions with a break between two adjacent missions, and maintenance actions can be executed in breaks to ensure the success of the subsequent missions. However, due to the limited maintenance resources, such as time and budget, it is impossible to carry out all the desired maintenance actions for aged and failed components. Alternatively, a subset of maintenance actions has to be selected from the set of all the optional maintenance actions, so as to maximize the success of the future missions. The preliminary model for selective maintenance of series–parallel systems with independent and identical copies of a component was generalized to a basic framework by Cassady and co-authors in the follow-up research works. As selective maintenance optimization exactly matches up with the many industrial and military scenarios where only a limited amount of maintenance

v

vi

Preface

resources have to be allocated among components of a system, it has been extensively studied in the past decade from various angles and implemented in a diversity of industrial applications. Research articles on this subject are continuously being published in journals and conference proceedings. Nevertheless, to the best of our knowledge, the subject has never been adequately or systematically reported in reliability book. The increased and sustained interest in this subject drives us to publish this book. This book systematically introduces the basic selective maintenance optimization model. It is, to a large extent, a collection of our recent research advances on selective maintenance optimization from the Center for System Reliability and Safety at the University of Electronic Science and Technology of China. The layout of this book is as following: Chapter 1 introduces the role of maintenance optimization in lifecycle management of engineering assets and gives an overview picture of research topics in maintenance optimization. It is followed by a systematical literature review on the existing research efforts on selective maintenance optimization. Chapter 2 offers an introduction to the basic mathematical model of selective maintenance problem, in which both a system and its components are assumed to be binary-state. Three selective maintenance models with distinct objectives functions and constraints are formulated. Chapter 3 discusses the selective maintenance optimization for multi-state systems with binary-capacitated components. The Kijima type II age reduction model, serving as a specific imperfect maintenance model, is incorporated in the selective maintenance optimization. The universal generating function is utilized to evaluate the probability of a system successfully completing the next mission. The genetic algorithm is introduced to resolve the resulting optimization model. Chapter 4 focuses on the selective maintenance optimization for multi-state systems with the load sharing mechanism. A joint optimization model is formulated to simultaneously optimize the load distribution and the allocation of the limited maintenance budget among components. The genetic algorithm is employed to solve the optimization problem. Chapter 5 discusses selective maintenance optimization under stochastic time durations of breaks and maintenance actions. The distribution of the number of completed maintenance actions in a break with time duration uncertainty is evaluated by using the saddlepoint approximation. A tailored ant colony optimization algorithm is developed to solve the resulting combinational optimization problem in the cases of large-scale systems. Chapter 6 presents a robust selective maintenance optimization model to treat the uncertainty produced by imperfect observations. A multi-objective optimization model is formulated with the aims of maximizing the expectation and simultaneously minimizing the variance of a system successfully completing the next mission. Chapter 7 takes account of uncertainties from both maintenance and inspection and introduces a joint selective maintenance and inspection optimization model. A finitehorizon mixed observability Markov decision process is formulated as the remaining resource is fully observable and the component states are partially observable. The

Preface

vii

dynamic programing and the deep reinforcement learning algorithm are implemented to resolve small-scale problems and large-scale problems, respectively. Chapter 8 discusses selective maintenance optimization for systems executing multiple consecutive missions. The uncertainties associated with the time duration of each future mission and the working time of each component in each future mission are addressed. The selective maintenance problem is formulated as a maxmin optimization model, and it is resolved by a customized simulated annealingbased genetic algorithm. Chapter 9 introduces a dynamic selective maintenance for multi-state systems operating multiple consecutive missions. The resulting sequential decision problem is formulated as a Markov decision process with a mixed integer-discrete-continuous state space. A deep reinforcement learning method is customized based on the actorcritic framework, and a postprocess is utilized to search for the optimal maintenance actions in a constrained large-scale action space. The target audience of this book is undergraduate and graduate students, reliability practitioners, and researchers. The readers should have background in basic probability theory, stochastic models, and optimization algorithms. The book offers a great mount of knowledge and insights on system maintenance modelling methods and optimization algorithms, with which readers can deal with many real-world engineering cases. This book is a collection of materials developed in the dissertations and journal/conference papers of several former and current graduate students from the Center for System Reliability and Safety at the University of Electronic Science and Technology of China. The majority of the chapters have been developed based on the dissertations and research works of Dr. Tao Jiang (M.Sc. student from 2014–2017 and Ph.D. student from 2017–2022), Dr. Yiming Chen (Ph.D. student from 2015–2022), Mr. Jian Gao (M.Sc. student from 2019–2022), Mr. Chujie Chen (M.Sc. student from 2012–2015), and Mr. Qin Zhang (M.Sc. student from 2020–2022). The book was edited with the additional assistance of Dr. Tangfan Xiahou (Ph.D. student from 2018–2022) and Dr. Mingang Yin (Post-doctoral research fellow from 2021–2023). We would like to express our sincere gratitude and appreciation to researchers and friends who have discussed with the concepts and models of this book, or have coauthored with us on some topics of this book. To name a few, Prof. Ming J. Zuo at University of Alberta, Prof. Wei Chen at Northwestern University, Prof. Min Xie at City University of Hong Kong, Prof. Lirong Cui at Qingdao University, Prof. Liudong Xing at University of Massachusetts-Dartmouth, Dr. Gregory Levitin at Israel Electric Corporation, Prof. Yi-Kuei Lin at Taiwan Yang Ming Chiao Tung, Prof. Tongdan Jin at Texas State University, Prof. Haitao Liao at University of Arkansas, Prof. Zhisheng Ye at National University of Singapore, Prof. Zhiguo Zeng at CentraleSupélec—Université Paris-Saclay, Prof. Yisha Xiang at Texas Tech University, Prof. Yuchang Mo at Huaqiao University, and Prof. Hui Xiao at Southwestern University of Finance and Economics. Last but not least, we would like to thank Prof. Hoang Pham at Rutgers University, who gave a great support to the publication of this book. It is also indeed our pleasure working with Mr. Kavitha Sathish and the Springer editorial team.

viii

Preface

The research works in this book received financial support from the National Natural Science Foundation of China under contact numbers 71922006 and 71771039. Chengdu, Sichuan, China June 2022

Yu Liu Hong-Zhong Huang Tao Jiang

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Overview of Maintenance Optimization . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Paradigms of Maintenance Optimization . . . . . . . . . . . . . . . . 1.1.2 System Degradation Characteristics . . . . . . . . . . . . . . . . . . . . . 1.1.3 Maintenance Efficiencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.4 Inspection Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.5 Multi-component Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.6 Maintenance Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.7 Optimization Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Selective Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 System Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Efficiency of Maintenance Actions . . . . . . . . . . . . . . . . . . . . . 1.2.3 Constraints of Maintenance Resources . . . . . . . . . . . . . . . . . . 1.2.4 Mission Characteristics and Operating Environment . . . . . . . 1.2.5 Solution Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 3 5 6 8 9 11 12 13 14 17 19 20 22 23

2 Basic Selective Maintenance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Problem Statements and Model Assumptions . . . . . . . . . . . . . . . . . . . 2.2.1 Problem Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Model Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Decision Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Probability of a System Successfully Completing a Mission . . . . . . 2.4.1 Survival Probability of a Component . . . . . . . . . . . . . . . . . . . . 2.4.2 Typical Lifetime Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Probability of a System Successfully Completing the Next Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Selective Maintenance Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Constraints of Selective Maintenance Problems . . . . . . . . . . .

31 31 33 33 33 34 35 35 36 36 38 38

ix

x

Contents

2.5.2 Optimization Models of Selective Maintenance Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Illustrative Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

39 40 43 43

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Problem Statements and Model Assumptions . . . . . . . . . . . . . . . . . . . 3.3 Imperfect Maintenance and Its Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Probability of a System Successfully Completing a Mission . . . . . . 3.5 Selective Maintenance Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 A Three-Component System . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.2 A Multi-state Coal Transportation System . . . . . . . . . . . . . . . 3.7 Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45 45 47 48 51 53 54 54 57 62 62

4 Selective Maintenance for Multi-state Systems with Loading Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Problem Statements and Model Assumptions . . . . . . . . . . . . . . . . . . . 4.3 Imperfect Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Failure Rate with Load Distribution . . . . . . . . . . . . . . . . . . . . . 4.3.2 Imperfect Maintenance Modelling . . . . . . . . . . . . . . . . . . . . . . 4.4 Probability of a System Successfully Completing a Mission . . . . . . 4.5 Selective Maintenance Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Illustrative Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65 65 66 67 67 68 69 71 72 74 74

5 Selective Maintenance under Stochastic Time Durations of Breaks and Maintenance Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Problem Statements and Model Assumptions . . . . . . . . . . . . . . . . . . . 5.3 Probability of a System Successfully Completing a Mission . . . . . . 5.3.1 Probability Distribution of the Number of Completed Maintenance Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Saddlepoint Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 Probability of a System Successfully Completing the Next Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Selective Maintenance Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Selective Maintenance Optimization Modelling . . . . . . . . . . 5.4.2 Tailored Ant Colony Optimization Algorithm . . . . . . . . . . . . 5.5 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

77 77 78 81 81 83 85 86 86 87 90

Contents

xi

5.5.1 A Four-Component System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.2 A Multi-state Coal Transportation System . . . . . . . . . . . . . . . 5.6 Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

90 92 98 99

6 Robust Selective Maintenance under Imperfect Observations . . . . . . . 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Problem Statements and Model Assumptions . . . . . . . . . . . . . . . . . . . 6.3 Imperfect Maintenance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Survival Probability of a Component under Imperfect Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 State and Effective Age under Imperfect Observations . . . . . 6.4.2 State and Effective Age after Maintenance . . . . . . . . . . . . . . . 6.4.3 Survival Probability of a Component . . . . . . . . . . . . . . . . . . . . 6.5 Probability of a System Successfully Completing a Mission . . . . . . 6.6 Robust Selective Maintenance Modelling . . . . . . . . . . . . . . . . . . . . . . 6.7 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7.1 A Five-Component System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7.2 A Coal Transportation System . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

101 101 102 104

7 Selective Maintenance and Inspection Optimization for Partially Observable Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Problem Statements and Model Assumptions . . . . . . . . . . . . . . . . . . . 7.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.2 Imperfect Maintenance Model . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.3 Imperfect Inspection Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Joint Selective Maintenance and Inspection Optimization . . . . . . . . 7.3.1 Probability of a System Successfully Completing a Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2 Mixed Observability Markov Decision Process . . . . . . . . . . . 7.3.3 Dynamic Programming Algorithm . . . . . . . . . . . . . . . . . . . . . . 7.3.4 Deep Reinforcement Learning Algorithm . . . . . . . . . . . . . . . . 7.4 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 A Five-Component System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.2 A Multi-state Coal Transportation System . . . . . . . . . . . . . . . 7.5 Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Selective Maintenance for Systems Operating Multiple Consecutive Missions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Problem Statements and Model Assumptions . . . . . . . . . . . . . . . . . . . 8.3 Imperfect Maintenance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

105 105 108 108 109 110 113 113 118 119 120 123 123 124 124 126 127 128 128 129 132 133 135 135 140 142 144 147 147 148 149

xii

Contents

8.4 Survival Probability of a Component . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Probability of a System Successfully Completing Missions . . . . . . . 8.5.1 Probability of a Component Successfully Completing Future Missions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.2 Probability of a System Successfully Completing Future Missions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6 Selective Maintenance Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.1 Selective Maintenance Optimization Model . . . . . . . . . . . . . . 8.6.2 Customized Simulated Annealing-Based Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7.1 A Five-Component System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7.2 A Coal Transportation System . . . . . . . . . . . . . . . . . . . . . . . . . 8.8 Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Dynamic Selective Maintenance for Multi-state Systems Operating Multiple Consecutive Missions . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Problem Statements and Model Assumptions . . . . . . . . . . . . . . . . . . . 9.3 Imperfect Maintenance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Dynamic Selective Maintenance Modelling . . . . . . . . . . . . . . . . . . . . 9.4.1 States and Effective Ages of Components at the End of a Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.2 Probability of System Successfully Completing a Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.3 Markov Decision Process Formulation . . . . . . . . . . . . . . . . . . 9.5 Customized Deep Reinforcement Learning Method . . . . . . . . . . . . . 9.5.1 Actor-Critic Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.2 Agent Training: Experience Replay and Target Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.1 A Four-Component System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.2 A Multi-state Coal Transportation System . . . . . . . . . . . . . . . 9.7 Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

150 153 153 153 154 154 156 157 157 162 164 165 167 167 168 169 171 171 173 174 176 176 179 181 181 184 187 187

Appendix: Parameters for the Multi-state Coal Transportation System in Chapter 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Chapter 1

Introduction

1.1 Overview of Maintenance Optimization Reliability refers to the ability of a system or component to perform its intended functions under the stated conditions for a specified period of time [36]. Advanced engineered systems are designed for greater size, higher precision, and more complex functionality, along with the involvement of state-of-the-art artificial intelligent techniques. The reliability of these systems (e.g., nuclear plants, wind turbines, aircrafts, power systems, and machining centers) has been intensively studied across their entire lifecycle. Failure of these advanced systems often results in unexpected production delays and/or significant economic losses and can even cause severe threats to human life. Examples of recent major accidents, as shown in Fig. 1.1, include the Fukushima Daiichi nuclear power plant disaster in 2011, Samsung Galaxy Note 7 battery explosion in 2016, Italy Apulia train crash in 2016, and Boeing 737 Max crash in 2019. However, the causes of failures are diverse, ranging from the system design and manufacturing stages to the system operation and maintenance stages [121]. Industrial practitioners strive to minimize the occurrence and consequences of failures. Consequently, many attempts have been made by both industry and academia in the past decades to understand why and with which patterns system failures occur [143]. Figure 1.2 shows a typical pattern of the hazard rate function of engineered systems, which is well-known as a bathtub curve because of its shape. Systems with such a hazard rate function experience a decreasing failure rate at the early stage of their lifecycle (also called infant mortality), followed by a nearly constant failure rate stage (also called useful life) and by an increasing failure rate stage (also called wear-out). In the infant mortality region, the failure rate of systems is high because of defective components, manufacturing defects, and poor quality control. By removing defective components, the failure rate function of the system will continuously fall, and it will reach the useful life region. The failure rate function in the useful life region is fairly constant, and system failures are caused randomly by environmental loads, human errors, and chance events. Engineered systems are expected to operate for as long as © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_1

1

2

1 Introduction

Fig. 1.1 Examples of major accidents in recent years

Fig. 1.2 Bathtub curve

possible in this region. Eventually, the failure rate function of the systems rises again in the wear-out region as the components in the systems start to deteriorate. Typical causes of component deteriorations include fatigue, corrosion, friction, and aging. By replacing or recovering the wear-out components in a timely manner, the failure rate function of the system will decline immediately, leading to a lower probability of failure in the future than that of the case without replacement or recovery. Such a system that can be repaired in its lifecycle is called a repairable system.

1.1 Overview of Maintenance Optimization

3

Fig. 1.3 Maintenance decision-making

Component replacement is a typical maintenance activity in industrial practice. Further, all activities aimed at maintaining a system in or restoring it to the physical state considered necessary for the fulfillment of its production function can be regarded as maintenance activities [43]. Examples of maintenance activities for engineered systems include oil change in rotating systems, rotor balance of mechanical systems, shaft/coupling alignment, filter replacement, and coating of corroded components. Despite that maintenance activities can prolong the usage lifetime and maintain the performance capacity of systems, unoptimized maintenance planning may not guarantee a reduction in operation costs and the fulfillment of reliability targets, and systems are over-maintained or under-maintained during the operation stage. By constructing a proper maintenance decision model, the maintenance plan for engineered systems can be optimized to minimize maintenance costs and/or maximize the reliability or availability of a specific system. An example of maintenance decision making is delineated in Fig. 1.3. With an increase in the preventive maintenance (PM) frequency, the cost associated with corrective maintenance (CM) declines, whereas that of PM increases. From the perspective of the total maintenance cost, an optimal PM frequency that possesses a minimal total maintenance cost is readily found. Such an optimal maintenance strategy can be identified by formulating a maintenance decision model and resolving it using an appropriate optimization algorithm.

1.1.1 Paradigms of Maintenance Optimization Looking back on the history of maintenance optimization, the paradigm of maintenance strategy has shifted from CM to PM and then to condition-based and predictive maintenance. The following are the definitions of these maintenance paradigms: CM is also called reactive maintenance or run-to-failure strategy; that is, maintenance activities will be confined to reactive tasks of repair actions or component replacement upon the failure of components or systems. Examples of systems with CM strategies include light bulb replacement and computer repair. Non-essential items and systems in which failure causes marginal economic loss can be replaced

4

1 Introduction

or repaired when they run to failure. The advantages of CM include low cost associated with monitoring, minimal planning requirements, and simplicity of processing. Conversely, the disadvantages of CM are unpredictable failures and unscheduled downtime, shorter lifecycle, and potential for high long-term cost. PM (also known as preventative maintenance) is conducted regularly on a population of identical engineered systems to reduce their likelihood of failure. It can be considered a proactive maintenance paradigm and further divided into two types: time-based PM and usage-based PM. In the former case, PM is executed at a fixed calendar time instant. For example, this could be a weekly or monthly maintenance routine for the production lines. In the latter case, PM is triggered after a set number of production cycles, hours in use, or even distance travelled. For example, the oil filter of a vehicle is periodically replaced every 10,000 mileages. Improved reliability, reduced cost associated with unexpected failures, and less disruption or unscheduled downtime are key features of PM. However, such a maintenance paradigm incurs additional costs and personnel for executing PM actions, and a poorly prepared PM plan may lead to a specific individual system from a population being over-maintained or under-maintained. Condition-based maintenance (CBM) is also a proactive maintenance paradigm. Using advanced sensing techniques, the actual health status of engineered systems can be monitored to facilitate a timely and cost-efficient proactive maintenance plan for each individual system. CBM is triggered when certain monitored indicators show signs of decreasing performance or upcoming failure. The indicators could be the vibration signals of rotating machines, temperatures, and debris of lubricating oil. In specific engineering applications, condition monitoring data can be collected continuously or periodically at certain time intervals [49]. As CBM leverages the monitoring data of each individual system, unnecessary maintenance tasks can be avoided unlike either time-based or usage-based PM. Meanwhile, the repair cost and sudden downtime caused by random failures can be reduced further. However, equipping condition monitoring incurs additional upfront system setup costs. In most cases, these costs are offset by the potential costs of unnecessary maintenance or unexpected failures. Predictive maintenance (PdM) is an emerging maintenance paradigm in recent years, and it is flourishing very fast with the development of predictive techniques. Slightly different from CBM, PdM not only monitors the actual health status of engineered systems using sensors or inspection instruments, but also predicts the evolution of the health status and remaining useful life of each individual system using historical monitoring data and future mission profiles [49]. Such a prediction activity that facilitates PdM is termed a prognosis in the reliability community [113]. Physical model-based and data-driven methods are two major research lines of prognosis. Recently, machine learning and artificial intelligence techniques, such as support vector machines, deep neural networks, and generative adversarial networks, have been intensively implemented as data-driven tools for prognosis [70, 144]. With progressively updated prognosis results, the PdM strategy is dynamically scheduled for each individual system. Because maintenance is only executed as required for each system when failure is imminent, PdM is often more cost-effective than the

1.1 Overview of Maintenance Optimization

5

other maintenance paradigms. Similar to CBM, PdM requires condition monitoring devices for data acquisition. However, its effectiveness relies heavily on prognosis accuracy [8].

1.1.2 System Degradation Characteristics The health status of an engineered system inevitably deteriorates over time. To schedule effective maintenance planning, decision makers should first mathematically characterize the deterioration behaviors of a system. Throughout the entire history of maintenance optimization, the most straightforward and simplest method of characterizing the failure behavior of a system is to assume an appropriate lifetime distribution (e.g., exponential and Weibull distributions). However, the lifetime distribution of a repairable system is based on a coarse probabilistic model. It is more suitable to consider an underlying stochastic process to manifest a set of stages that the system experienced before completely failing. Owing to the development of advanced sensors and inspection techniques, the degradation trajectories of engineered systems can be recorded completely or partially. A plethora of degradation models have been developed over the past few decades to characterize deterioration behaviors in an effective and accurate manner. The degradation models of repairable systems can generally be categorized based on two categories of state spaces: discrete-state and continuous-state degradation models. For engineered systems in which health status can be easily distinguished into different distinct states, discrete-state degradation models can be utilized. Discrete-state degradation models can be further categorized into binary-state, three-state, and generalized multi-state models. Essentially, the lifetime distribution of a repairable system is a binary-state degradation model (perfectly functioning and completely failed). The three-state degradation model herein refers to the delay-time model wherein the deterioration process of a system is defined as a two-stage process [125]. The first stage is the normal operating stage from the new stage to the point where a hidden defect has been identified, whereas the second stage is defined as the failure delay time from the time of defect identification to failure [125]. Moreover, many engineered systems can govern multiple (more than three) distinct states. For concreteness, power systems can operate at different performance capacities. Another typical example is that the health status of a cutting tool can be roughly classified into “normal,” “moderately worn out,” “seriously worn out,” and “completely worn out.” The Markov property is a conventional and basic hypothesis in multi-state degradation models. Markov models, such as discrete-time and continuous-time (homogenous) Markov chains, and semi-Markov models are effective tools for characterizing the multi-state degradation trajectories of degradation systems. Additionally, the hidden Markov and hidden semi-Markov models are implemented if the uncertainty associated with inspections is considered. Based on whether the system can transit from its current state to a non-adjacent state, multi-state degradation models can be

6

1 Introduction

distinguished into progressive and non-progressive models. The delay-time model can also be extended to a generalized multi-state degradation model by considering defect severity. Typically, the deterioration of engineered systems is gradual over time and difficult to classify into multiple distinct states. In such cases, continuous-state degradation models should be used. Continuous-state degradation models in the context of maintenance optimization mainly include stochastic process-based models, typically Wiener, Gamma, and inverse Gaussian processes. The Wiener process is appropriate for characterizing non-monotonically increasing (or decreasing) degradation, such as the resistance of an electronic component and the capacity of batteries. Degradation in the form of cumulative damage, that is, monotonic increasing (or decreasing) degradation, can be modeled by both Gamma and inverse Gaussian processes, such as the wear process and fatigue crack propagation. Inverse Gaussian process-based degradation model has recently gained significant attention owing to their ability and flexibility in incorporating random effects and covariates [133]. In many real-world situations, degradation processes of engineered systems may be caused or indicated by various time-varying environmental factors and/or external shocks. Environmental factors include general external factors (e.g., temperature, vibration, load, and running speed) and internal factors (e.g., inherent deterioration mechanism and deterioration level). For example, the wear process of a cutting tool can be affected by running speed and material properties. Meanwhile, the proportional hazard model (PHM), as one of the most reported covariate-based models, has been extensively applied in reliability analysis and maintenance optimization owing to its flexibility and simplicity. The conventional PHM assumes that the hazard rate function comprises two multiplicative parts: a baseline hazard rate function and a function of covariates. Time-varying environmental factors per se can be easily incorporated into degradation processes. Various PHMs and their variants have been developed to accommodate various industrial scenarios. However, the influence of shocks can be versatile: (1) some parameters in a degradation model are related to the number of shocks, (2) an additional degradation increment is incurred directly by the occurrence of a shock, and (3) a shock can induce a system failure, which poses the case of competing failure.

1.1.3 Maintenance Efficiencies In maintenance optimization problems, various maintenance actions are optional to retain a deteriorated system or recover it to a better condition. Repair refers to actions carried out upon the failure of a system and thus corresponds to CM. Before implementing a maintenance action on a deteriorated system, decision makers must determine the degree to which the system can be restored by the action. Maintenance actions can be classified into the following five categories: Perfect maintenance or prefect repair can restore the condition of a system to “as good as new”; that is, the condition of aged or failed system is recovered to the status that is the same as a brand-new system. A typical example of perfect

1.1 Overview of Maintenance Optimization

7

maintenance/repair is the replacement of a deteriorated/failed system with a new one. Minimal repair can recover a failed system to the condition just before failure, i.e., “as bad as old.” The operation of repaired system continuous, while its failure intensity is the same as that prior to the failure. For example, changing a broken fan belt can be regarded as minimal repair because the failure intensity of the entire car remains unchanged. Imperfect maintenance or imperfect repair can restore a system to a condition somewhere between the extremes of “as good as new” and “as bad as old,” and the system after maintenance “looks” younger. For instance, tuning an engine can greatly improve its performance, but cannot make it as good as a brand-new one. Worse maintenance or worse repair will make the system condition worse than that before maintenance although it still sojourn in or is be recovered to its functioning state. For example, inappropriate maintenance actions may damage some parts of a system, leading to increased failure intensity. Worst maintenance or worst repair will break down a system. For example, the wrong operation of repair persons may destroy the system. The concepts of perfect and imperfect maintenance can be understood in detail. For a multi-component system, replacing a broken crucial component is perfect maintenance for the component, but imperfect maintenance for the entire system. The mathematical formulation of the degree to which an imperfect maintenance or repair action can rejuvenate a system is an important issue in maintenance optimization and remains of great interest to both industry and academia. Various models for imperfect maintenance have been developed from different perspectives to accommodate diverse engineering scenarios. They can be generally classified into the following categories: The ( p, q) rule is a sort of imperfect maintenance treatment, under which a system will return to its “as good as new” state with probability p and to its “as bad as old” state with probability q = 1 − p [87]. The variants of ( p, q) rule include the agedependent ( p(t), q(t)) rule and multiple ( p, q) rules for multi-component systems. In the context of multi-state systems, most reported studies assume that an imperfect maintenance action can restore a system to a certain better state with probability one or recover (retain) the system to other states (in the current state) with different probabilities [74]. The imperfect maintenance treatment for multi-state systems is inherently an extension of the ( p, q) rule. Virtual age models assume an age reduction mechanism by which an imperfect maintenance action can reduce the physical age of a repaired system (therein called virtual age or effective age) [137]. Kijima type-I and II models are two basic and popular virtual age models. In Kijima type-I model, the effective age is reduced by an amount proportional to the additional age accumulated since the last maintenance, whereas in Kijima type-II model, the effective age is reduced by an amount proportional to the effective age just before maintenance [66, 67]. Additionally, the (α, β) rule assumes that the lifetime of a system will be reduced to a fraction α (0 < α < 1) of that immediately preceding it after maintenance, and the lifetime thereby decreases as the number of maintenance increases [96, 123, 124].

8

1 Introduction

In the (α, β) rule, the interarrival times between successive maintenance constitute a quasi-renewal process if the maintenance time can be neglected [124]. Improvement factor models assume a failure intensity reduction mechanism by which an imperfect maintenance action can generate a reduction to the failure intensity of a repaired system [84]. The reduction in the failure intensity can be fixed or proportional and may depend on the system age and number of maintenances. Additionally, the improvement factor treatment can be extended to continuous degradation systems wherein reductions in both the degradation level and degradation rate can be considered [71, 137]. The improvement factor and virtual age models can also be understood from linear and nonlinear perspectives [126]. In the improvement factor models, the reduction mechanism of failure intensity leads to a linear relationship between the failure intensities before and after maintenance. The baseline failure rate function is typically nonlinear, whereas in virtual age models, the age reduction mechanism results in a nonlinear relationship between the failure intensities before and after maintenance. Linear and nonlinear models can be transformed into one another. Meanwhile, a hybrid model is then naturally generated by combining the linear and nonlinear models [126]. Many imperfect maintenance models have been developed to mathematically characterize the efficiencies of maintenance actions. The selection of an appropriate imperfect maintenance model is a challenging problem and may be applicationdependent. The model selection and model average approaches have also been addressed in the recent literature [76].

1.1.4 Inspection Strategies The effectiveness of maintenance strategies plays a vital role in enhancing system reliability during the operation phase. PM refers to maintenance performed on a functioning, but aged, system to avoid system failure. Traditional time-based PM models (e.g., age replacement, block replacement, and periodic maintenance models) were formulated based on the elapsed time, which were developed using probabilistic models of system degradation [3]. However, owing to the development of advanced sensor and inspection techniques, CBM, which combines observations from condition monitoring and the system reliability model, can derive a more effective maintenance strategy. In CBM, maintenance actions are performed only when the observed information indicates that the system or components have severely deteriorated. Therefore, unnecessary maintenance actions and associated risks can be reduced [53]. From the inspection accuracy perspective, inspection actions can be categorized as perfect and imperfect inspections. Perfect inspection can fully reveal the system state, whereas imperfect inspection provides only partial information regarding the system state. Based on the inspection activity frequency, inspection activities can be generally classified into three categories as follows:

1.1 Overview of Maintenance Optimization

9

Continuous inspection can constantly monitor system condition and trigger warning signals for PM when the system severely deteriorates. Continuous inspection can provide real-time observation data on system conditions and reduce unnecessary maintenance actions. However, continuous inspection is often costly and may not be implemented owing to limited monitoring techniques or system design constraints. Periodic inspection periodically inspects a system with a deterministic time interval. Unlike continuous inspection, periodic inspection is more cost-effective. However, some warning signals may not be detected in the interval between two adjacent inspections by discrete periodic inspection, leading to the failure of the system with a high failure cost. Non-periodic inspection inspects the system condition with discrete time-varying intervals. In some industrial scenarios, inspecting the system periodically may not be practical because of the high inspection costs. Hence, a more cost-effective irregular inspection frequency is adopted. The more aged the system, the shorter the time interval between two adjacent inspections. Similar to periodic inspection, non-periodic inspection is also associated with a high failure cost owing to sudden failures that are not detected earlier. As inspections may share the same resources with maintenance activities (e.g., time, budget, and manpower), joint inspection and maintenance policy optimization have also been investigated, which can be generally classified into three categories: Maintenance policy optimization with fixed inspection policy assumes that the inspection frequency is continuous or periodic, and the interval of the periodic inspection and the accuracy of inspections are pre-specified [17, 88]. Maintenance policy optimization mainly determines the PM threshold for replacement or imperfect maintenance. Moreover, some studies have mapped inspection information to maintenance actions directly [68]. Selective maintenance optimization is a typical CBM method that assumes that all components are perfectly inspected before maintenance decision making under limited maintenance resources [75]. Joint optimization of inspection interval and maintenance policy assumes that the inspection frequency is periodic or non-periodic, and the accuracy of inspection actions is deterministic. Further, the inspection interval and next inspection time must be optimized for periodic and non-periodic inspection strategies, respectively [16]. Joint optimization of imperfect inspection and maintenance policy assumes that the inspection frequency is periodic, while the inspection should be jointly optimized with the maintenance policy. In each decision epoch, decision makers must select imperfect inspection actions (including perfect inspection as a special case) and maintenance actions based on observations from inspections [25, 94].

1.1.5 Multi-component Systems Engineering systems typically comprise numerous interacting components. Maintenance optimization of multi-component systems is more challenging than that of single-component systems owing to complex interactions among components.

10

1 Introduction

On the one hand, the system state depends not only on the state of its components but also on system configurations (e.g., series, parallel, series-parallel, and k-out-of-n). System configurations have a significant impact on the maintenance strategy. On the other hand, inter-component dependency leads to a sophisticated maintenance decision among components. In general, three types of dependencies among the components are encountered in maintenance optimization: economic dependence, structural dependence, and stochastic dependence [3, 122]. Recently, another type of dependency, namely, resource dependency, was introduced as a new perspective [53, 55]. Economic dependence is the most common type of dependencies in maintenance optimization. It implies that maintaining multiple components simultaneously is more economical than maintaining them separately [56]. Maintaining systems with economic dependence will incur a common system-level cost, referred to as the setup cost, due to mobilizing repair crews, safety provisions, disassembling machines, and downtime losses. The setup cost was shared by simultaneously conducting all maintenance activities. Therefore, such a sharing strategy can save considerable costs by maintaining several components together, particularly for a scenario with a high setup cost. Structural dependence concerns about the physical relation among components. This refers to the maintenance of a component that requires the maintenance or dismantlement of other components [31]. That is, to access a component for maintenance, obstructing components that block the disassembling path of the maintained component should be disassembled. Therefore, the dismantling sequence must be considered in the maintenance strategy, as an inappropriate dismantling sequence will incur additional time and cost. Stochastic dependence indicates that the deterioration process of a component is dependent on the state or deterioration process of other components. Stochastic dependence among components can be further classified into three major types [55]. The first type is failure-induced damage, which can be applied to cases wherein the failure of a component causes immediate damage or even an immediate failure of other components. The second type is load sharing, wherein multiple components share a total system load. In this case, failure or deterioration of a component may increase the load on other components, leading to the accelerated deterioration of other components. The third type of stochastic dependence is common-mode deterioration/failure, which can be applied in cases wherein several components fail or deteriorate simultaneously owing to similar operating conditions. Resource dependence implies that maintenance activities of multiple components are supported through a shared and limited set of maintenance resources (e.g., repairmen, facilities, or spares) [75, 78, 104, 120]. This type of dependence is logistical. Resource dependence leads to additional constraints for maintenance optimization. For example, maintenance actions can only be scheduled if the required resources are available. Therefore, maintenance optimization with resource dependence typically focuses on the system level rather than the component level.

1.1 Overview of Maintenance Optimization

11

1.1.6 Maintenance Objectives The objective function of maintenance optimization has a significant impact on the resulting maintenance policy. In the literature, maintenance optimization models with various objectives have been developed in accordance with the interests of decision makers. Some typical optimization objectives are as follows: Minimizing cost. Cost measures have been intensively utilized in maintenance optimization problems. They can be classified into two categories: (1) the cost associated with maintenance activities (e.g., the cost of maintenance materials and spare parts, labor cost, and inspection cost) and (2) the cost attributed to system failures (e.g., the cost caused by accidents and the cost of downtime during system failure). By executing maintenance activities, the costs associated with system failures can be reduced by sacrificing the cost of maintenance activities. Therefore, an optimal maintenance policy necessitates a trade-off between the two cost categories. Regarding the planning horizon, maintenance models can be classified into two categories: infinite horizon and finite horizon, which focus on long-term and shortterm maintenance planning [95, 122]. The cost rate, that is, the expected cost per unit time, and the (expected) total cost are the two typical objectives in maintenance optimization models over infinite and finite horizons, respectively. Additionally, by incorporating the value of time, the optimization objective with the concept of discounted cost (e.g., the total discounted cost over finite and infinite horizons) has been considered [23, 72, 128]. Maximizing availability. Many engineered systems (e.g., communication, manufacturing, and power systems) are required to operate with minimal system downtime. In such cases, maximizing system availability, which is the ratio of the system operating time to the planned horizon, is an appropriate objective to be optimized. Downtime caused by system failures can be reduced by maintenance activities, whereas excessive maintenance activities also increase system downtime. Therefore, downtime caused by maintenance and system failure should be balanced. Maximizing mission success probability. Many engineered systems and military equipment (e.g., space shuttles and airplanes) are intended to complete specific missions. It is expected to maximize the probability of mission success at a specific time. In most cases, maintenance optimization aims to maximize the mission success probability subject to the constraints of maintenance resources, such as maintenance cost, maintenance time, and repairsmen [12]. Multi-objective optimization. In practice, the ideal maintenance policy is to achieve the highest system reliability/availability with minimal maintenance resources. However, this is often unrealistic as different optimization objectives may conflict with one another. Therefore, in multi-objective maintenance models, the trade-offs between and among multiple optimization objectives should be properly addressed. In the literature, physical programming, non-dominated sorting genetic algorithms, multi-objective particle swarm optimization, and strength Pareto evolutionary algorithm II have been intensively implemented to address multi-objective maintenance optimization problems [109, 119].

12

1 Introduction

1.1.7 Optimization Algorithms Maintenance optimization is used to determine the optimal maintenance strategies to achieve maximum system availability or minimum cost. Serval optimization algorithms have been developed to resolve complex maintenance optimization problems. In general, optimization algorithms for maintenance problems can be classified into the following four categories. Exact algorithms. Maintenance optimization is often formulated as an integer programming or mixed-integer programming model. Many exact algorithms (e.g., the enumeration method [15], dynamic programming [86], and branch-andbound algorithms [37]) can be utilized to find the global optimum solution of maintenance optimization problems. However, owing to the increased complexity of maintenance models, the resulting optimization problems often cannot be converted into simple mathematical programming. Further, is difficult and time consuming to find an exact optimal solution. Heuristic algorithms. To overcome the aforementioned challenge of exact algorithms, many heuristic algorithms have been introduced to seek a near-optimal solution within a reasonable time. However, heuristic algorithms are typically problemdependent and developed based on cognitive experiences or intuitive arguments of the studied maintenance optimization. In the literature, the greedy heuristic [11], construction heuristic [79, 142], and priority-based heuristic [112] have been designed to address some specific maintenance optimization problems. Note that heuristics can also lead to poor performance, owing to cognitive biases of decision makers. Metaheuristic algorithms. Metaheuristic is a high-level problem-independent algorithmic framework that provides a set of guidelines or strategies for developing heuristic algorithms [114]. A large number of metaheuristic algorithms, including genetic algorithm (GA) [75], particle swarm optimization (PSO) [21], differential evolution algorithm (DE) [92], ant colony optimization (ACO) [77], tabu search (TS), and simulated annealing (SA) [39], have been developed for maintenance optimization problems. Artificial intelligence (AI)-based methods. With the rapid development of AI in recent years, various AI-based methods have been developed to resolve the sequential maintenance decision problems. By formulating sequential maintenance decision problems as a Markov decision process (MDP), many reinforcement learning approaches (e.g., the value iteration [9] policy iteration [10], and Q-learning [134]) have been developed to resolve the MDP in a computationally efficient manner. However, MDP often suffers from the “curse of dimensionality” as the dimensions of the state space and action space grow exponentially with increasing problem size. To overcome the “curse of dimensionality,” deep reinforcement learning (DRL) techniques (e.g., deep Q network algorithm (DQN) [138], deep deterministic policy gradient (DDPG) [24], and proximal policy optimization (PPO) [97]) have been introduced to estimate the optimal maintenance policy. The aforementioned single-agent DRL algorithms still have limitations in terms of large-scale problems. Serval multiagent reinforcement learning methods (e.g., hierarchical coordinated reinforcement

1.2 Selective Maintenance

13

learning [141], deep centralized multi-agent actor-critic framework [6], and valuedecomposition multi-agent actor-critic algorithm [115]) have been implemented to address large-scale maintenance optimization problems.

1.2 Selective Maintenance In many military and industrial environments, systems are typically required to perform successive assigned missions with a break between two adjacent missions. Maintenance actions can be executed in breaks to ensure the success of subsequent missions. However, because of the constraints of maintenance resources (e.g., time, budget, maintenance equipment, and personnel), repairing all aged or failed components is impractical during a break between two adjacent missions. Therefore, only a subset of maintenance actions can be selected from the set of optional maintenance actions. This maintenance optimization problem is called “selective maintenance.” For example, in a logistics system, a sequence of transportation missions is arranged for a truck, which can be maintained with a break between two adjacent missions. An aircraft is scheduled to complete multiple flights where maintenance actions can be performed during the break between two consecutive flights. An illustration of the selective maintenance problem is presented in Fig. 1.4, where a component is selected at the end of a mission to restore the state of a system. Owing to the popularity of maintenance decisions with limited maintenance resources in engineering practice, selective maintenance has been of great interest in various fields (e.g., manufacturing systems [118, 135, 142], transmission systems [75, 92], aero-engines [127], mechanical structures [28], bridge structures [48], military equipment [18], and railway networks [38]). Pioneered studies on selective maintenance can be traced back to Rice et al. [102], Cassady et al. [15], and Cassady et al. [14]. Studies on selective maintenance were comprehensively reviewed by Xu et al. [129] and Cao et al. [12]. In this chapter, an overview of selective maintenance is presented from the perspectives of system modelling, efficiency of maintenance actions, constraints of maintenance resources, mission characteristics, operating environment, and solution algorithms.

Fig. 1.4 An illustration of the selective maintenance problem

14

1 Introduction

1.2.1 System Modelling Systems in the literature on selective maintenance optimization comprise multiple components. System modelling is the basis of selective maintenance problems. Rice et al. [102] introduced a selective maintenance decision model for series-parallel systems, which comprise multiple series-connected subsystems, where each subsystem consists of multiple independent and identical copies of a component. Cassady et al. [15] systematically developed a basic selective maintenance model for systems having multiple exponentially distributed components. However, in most engineering systems, the failure intensity of a component can vary with age, and the exponential distribution cannot accurately characterize the failure behaviors of the components. A selective maintenance problem was investigated by Cassady et al. [14] for components that follow Weibull distributions. By considering the joint effect of maintainable and non-maintainable failure modes on the failure intensity of a component, Pandey and Zuo [90] developed a selective maintenance model for systems subjected to these two types of failure modes. Ruiz et al. [103] discussed a selective maintenance problem for systems with dependent failure modes. Ikonen et al. [46] utilized a bathtub-shaped failure rate model to characterize the failure behaviors of components. A data-driven selective maintenance method was introduced by Hesabi et al. [44] using a deep learning predictive model to estimate the RUL of components. The aforementioned models assume that systems and their components have only two possible states, that is, either perfectly functioning or completely failed, during their lifetime. In many engineering environments, systems can exhibit multiple intermediate states/performance capacities between the two extremes, and such systems are called multi-state systems (MSSs). Meng et al. [85] studied a selective maintenance problem for series-parallel MSSs, of which components are modeled by discrete-time Markov processes. In their work, the state of a subsystem was determined by the best state of its components, and that of the system was equal to the worst state of the subsystems. A mission is successfully completed if the state of the entire system is greater than or equal to a prespecified level. Liu and Huang [75] developed a selective maintenance problem for MSSs consisting of multiple binary-state components, and the performance capacity of the system was evaluated by using the universal generating function (UGF) method. In this study, the condition of a component is completely determined by its effective age, which can be restored by maintenance actions. By characterizing the deterioration process of a multi-state component through a continuous-time Markov process, Pandey et al. [91] extended the work of Liu and Huang [75] for MSSs with multi-state components. In addition, in many industrial systems, the deterioration state of a system or its components can be continuous. Two selective maintenance problems for systems with continuously degrading components were developed by Khatab and Aghezzaf [57] and Khatab et al. [64], wherein a gamma process was used to characterize the deterioration process of a component. The selective maintenance models, summarized from

1.2 Selective Maintenance

15

Table 1.1 Summary of selective maintenance models from the perspective of the possible states and deterioration patterns of systems and components Authors Year Component System states States Deterioration model Rice et al. [102]

1998

Binary

Meng et al. [85] Cassady et al. [15]

1999 2001

Multiple Binary

Cassady et al. [14] Liu and Huang [75] Pandey et al. [91] Pandey and Zuo [90]

2001 2010 2013 2014

Binary Binary Multiple Binary

Khatab and Aghezzaf [57] Khatab et al. [64] Ikonen et al. [46]

2016

Binary

2018 2020

Binary Binary

Ruiz et al. [103]

2020

Binary

Exponential distribution Markov process Exponential distribution Weibull distribution Effective age model Markov process Two types of failure modes Gamma process

Binary Multiple Binary Binary Multiple Multiple Binary Binary

Gamma process Binary Bathtub-shaped failure Binary rate model Multiple failure modes Binary

the perspective of the possible states and deterioration patterns of the systems and components, are listed in Table 1.1. Meanwhile, owing to the absence of data and an uncertain environment, in reality, the performance capacity and deterioration process of a system and its components can be uncertain. In this context, selective maintenance models for fuzzy systems were reported by Cao et al. [13], Zhang et al. [136], Kamal et al. [54], etc. Jiang and Liu [51] incorporated the uncertainties associated with imperfect observations into the states and effective ages of components, and a robust multi-objective selective maintenance problem was developed. Based on Dempster-Shafer theory, Galante et al. [41] developed an approach to address epistemic uncertainty in selective maintenance problems. In the above-mentioned studies on selective maintenance, the components are assumed to be independent of one another. In many practical applications, this assumption may not always hold. In most of the existing literature, the dependencies among components are distinguished into three types, as mentioned in Sect. 1.1.5: Economic dependence: Maintaining multiple components can be more economical than maintaining each component separately. For example, by sharing setup, tools, materials, labor, and maintenance resources (e.g., cost and time) can be saved in many industrial systems [29].

16

1 Introduction

Stochastic dependence: The deterioration process of a component is dependent on the state of other components. For example, a fire from a failed component can destroy adjacent components. Another example is that defects in a bearing can cause excessive vibration, which may accelerate the deterioration of related shafts and gears [116]. The stochastic dependency among components can be further classified into two types [26, 116]. Type-I stochastic dependency (i.e., immediate failure dependence) applies when the failure of a component can cause immediate failures of the affected components. Type-II stochastic dependency (i.e., gradual degradation dependence) applies when the failure or degradation of a component can accelerate the deterioration of its affected components. Structural dependence: The maintenance of a component requires the maintenance or dismantlement of other components. For example, a gearbox unit should be initially separated to replace a shaft in the gearbox, whereas all gears and bearings attached to the shaft should be sequentially disassembled [28]. In the context of selective maintenance problems, because selected maintenance actions are executed during the break between two missions, economic dependence is prevalent in the literature. Considering both economic and type-II stochastic dependences, Maaroufi et al. [81] investigated a selective maintenance problem for binarystate systems. A selective maintenance model was developed by Dao et al. [29] for MSSs with economic dependence. Xu et al. [130] distinguished economic dependence in three categories: economic dependence in system, economic dependence in subsystem, and economic dependence between subsystems. Further, a selective maintenance problem for binary-state systems with the three economic dependences was proposed. A selective maintenance problem for MSSs with stochastic dependence was developed by Dao et al. [26], where both the type-I and type-II stochastic dependences were investigated. Xu et al. [28] discussed a selective maintenance model for MSSs with both economic and structural dependences, and the assembly sequence of components was determined. Table 1.2 summarizes the dependences in the literature on selective maintenance.

Table 1.2 Dependences in the literature on selective maintenance Authors Year System state Dependence Economic Stochastic √ √ Maaroufi et al. [81] 2013 Multiple √ Dao et al. [29] 2014 Multiple √ Xu et al. [130] 2016 Binary √ Dao and Zuo [26] 2016 Multiple √ Dao and Zuo [28] 2017 Multiple √ Shahraki et al. [107] 2020 Multiple

Structural

√ √

1.2 Selective Maintenance

17

The aforementioned studies have been dedicated to the selective maintenance strategy for a single system. Moreover, in many engineering practices, multiple systems can be grouped into fleet systems (e.g., transport fleets, unmanned aerial vehicles, and production systems with multiple production lines). Schneider and Cassady [105] and Rainwater et al. [99] investigated a fleet-level selective maintenance problem, and the mission of a fleet system was completed if each system successfully completes its own mission. Based on [105] and [99], Schneider and Cassady [106] further investigated cases wherein some systems could cancel their missions. A selective maintenance model for truck fleet systems conducting longdistance highway transportation missions was investigated by Lan et al. [69]. Yang et al. [132] developed a fleet-level selective maintenance problem for multi-phase, short-term, continuous missions.

1.2.2 Efficiency of Maintenance Actions Maintenance actions can be distinguished into PM and CM actions based on the state of the component to be maintained. A CM action is executed after a component fails, whereas a PM action occurs when the system is operating. In accordance with the degree to which the condition of a component is recovered by a maintenance action, maintenance actions can be classified into five types, as described in Sect. 1.1.3: (1) perfect repair/maintenance, (2) minimal repair, (3) imperfect repair/maintenance, (4) worse repair/maintenance, and (5) worst repair/maintenance. In early studies on selective maintenance [15, 85, 102], a failed or aged component can only be repaired by replacement (perfect maintenance). Cassady et al. [14] developed a selective maintenance problem under minimal maintenance; it is assumed that a component can be restored from a failed state to a functioning state by minimal repair or perfect maintenance. Henceforth, perfect maintenance and minimal repair are assumed to be the basic optional maintenance actions in the literature on selective maintenance. In many engineering practices, maintenance actions can be characterized as imperfect maintenance. Liu and Huang [75] first incorporated the concept of imperfect maintenance into selective maintenance models, and an age reduction model (i.e., the Kijima type-II model) was utilized to characterize the influences of imperfect maintenance actions. In this study, an exponential function is proposed to formulate the relationship between maintenance quality and consumed maintenance cost. For systems executing multiple consecutive missions, Kijima type-I model was utilized by Jiang and Liu [52] for a selective maintenance problem, considering uncertainties associated with the operating time of each component and the duration of future missions. Based on a hybrid imperfect maintenance model wherein imperfect maintenance action can both reduce the effective age and change the failure intensity (hazard rate) function of a component, Pandey et al. [92] proposed a selective maintenance model under imperfect maintenance. In the framework of selective maintenance problems, an imperfect maintenance model for multi-state components was

18

1 Introduction

developed by Pandey et al. [91], and a component can be restored to a specific state. As an imperfect maintenance model for continuously degrading components, Khatab and Aghezzaf [57], Khatab et al. [64], and Khatab et al. [63] utilized the degradation reduction coefficient model [33] in selective maintenance models. Most studies on selective maintenance assume that the relationship between the efficiency of maintenance actions and maintenance resources consumed is determined. However, owing to the difference in materials, levels of maintenance personnel, maintenance environment, etc., consuming the same maintenance resources may lead to different maintenance results. Khatab and Aghezzaf [58] addressed a selective maintenance problem under the uncertain quality of imperfect maintenance actions; it is assumed that the age reduction factor in the Kijima type-II model is a continuous random variable dependent on the consumed maintenance cost. Based on their work, Duan et al. [34] developed a selective maintenance problem under stochastic maintenance quality, assuming that the mean age reduction factor is deter-

Table 1.3 Typical literature on imperfect maintenance models Authors Year Component System state state Liu and Huang [75] Maaroufi et al. [82] Pandey et al. [92] Pandey et al. [91] Khatab and Aghezzaf [57]

Imperfect maintenance model

Uncertainty of maintenance

2010

Binary

Multiple

Kijima type-II

2013

Binary

Binary

Kijima type-II

2013

Binary

Binary

Hybrid model

2013

Multiple

Multiple

2016

Binary

Binary

Khatab and 2016 Aghezzaf [58] Lan et al. [69] 2017 Khatab et al. 2018 [63]

Binary

Binary

Repair to specified state Degradation reduction coefficient model √ Kijima type-II

Binary Binary

Binary Binary

Duan et al. [34] Zhang et al. [139]

2018

Binary

Binary

2019

Multiple

Multiple

Jiang and Liu [52]

2020

Binary

Binary

√ Kijima type-II Degradation reduction coefficient model √ Kijima type-II Randomly repair to a better state Kijima type-I

√

1.2 Selective Maintenance

19

mined by the consumed maintenance cost and time. Zhang et al. [139] introduced a selective maintenance model assuming that the component can be randomly restored to a better state by maintenance actions. Table 1.3 lists the imperfect maintenance models used in studies on selective maintenance. Cases of worse maintenance are rarely discussed in the literature on selective maintenance. Zhao et al. [140] investigated a selective maintenance problem wherein human errors can lead to worse maintenance.

1.2.3 Constraints of Maintenance Resources Maintenance resources are assumed to be limited within the framework of selective maintenance. The two most common constraints in the literature on selective maintenance are maintenance budget and time. Rice et al. [102] considered a selective maintenance problem under the constraint of total maintenance time. Cassady et al. [15] developed a selective maintenance model with the constraints of both maintenance budget and time. Additionally, Maillart et al. [83] and Iyoob et al. [47] investigated the cases wherein different maintenance actions consume various types and amounts of maintenance resources. In addition to the constraints of total maintenance cost and time, the number of maintenance teams (also called maintenance personnel, maintenance crews, and repairmen) is an important constraint of maintenance resources, which has a significant influence on selective maintenance decisions. If selected maintenance actions can only be executed in series (i.e., only a single maintenance team is available), the total maintenance time cannot exceed the duration of the break between two adjacent missions. In many real engineering environments with sufficient maintenance personnel, equipment, and workplaces, selected maintenance actions can be executed simultaneously by multiple maintenance teams. Galante and Passannanti [40] addressed a selective maintenance problem wherein multiple maintenance teams can accelerate the execution of maintenance actions. Based on the abilities and expertise of different maintenance teams, the joint optimization problem of selective maintenance and maintenance team assignment was investigated by Aghezzaf et al. [1], Khatab et al. [65], Diallo et al. [30], Cao et al. [12], Chaabane et al. [19], Yang et al. [132], and Chaabane et al. [20]. Most studies on selective maintenance assume that maintenance resources consumed by maintenance actions are deterministic. However, such an assumption may not always be valid in many engineering practices. Owing to the uncertainty of operating conditions, proficiency of maintenance teams, etc., the resource consumption for executing maintenance actions may be uncertain. Ali et al. [4] provided a selective maintenance problem with probabilistic maintenance time constraints, which were satisfied with a prespecified probability. Further, the maintenance time duration of components was characterized by normal distributions. Ali et al. [5] introduced a selective maintenance model that assumes the maintenance cost to be normally distributed.

20 Table 1.4 Typical literature on imperfect maintenance models

1 Introduction Authors

Years

Ali et al. [4] Ali et al. [5] Khatab et al. [62] Khatab et al. [61] Liu et al. [77] Cao et al. [13]

2011 2013 2017 2017 2018 2018

Uncertainty Resource consumption Break Time Cost √ √ √ √ √ √ √ √ √

In addition to the uncertainty in the consumption of maintenance resources, maintenance resource constraints (i.e., available maintenance resources) may be uncertain in many military and industrial environments. For example, the arrival time of the next attack is difficult to predict in military environments. Further, the arrival times of emergency events (e.g., fires, earthquakes, and blizzards) are typically difficult or even impossible to know precisely in advance. In these circumstances, the duration of breaks between two adjacent missions can be uncertain. In the context of binary-state systems, Khatab et al. [60, 62] developed selective maintenance problems under stochastic durations of missions and breaks. Khatab et al. [62] further discussed a selective maintenance problem under the stochastic time durations of missions, breaks, and maintenance time. Because performing selected maintenance actions may be interrupted by the next mission, the earlier the maintenance action is performed, the more likely it is to be completed. Hence, Liu et al. [77] developed a selective maintenance model for MSSs and optimized the sequence of maintenance actions. By characterizing the duration of breaks and missions as fuzzy values, Cao et al. [13] proposed a selective maintenance model for fuzzy MSSs. Table 1.4 presents a summary of the contributions to selective maintenance problems that consider the uncertainty of the break duration and maintenance resource consumption.

1.2.4 Mission Characteristics and Operating Environment As maintenance actions are performed in the break between two adjacent missions to ensure the success of subsequent missions, the characteristics of the missions and the operating environment of the systems may influence the decision-making of selective maintenance problems. In most studies on selective maintenance, the mission duration is assumed to be determined in advance. However, in certain realworld situations, the time duration of a mission can be uncertain. For example, a flight mission may be randomly prolonged owing to weather. Hence, in the framework of

1.2 Selective Maintenance

21

selective maintenance, Djelloul et al. [32] and subsequent studies [57] and [61] characterized the uncertainty of mission duration using random variables. Most existing studies on selective maintenance work have been devoted to the success of a single future mission. In practice, the maintenance actions executed in previous breaks can affect the condition of a system, which may further affect the success of subsequent missions. Cassady et al. [14] simulated the evolution of a binary-state system executing multiple missions with a given selective maintenance strategy. They found that the distribution of the system state and the probability of a mission being successfully completed converge to steady values after multiple missions and breaks. Assuming that the lifetime of each component follows an exponential distribution, Iyoob et al. [47] proposed a selective maintenance problem over infinite missions and utilized a discrete-time Markov model to formulate the system state. For systems with exponentially distributed components, Maillart et al. [83] formulated two selective maintenance problems over finite and infinite missions by MDP , respectively, and maintenance actions were dynamically selected at each break based on the state of each component. An approximate dynamic programming algorithm was used by Ahadi and Sullivan [2] to address the selective maintenance optimization proposed in [83] when the number of components in a series-parallel system was extremely large. In accordance with the assumption that a component that fails in a mission can be recovered to a functioning state through minimal maintenance actions, Khatab et al. [59] and Pandey et al. [93] developed selective maintenance problems subject to multiple missions. Yang et al. [132] investigated a joint optimization problem of selective maintenance and fleet system assignments for multi-phase, short-term, continuous missions. To minimize the total cost under the system reliability constraint, Zhang et al. [136] proposed a selective maintenance model for MSSs for multiple consecutive missions. In the framework of selective maintenance, Chaabane et al. [20] and Liu et al. [73] jointly optimized the selected maintenance actions and assignment of multiple repairpersons for multiple consecutive missions. By considering the uncertainties in the operating time of the components and durations of multiple future missions, Jiang and Liu [52] proposed a selective maintenance strategy specified at the beginning of the first mission. Gao et al. [42] investigated a selective maintenance model over multiple consecutive missions under the uncertainty associated with the time duration of each mission and the constraint of maintenance time. Liu et al. [78] developed a dynamic selective maintenance strategy for MSSs to maximize the number of successes of multiple consecutive missions. They achieved a customized DRL method to overcome the “curse of dimensionality.” Inspired by the DRL method proposed by Liu et al. [78], Xu et al. [131] proposed a hybrid discrete differential evolution and deep Q-network algorithm for multi-component binary-state systems to minimize the future expected cost. In addition to the uncertainty of mission duration and the number of missions, practical operating environments and specific missions may have an impact on maintenance decisions (e.g., influencing the degradation processes of components [22, 27, 110, 111] and adding specific decision variables [22, 117]). Chen et al. [22] investigated a selective maintenance problem for MSSs, and the effect of loads on the

22

1 Introduction

deteriorating process of components and systems was considered. Dao and Zuo [27] developed a selective maintenance model for MSS under stochastic loading. They also utilized the Monte Carlo method to evaluate the probability of a system successfully completing a mission. A selective maintenance model under quality control and production scheduling was developed by Tambe and Kulkarni [117]. They considered the costs associated with quality control actions such as the sampling procedure. Hou and Qian [45] examined a selective maintenance problem under the scenarios that the operating time of each subsystem may be different in a mission. A selective maintenance model for random phased-mission systems was developed in Jia et al. [50]. Sharma et al. [108] proposed a selective maintenance problem for army equipment and utilized a simulation-based method to forecast the requirement of spare parts. Turan et al. [120] developed a selective maintenance problem for MSSs under a dynamic operating environment, which was characterized via a Markov process.

1.2.5 Solution Algorithms An enumeration method was used by Cassady et al. [15] to address the basic selective maintenance model. However, the amount of computing resources consumed will significantly increase with the scale of the system. Schneider and Cassady [105] addressed a selective maintenance problem for a fleet system using an enumeration method, and Rainwater et al. [99] further improved the algorithm in [105]. To improve the computational efficiency, Rajagopalan and Cassady [100, 101] proposed branch-and-bound methods for selective maintenance problems. Galante and Passannanti [40] developed an exact algorithm for selective maintenance problems of series-parallel systems. Five rules were proposed by Maaroufi et al. [81] to reduce the solution space of a selective maintenance problem for systems that are subject to propagated failures with global effects and failure isolation phenomena. Maillart et al. [83] resolved selective maintenance problems over the finite and infinite horizon by using dynamic programming and policy iteration algorithms, respectively. For binary-state systems, Ahadi and Sullivan [2] proposed an approximate dynamic programming method for selective maintenance problems that was formulated as a high-dimensional Markov decision process. Most of the aforementioned optimization algorithms are intractable for large-scale systems with more possible states and optional maintenance actions. Because of the efficient computational performance in solving various complex and nonlinear optimization problems, metaheuristic algorithms have been introduced to resolve selective maintenance problems. Lust et al. [79] utilized a tabu search algorithm, along with a branch-and-bound algorithm, to address a basic selective maintenance problem for binary state systems. A genetic algorithm was utilized by Liu and Huang [75] to address a selective maintenance problem for MSSs. A particle swarm optimization algorithm was used by Lv et al. [80] to address the selective maintenance problem considering the diagnostic uncertainty of built-in test equipment. Pandey et al. [91] solved a selective maintenance problem for an MSS consisting of multi-state compo-

References

23

nents by using a differential evolution algorithm. A simulated annealing algorithm was developed by Tambe and Kulkarni [117] to address the joint optimization of selective maintenance, quality control, and production scheduling. Backward search and genetic algorithms were utilized by Dao and Zuo [28] for the selective maintenance problem with structural dependence. An ant colony optimization algorithm was developed by Liu et al. [77] to determine the optimal sequence of maintenance actions. A dynamic artificial bee colony algorithm was utilized by Zhang et al. [136] to optimize a selective maintenance problem. Owing to the rise in AI, deep-learning-based optimization algorithms have demonstrated the ability to resolve complex and high-dimensional problems. For MSSs executing multiple missions, Liu et al. [78] customized a deep reinforcement learning method to overcome the “curse of dimensionality” in dynamic selective maintenance problems, which was formulated as a discrete-time finite-horizon Markov decision process. Based on [78], Xu et al. [131] proposed an algorithm that combines a deep Q-network and discrete differential evolution algorithm.

References 1. Aghezzaf EH, Diallo C, Khatab A, Venkatadri U (2017) A joint selective maintenance and multiple repair-person assignment problem. In: The 7th IESM conference, Saarbrücken, Germany, pp 317–322 2. Ahadi K, Sullivan KM (2019) Approximate dynamic programming for selective maintenance in series-parallel systems. IEEE Trans Reliab 69(3):1147–1164 3. Alaswad S, Xiang Y (2017) A review on condition-based maintenance optimization models for stochastically deteriorating system. Reliab Eng Syst Saf 157:54–63 4. Ali I, Khan MF, Raghav YS, Bari A et al (2011) Allocation of repairable and replaceable components for a system availability using selective maintenance with probabilistic maintenance time constraints. Am J Oper Res 1(3):147–154 5. Ali I, Raghav YS, Khan MF, Bari A (2013) Selective maintenance in system reliability with random costs of repairing and replacing the components. Commun Stat-Simul Comput 42(9):2026–2039 6. Andriotis C, Papakonstantinou K (2019) Managing engineering systems with large state and action spaces through deep reinforcement learning. Reliab Eng Syst Saf 191:106483 7. ANSAit (2016) Strage ferroviaria in puglia: 23 morti. https://www.ansa.it/puglia/notizie/ 2016/07/12/scontro-tra-due-treni-in-puglia_e5d6c31d-dabb-477a-9607-f4418270d4d2. html 8. Baraldi P, Mangili F, Zio E (2013) Investigation of uncertainty treatment capability of modelbased and data-driven prognostic methods using simulated data. Reliab Eng Syst Saf 112:94– 108 9. Broek MAUH, Teunter RH, De Jonge B, Veldman J (2021) Joint condition-based maintenance and condition-based production optimization. Reliab Eng Syst Saf 214:107743 10. Broek MAUH, Teunter RH, de Jonge B, Veldman J (2021) Joint condition-based maintenance and load-sharing optimization for two-unit systems with economic dependency. Eur J Oper Res 295(3):1119–1131 11. Budai G, Huisman D, Dekker R (2006) Scheduling preventive railway maintenance activities. J Oper Res Soc 57(9):1035–1044 12. Cao W, Jia X, Hu Q, Zhao J, Wu Y (2018) A literature review on selective maintenance for multi-unit systems. Qual Reliab Eng Int 34(5):824–845

24

1 Introduction

13. Cao W, Jia X, Liu Y, Hu Q (2018) Selective maintenance optimization for fuzzy multi-state systems. J Intell Fuzzy Syst 34(1):105–121 14. Cassady CR, Murdock WP, Pohl EA (2001) Selective maintenance for support equipment involving multiple maintenance actions. Eur J Oper Res 129(2):252–258 15. Cassady CR, Pohl EA, Murdock WP (2001) Selective maintenance modeling for industrial systems. J Qual Maint Eng 7(2):104–117 16. Cavalcante CA, Lopes RS, Scarf PA (2018) A general inspection and opportunistic replacement policy for one-component systems of variable quality. Eur J Oper Res 266(3):911–919 17. Cavalcante CA, Lopes RS, Scarf PA (2021) Inspection and replacement policy with a fixed periodic schedule. Reliab Eng Syst Saf 208:107402 18. Certa A, Galante G, Lupo T, Passannanti G (2011) Determination of pareto frontier in multiobjective maintenance optimization. Reliab Eng Syst Saf 96(7):861–867 19. Chaabane K, Khatab A, Aghezzaf EH, Diallo C, Venkatadri U (2018) Outsourcing selective maintenance problem in failure prone multi-component systems. IFAC-PapersOnLine 51(11):525–530 20. Chaabane K, Khatab A, Diallo C, Aghezzaf EH, Venkatadri U (2020) Integrated imperfect multimission selective maintenance and repairpersons assignment problem. Reliab Eng Syst Saf 199:106895 21. Chalabi N, Dahane M, Beldjilali B, Neki A (2016) Optimisation of preventive maintenance grouping strategy for multi-component series systems: particle swarm based approach. Comput Ind Eng 102:440–451 22. Chen C, Liu Y, Huang HZ (2012) Optimal load distribution for multi-state systems under selective maintenance strategy. In: 2012 International conference on quality, reliability, risk, maintenance, and safety engineering, pp 436–442 23. Chen N, Ye ZS, Xiang Y, Zhang L (2015) Condition-based maintenance using the inverse Gaussian degradation model. Eur J Oper Res 243(1):190–199 24. Chen Y, Liu Y, Xiahou T (2021) A deep reinforcement learning approach to dynamic loading strategy of repairable multistate systems. IEEE Trans Reliab 71(1):484–499 25. Corotis RB, Hugh Ellis J, Jiang M (2005) Modeling of risk-based inspection, maintenance and life-cycle cost with partially observable Markov decision processes. Struct Infrastruct Eng 1(1):75–84 26. Dao CD, Zuo MJ (2015) Selective maintenance for multistate series systems with s-dependent components. IEEE Trans Reliab 65(2):525–539 27. Dao CD, Zuo MJ (2017) Optimal selective maintenance for multi-state systems in variable loading conditions. Reliab Eng Syst Saf 166:171–180 28. Dao CD, Zuo MJ (2017) Selective maintenance of multi-state systems with structural dependence. Reliab Eng Syst Saf 159:184–195 29. Dao CD, Zuo MJ, Pandey M (2014) Selective maintenance for multi-state series-parallel systems under economic dependence. Reliab Eng Syst Saf 121:240–249 30. Diallo C, Venkatadri U, Khatab A, Liu Z, Aghezzaf EH (2019) Optimal joint selective imperfect maintenance and multiple repairpersons assignment strategy for complex multicomponent systems. Int J Prod Res 57(13):4098–4117 31. Dinh DH, Do P, Iung B (2022) Multi-level opportunistic predictive maintenance for multicomponent systems with economic dependence and assembly/disassembly impacts. Reliab Eng Syst Saf 217:108055 32. Djelloul I, Khatab A, Aghezzaf EH, Sari Z (2015) Optimal selective maintenance policy for series-parallel systems operating missions of random durations. In: International conference on computers & industrial engineering. Metz, France, pp 28–30 33. Do P, Voisin A, Levrat E, Iung B (2015) A proactive condition-based maintenance strategy with both perfect and imperfect maintenance actions. Reliab Eng Syst Saf 133:22–32 34. Duan C, Deng C, Gharaei A, Wu J, Wang B (2018) Selective maintenance scheduling under stochastic maintenance quality with multiple maintenance actions. Int J Prod Res 56(23):7160–7178

References

25

35. EasyAcc (2016) Samsung Galaxy Note 7 battery explosion risk. https://blog.easyacc.com/ 2016/09/02/samsung-galaxy-note-7-battery-explosion-risk/ 36. Elsayed EA (2012) Reliability engineering. Wiley, New York 37. Escudero L (1982) On maintenance scheduling of production units. Eur J Oper Res 9(3):264– 274 38. Fecarotti C, Andrews J, Pesenti R (2021) A mathematical programming model to select maintenance strategies in railway networks. Reliab Eng Syst Saf 216:107940 39. Feng H, Xi L, Xiao L, Xia T, Pan E (2018) Imperfect preventive maintenance optimization for flexible flowshop manufacturing cells considering sequence-dependent group scheduling. Reliab Eng Syst Saf 176:218–229 40. Galante G, Passannanti G (2009) An exact algorithm for preventive maintenance planning of series-parallel systems. Reliab Eng Syst Saf 94(10):1517–1525 41. Galante GM, La Fata CM, Lupo T, Passannanti G (2020) Handling the epistemic uncertainty in the selective maintenance problem. Comput Ind Eng 141:106293 42. Gao H, Zhang X, Yang X, Zheng B (2021) Optimal selective maintenance decision-making for consecutive-mission systems with variable durations and limited maintenance time. Math Probl Eng. https://doi.org/10.1155/2021/5534659 43. Geraerds W (1985) The cost of downtime for maintenance: preliminary considerations. Maintenance Manag Int 5(1):13–21 44. Hesabi H, Nourelfath M, Hajji A (2022) A deep learning predictive model for selective maintenance optimization. Reliab Eng Syst Saf 219:108191 45. Hou J, Qian Y (2015) Selective maintenance model based on different mission duration time. In: 2015 IEEE international conference on mechatronics and automation, pp 2491–2495 46. Ikonen TJ, Mostafaei H, Ye Y, Bernal DE, Grossmann IE, Harjunkoski I (2020) Large-scale selective maintenance optimization using bathtub-shaped failure rates. Comput Chem Eng 139:106876 47. Iyoob IM, Cassady CR, Pohl EA (2006) Establishing maintenance resource levels using selective maintenance. Eng Econ 51(2):99–114 48. Janjic AD, Popovic DS (2007) Selective maintenance schedule of distribution networks based on risk management approach. IEEE Trans Power Syst 22(2):597–604 49. Jardine AK, Lin D, Banjevic D (2006) A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mech Syst Signal Process 20(7):1483–1510 50. Jia X, Cao W, Hu Q (2019) Selective maintenance optimization for random phased-mission systems subject to random common cause failures. Proc Inst Mech Eng, Part O: J Risk Reliab 233(3):379–400 51. Jiang T, Liu Y (2020) Robust selective maintenance strategy under imperfect observations: a multi-objective perspective. IISE Trans 52(7):751–768 52. Jiang T, Liu Y (2020) Selective maintenance strategy for systems executing multiple consecutive missions with uncertainty. Reliab Eng Syst Saf 193:106632 53. de Jonge B, Scarf PA (2020) A review on maintenance optimization. Eur J Oper Res 285(3):805–824 54. Kamal M, Modibbo UM, AlArjani A, Ali I (2021) Neutrosophic fuzzy goal programming approach in selective maintenance allocation of system reliability. Complex Intell Syst 7(2):1045–1059 55. Keizer MCO, Flapper SDP, Teunter RH (2017) Condition-based maintenance policies for systems with multiple dependent components: a review. Eur J Oper Res 261(2):405–420 56. Keizer MCO, Teunter RH, Veldman J, Babai MZ (2018) Condition-based maintenance for systems with economic dependence and load sharing. Int J Prod Econ 195:319–327 57. Khatab A, Aghezzaf E (2016) Selective maintenance optimization for series-parallel systems with continuously monitored stochastic degrading components subject to imperfect maintenance. IFAC-PapersOnLine 49(28):256–261 58. Khatab A, Aghezzaf EH (2016) Selective maintenance optimization when quality of imperfect maintenance actions are stochastic. Reliab Eng Syst Saf 150:182–189

26

1 Introduction

59. Khatab A, Aghezzaf EH, Claver D (2015) Maintenance optimization of series-parallel systems operating missions with scheduled breaks. In: 2015 27th international conference on microelectronics, pp 138–141 60. Khatab A, Aghezzaf EH, Djelloul I, Sari Z (2016) Selective maintenance for series-parallel systems when durations of missions and planned breaks are stochastic. IFAC-PapersOnLine 49(12):1222–1227 61. Khatab A, Aghezzaf EH, Diallo C, Djelloul I (2017) Selective maintenance optimisation for series-parallel systems alternating missions and scheduled breaks with stochastic durations. Int J Prod Res 55(10):3008–3024 62. Khatab A, Aghezzaf EH, Djelloul I, Sari Z (2017) Selective maintenance optimization for systems operating missions and scheduled breaks with stochastic durations. J Manuf Syst 43:168–177 63. Khatab A, Aghezzaf EH, Diallo C, Venkatadri U (2018) Condition-based selective maintenance for multicomponent systems under environmental and energy considerations. In: 2018 IEEE international conference on industrial engineering and engineering management, pp 212–216 64. Khatab A, Diallo C, Aghezzaf EH, Venkatadri U (2018) Condition-based selective maintenance for stochastically degrading multi-component systems under periodic inspection and imperfect maintenance. Proc Inst Mech Eng, Part O: J Risk Reliab 232(4):447–463 65. Khatab A, Diallo C, Venkatadri U, Liu Z, Aghezzaf EH (2018) Optimization of the joint selective maintenance and repairperson assignment problem under imperfect maintenance. Comput Ind Eng 125:413–422 66. Kijima M (1989) Some results for repairable systems with general repair. J Appl Probab 26(1):89–102 67. Kijima M, Morimura H, Suzuki Y (1988) Periodical replacement problem without assuming minimal repair. Eur J Oper Res 37(2):194–203 68. Kurt M, Kharoufeh JP (2010) Monotone optimal replacement policies for a Markovian deteriorating system in a controllable environment. Oper Res Lett 38(4):273–279 69. Lan P, Lin M, Naichao W (2017) A fleet-level selective maintenance model for long-distance highway transportation considering stochastic repair quality. In: 2017 2nd international conference on system reliability and safety, pp 348–353 70. Lei Y, Li N, Guo L, Li N, Yan T, Lin J (2018) Machinery health prognostics: a systematic review from data acquisition to RUL prediction. Mech Syst Signal Process 104:799–834 71. Liao H, Elsayed EA, Chan LY (2006) Maintenance of continuously monitored degrading systems. Eur J Oper Res 175(2):821–835 72. Liu B, Pandey MD, Wang X, Zhao X (2021) A finite-horizon condition-based maintenance policy for a two-unit system with dependent degradation processes. Eur J Oper Res 295(2):705–717 73. Liu L, Yang J, Kong X, Xiao Y (2022) Multi-mission selective maintenance and repairpersons assignment problem with stochastic durations. Reliab Eng Syst Saf 219:108209 74. Liu Y, Huang HZ (2010) Optimal replacement policy for multi-state system under imperfect maintenance. IEEE Trans Reliab 59(3):483–495 75. Liu Y, Huang HZ (2010) Optimal selective maintenance strategy for multi-state systems under imperfect maintenance. IEEE Trans Reliab 59(2):356–367 76. Liu Y, Huang HZ, Zhang X (2011) A data-driven approach to selecting imperfect maintenance models. IEEE Trans Reliab 61(1):101–112 77. Liu Y, Chen Y, Jiang T (2018) On sequence planning for selective maintenance of multi-state systems under stochastic maintenance durations. Eur J Oper Res 268(1):113–127 78. Liu Y, Chen Y, Jiang T (2020) Dynamic selective maintenance optimization for multi-state systems over a finite horizon: a deep reinforcement learning approach. Eur J Oper Res 283(1):166–181 79. Lust T, Roux O, Riane F (2009) Exact and heuristic methods for the selective maintenance problem. Eur J Oper Res 197(3):1166–1177

References

27

80. Lv XZ, Yu YL, Zhang L, Liu YF, Chen LY (2011) Stochastic program for selective maintenance decision considering diagnostics uncertainty of built-in test equipment. In: 2011 international conference on quality, reliability, risk, maintenance, and safety engineering, pp 584–589 81. Maaroufi G, Chelbi A, Rezg N (2013) Optimal selective renewal policy for systems subject to propagated failures with global effect and failure isolation phenomena. Reliab Eng Syst Saf 114:61–70 82. Maaroufi G, Chelbi A, Rezg N (2013) A selective maintenance policy for multi-component systems involving replacement and imperfect preventive maintenance actions. In: Proceedings of 2013 international conference on industrial engineering and systems management, pp 1–8 83. Maillart LM, Cassady CR, Rainwater C, Schneider K (2009) Selective maintenance decisionmaking over extended planning horizons. IEEE Trans Reliab 58(3):462–469 84. Malik MAK (1979) Reliable preventive maintenance scheduling. AIIE Trans 11(3):221–228 85. Meng MH, Zuo M, et al. (1999) Selective maintenance optimization for multi-state systems. In: 1999 IEEE Canadian conference on electrical and computer engineering, vol 3, pp 1477– 1482 86. Moghaddam KS, Usher JS (2011) Preventive maintenance and replacement scheduling for repairable and maintainable systems using dynamic programming. Comput Ind Eng 60(4):654–665 87. Nakagawa T (1979) Optimum policies when preventive maintenance is imperfect. IEEE Trans Reliab 28(4):331–332 88. Neves ML, Santiago LP, Maia CA (2011) A condition-based maintenance policy and input parameters estimation for deteriorating systems under periodic inspection. Comput Ind Eng 61(3):503–511 89. Open Source Investigations (2011) Us government report declassified: the aftermath of the fukushima disaster. https://www.opensourceinvestigations.com/japan/declassified-theaftermath-of-the-fukushima-disaster/ 90. Pandey M, Zuo MJ (2014) Selective maintenance considering two types of failure modes. Int J Strateg Eng Asset Manage 2(1):37–62 91. Pandey M, Zuo MJ, Moghaddass R (2013) Selective maintenance modeling for a multistate system with multistate components under imperfect maintenance. IIE Trans 45(11):1221– 1234 92. Pandey M, Zuo MJ, Moghaddass R, Tiwari M (2013) Selective maintenance for binary systems under imperfect repair. Reliab Eng Syst Saf 113:42–51 93. Pandey M, Zuo MJ, Moghaddass R (2016) Selective maintenance scheduling over a finite planning horizon. Proc Inst Mech Eng, Part O: J Risk Reliab 230(2):162–177 94. Papakonstantinou KG, Andriotis CP, Shinozuka M (2018) POMDP and MOMDP solutions for structural life-cycle cost minimization under partial and mixed observability. Struct Infrastruct Eng 14(7):869–882 95. Pham H, Wang H (1996) Imperfect maintenance. Eur J Oper Res 94(3):425–438 96. Pham H, Wang H (1996) Optimal maintenance policies for several imperfect maintenance models. Int J Syst Sci 27(6):543–551 97. Pinciroli L, Baraldi P, Ballabio G, Compare M, Zio E (2021) Deep reinforcement learning based on proximal policy optimization for the maintenance of a wind farm with multiple crews. Energies 14(20):6743 98. Post Magazine (2021) How decades of deskilling at boeing doomed two 737 max flights -author dissects the death of its safety culture at the hands of corporate profiteers. https://www.scmp.com/magazines/post-magazine/books/article/3160046/ how-decades-deskilling-boeing-doomed-two-737-max 99. Rainwater C, Honeycutt J, Richard Cassady PhD C, P, Scott Mason PhD P, (2004) Solving selective maintenance problems for fleets of systems. In: IIE Annual conference. Houston, USA, pp 1–6 100. Rajagopalan R, Cassady CR (2004) Solving selective maintenance problems. In: IIE annual conference. Houston, USA, pp 1–5

28

1 Introduction

101. Rajagopalan R, Cassady CR (2006) An improved selective maintenance solution approach. J Qual Maint Eng 12(2):172–185 102. Rice W, Cassady C, Nachlas J (1998) Optimal maintenance plans under limited maintenance time. In: Proceedings of the seventh industrial engineering research conference, Banff, Canada, pp 1–3 103. Ruiz C, Pohl EA, Liao H (2020) Selective maintenance modeling and analysis of a complex system with dependent failure modes. Qual Eng 32(3):509–520 104. Safaei N, Banjevic D, Jardine AK (2011) Workforce-constrained maintenance scheduling for military aircraft fleet: a case study. Ann Oper Res 186(1):295–316 105. Schneider K, Cassady CR (2004) Fleet performance under selective maintenance. In: Annual symposium reliability and maintainability, pp 571–576 106. Schneider K, Cassady CR (2015) Evaluation and comparison of alternative fleet-level selective maintenance models. Reliab Eng Syst Saf 134:178–187 107. Shahraki AF, Yadav OP, Vogiatzis C (2020) Selective maintenance optimization for multi-state systems considering stochastically dependent components and stochastic imperfect maintenance actions. Reliab Eng Syst Saf 196:106738 108. Sharma P, Kulkarni MS, Yadav V (2017) A simulation based optimization approach for spare parts forecasting and selective maintenance. Reliab Eng Syst Saf 168:274–289 109. Sheikhalishahi M, Eskandari N, Mashayekhi A, Azadeh A (2019) Multi-objective open shop scheduling by considering human error and preventive maintenance. Appl Math Model 67:573–587 110. Shen J, Cui L (2016) Reliability performance for dynamic systems with cycles of K regimes. IIE Trans 48(4):389–402 111. Shen J, Cui L (2017) Reliability performance for dynamic multi-state repairable systems with K regimes. IISE Trans 49(9):911–926 112. Shi Y, Zhu W, Xiang Y, Feng Q (2020) Condition-based maintenance optimization for multi-component systems subject to a system reliability requirement. Reliab Eng Syst Saf 202:107042 113. Si XS, Wang W, Hu CH, Zhou DH (2011) Remaining useful life estimation-a review on the statistical data driven approaches. Eur J Oper Res 213(1):1–14 114. Sörensen K, Glover F (2013) Metaheuristics. Encycl Oper Res Manage Sci 62:960–970 115. Su J, Huang J, Adams S, Chang Q, Beling PA (2022) Deep multi-agent reinforcement learning for multi-level preventive maintenance in manufacturing systems. Expert Syst Appl 192:116323 116. Sun Y, Ma L, Mathew J, Zhang S (2006) An analytical model for interactive failures. Reliab Eng Syst Saf 91(5):495–504 117. Tambe PP, Kulkarni MS (2014) A novel approach for production scheduling of a high pressure die casting machine subjected to selective maintenance and a sampling procedure for quality control. Int J Syst Assur Eng Manage 5(3):407–426 118. Tambe PP, Kulkarni MS (2016) Selective maintenance optimization under schedule and quality constraints. Int J Qual Reliab Manage 33(7):1030–1059 119. Tian Z, Lin D, Wu B (2012) Condition based maintenance optimization considering multiple objectives. J Intell Manuf 23(2):333–340 120. Turan HH, Atmis M, Kosanoglu F, Elsawah S, Ryan MJ (2020) A risk-averse simulation-based approach for a joint optimization of workforce capacity, spare part stocks and scheduling priorities in maintenance planning. Reliab Eng Syst Saf 204:107199 121. Verma AK, Ajit S, Karanki DR et al (2010) Reliability and safety engineering. Springer, London 122. Wang H (2002) A survey of maintenance policies of deteriorating systems. Eur J Oper Res 139(3):469–489 123. Wang H, Pham H (1996) Optimal age-dependent preventive maintenance policies with imperfect maintenance. Int J Reliab Qual Saf Eng 3(02):119–135 124. Wang H, Pham H (1996) A quasi renewal process and its applications in imperfect maintenance. Int J Syst Sci 27(10):1055–1062

References

29

125. Wang W (2012) An overview of the recent advances in delay-time-based maintenance modelling. Reliab Eng Syst Saf 106:165–178 126. Wu S, Zuo MJ (2010) Linear and nonlinear preventive maintenance models. IEEE Trans Reliab 59(1):242–249 127. Xu J, Sun K, Xu L (2016) Integrated system health management-oriented maintenance decision-making for multi-state system based on data mining. Int J Syst Sci 47(13):3287– 3301 128. Xu J, Zhao X, Liu B (2022) A risk-aware maintenance model based on a constrained Markov decision process. IISE Trans 54(11):1072–1083 129. Xu QZ, Guo LM, Wang N, Fei R (2015) Recent advances in selective maintenance from 1998 to 2014. J Donghua Univ 32(6):986–994 130. Xu QZ, Guo Lm, Shi Hp, Wang N (2016) Selective maintenance problem for series-parallel system under economic dependence. Defence Technol 12(5):388–400 131. Xu Y, Pi D, Wu Z, Chen J, Zio E (2021) Hybrid discrete differential evolution and deep Q-network for multimission selective maintenance. IEEE Trans Reliab, pp 1–12. https://doi. org/10.1109/TR.2021.3111737 132. Yang D, Wang H, Feng Q, Ren Y, Sun B, Wang Z (2018) Fleet-level selective maintenance problem under a phased mission scheme with short breaks: a heuristic sequential game approach. Comput Ind Eng 119:404–415 133. Ye ZS, Chen N (2014) The inverse Gaussian process as a degradation model. Technometrics 56(3):302–311 134. Yousefi N, Tsianikas S, Coit DW (2020) Reinforcement learning for dynamic condition-based maintenance of a system with individually repairable components. Qual Eng 32(3):388–408 135. Yu P, Schneider K (2003) Selective maintenance strategies for serial production lines. In: IIE annual conference, pp 1–4 136. Zhang L, Zhang L, Shan H, Shan H (2019) Selective maintenance planning considering team capability based on fuzzy integral and dynamic artificial bee colony algorithm. IEEE Access 7:66553–66566 137. Zhang M, Gaudoin O, Xie M (2015) Degradation-based maintenance decision using stochastic filtering for systems under imperfect maintenance. Eur J Oper Res 245(2):531–541 138. Zhang N, Si W (2020) Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks. Reliab Eng Syst Saf 203:107094 139. Zhang X, Chen J, Han B, Li J (2019) Multi-mission selective maintenance modelling for multistate systems over a finite time horizon. Proc Inst Mech Eng Part O: J Risk Reliab 233(6):1040–1059 140. Zhao Z, Xiao B, Wang N, Yan X, Ma L (2019) Selective maintenance optimization for a multi-state system considering human reliability. Symmetry 11(5):652 141. Zhou Y, Li B, Lin TR (2022) Maintenance optimisation of multicomponent systems using hierarchical coordinated reinforcement learning. Reliab Eng Syst Saf 217:108078 142. Zhu H, Liu F, Shao X, Liu Q, Deng Y (2011) A cost-based selective maintenance decisionmaking method for machining line. Qual Reliab Eng Int 27(2):191–201 143. Zio E (2009) Reliability engineering: old problems and new challenges. Reliab Eng Syst Saf 94(2):125–141 144. Zio E (2022) Prognostics and health management (PHM): where are we and where do we (need to) go in theory and practice. Reliab Eng Syst Saf 218:108119

Chapter 2

Basic Selective Maintenance Model

2.1 Introduction Maintenance, involving both preventive and corrective actions carried out to retain a system or restore it to an acceptable operating condition, has been extensively implemented in industrial applications [1]. Maintenance decisions aim to identify an optimal trade-off between maintenance resource consumption and system reliability/performability improvement [8]. In many industrial and military environments, engineered systems are intended to execute a sequence of missions with breaks between two successive missions. For example, production systems operate during weekdays, and maintenance actions can only be executed on weekends. Overhaul maintenance can only be performed for military vessels during a break between two adjacent tasks. Each maintenance action consumes various resources (e.g., maintenance time, budget, manpower, and repair facilities). Owing to the limited resources in a break, not all components in a system can be repaired during each break. Alternatively, decision makers must identify a subset of components to be repaired to enhance the chance of success of the next mission. This maintenance-planning strategy is called selective maintenance [9]. Selective maintenance optimization aims to identify an optimal subset of maintenance actions from all desirable maintenance actions of all components. The basic mathematical model of selective maintenance is described in this section. Rice et al. [9] first introduced the selective maintenance model for a serial-parallel system having identical components. In the model, replacement is the only maintenance action that can be selected for each component, and the lifetime of each component complies with an exponential distribution. Over the last two decades, the selective maintenance problem has been explored intensively from various perspectives. Some extensions of the aforementioned selective maintenance model are reviewed below.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_2

31

32

2 Basic Selective Maintenance Model

Cassady et al. [4] developed a general framework for selective maintenance problem. The duration of the maintenance time, maintenance budget, and system survival probability can be regarded as either objective functions or constraints of the optimization problem. The following three variations of selective maintenance models were constructed: maximizing the system survival probability subject to maintenance time and budget constraints, minimizing maintenance cost subject to the minimum required system survival probability and maintenance time constraints, and minimizing maintenance time subject to maintenance budget and system survival probability constraints. Cassady et al. [3] further incorporated minimal repair and preventive replacement into the selective maintenance model. The failure intensity of the components was no longer constant, which is not applicable to many real-world industrial applications. Rather, the Weibull distribution was used to characterize the failure behaviors of the components in a repairable system. Schneider and Cassady [10] extended the selective maintenance problem to a fleet comprising multiple systems. Fleet reliability is defined as the probability of all systems in a fleet successfully completing their next missions. Assuming a constant failure rate and identical mission length, a selective maintenance model was formulated for this scenario with a sequence of identical missions. Iyoob et al. [5] incorporated the following four features as alternative decision variables in selective maintenance optimization: acquiring additional resources, establishing optimal constant resource capacities for sequential missions, integrating redundancy allocations, and resource allocation in the system design phase. Maillart et al. [7] investigated a selective maintenance model for a system performing multiple consecutive and identical missions. They found that only the corrective replacement was involved during the break between two adjacent missions. They concluded that a single- or a two-mission policy is likely to be a practical tradeoff for this type of system. This chapter introduced a basic mathematical model for the selective maintenance of binary-state systems. This model served as the basis for ensuing chapters with more advanced selective maintenance models. The remainder of this chapter was organized as follows. In Sect. 2.2, the problem statement and basic assumptions in selective maintenance were introduced. In Sect. 2.3, decision variables and the effect of these decision variables were discussed. In Sect. 2.4, the probability of a system successfully completing a future mission was evaluated. In Sect. 2.5, we introduced three basic selective maintenance models. In Sect. 2.6, an illustrative example was provided to demonstrate the effectiveness of selective maintenance. Finally, a brief description of the closure was provided in Sect. 2.7.

2.2 Problem Statements and Model Assumptions

33

2.2 Problem Statements and Model Assumptions 2.2.1 Problem Statements A system comprising several components is intended to complete a set of consecutive missions (Fig. 2.1). Maintenance activities can only be performed between two adjacent missions. At the beginning of the break, some components failed or aged. Because maintenance resources (e.g., budget and break duration) are limited, all failed or aged components cannot be repaired/replaced. A maintenance decision must be made at the beginning of the break to determine a subset of components among all components to be repaired to ensure the success of the next mission. This decision-making process is called selective maintenance [4]. The state of component l at the beginning of the kth (k ∈ {1, 2, . . . , K }) mission is represented by a binary variable, denoted by X l,k . Thus, we have  X l,k =

1 if component l is functioning . 0 if component l is failed

(2.1)

At the end of the kth mission, the state of component l is represented by a binary variable, denoted by Yl,k . Thus, we have  Yl,k =

1 if component l is functioning . 0 if component l is failed

(2.2)

Because maintenance activities cannot be performed during a mission, the relation X l,k ≥ Yl,k always holds. The effective ages of component l at the beginning and end of the kth mission are denoted by Al,k and Bl,k , respectively.

2.2.2 Model Assumptions The assumptions in the basic selective maintenance model are as follows [3, 6]:

?

? kth mission

kth break

?

(k+1)th mission

Selective Maintenance Decision

time

Fig. 2.1 An illustration of a system executing consecutive missions with a break between two adjacent missions

34

2 Basic Selective Maintenance Model

• A system comprises M repairable and s-independent components connected with an arbitrary configuration, such as series-parallel, bridge, and network. The system configuration remains unchanged throughout all the missions. • Both the system and its components are binary-state, either working perfectly or failed completely. • Maintenance actions can only be executed during breaks between two adjacent missions. A set of optional maintenance actions (including do nothing, minimal repair, and corrective/preventive replacement) can be selected to be performed on either failed or functioning components. • All the selected maintenance actions are sequentially conducted; that is, maintenance actions cannot be simultaneously executed by multiple repair teams or facilities. • The time durations of the kth mission and the kth break are denoted by z k and L k , respectively. The states and effective ages of all the components at the end of the kth mission are exactly known.

2.3 Decision Variables Maintenance actions can be categorized into two groups: minimal repair and replacement (i.e., perfect maintenance). The associated binary decision variables for component l are denoted by Wl (k) and Ul (k). Minimal repair: After a minimal repair, a failed component will be restored to functioning. However, the component state remains “as bad as old.” That is, minimal repair can only recover the failed component from the failed state to the functioning state without altering the effective age and failure intensity of the component. Whether a minimal repair is performed on a failed component during the kth break is denoted by a decision variable Wl (k). Thus, ⎧ ⎪ ⎨1 if a minimal repair is executed Wl (k) = on component l during the kth break . ⎪ ⎩ 0 otherwise

(2.3)

Replacement: After a replacement, the failed/deteriorated component will be “as good as new”; that is, the component can be considered as a brand-new one. Whether replacing a component during the kth break or not is denoted by a decision variable Ul (k). Thus,

2.4 Probability of a System Successfully Completing a Mission Table 2.1 The efficiency of each type of maintenance action Types of maintenance actions Yl,k Minimal repair on a failed component Replacement on a failed component Replacement on a functioning component

0 0 1

35

X l,k+1

Al,k+1

1 1 1

Bl,k 0 0

⎧ ⎪ ⎨1 if a replacement is executed Ul (k) = on component l during the kth break . ⎪ ⎩ 0 otherwise

(2.4)

The efficiency of each type of maintenance action is quantified by components’ state and age (Table 2.1).

2.4 Probability of a System Successfully Completing a Mission 2.4.1 Survival Probability of a Component Without maintenance actions, the lifetime of a component can be modeled by a random variable with a baseline lifetime distribution. If a failed component is recovered by minimal repair, the component is restored to the functioning state without any change in its effective age. After minimal repair, the time to the next failure follows the conditional distribution, given that the effective age of a component at the beginning of a mission is equal to that of the component at the end of the last mission. The survival probability of component l at the end of the (k + 1)th mission, denoted by rl (z k+1 ), is given by   rl (z k+1 ) = Pr T > Al,k+1 + z k+1  T > Al,k+1 · X l,k+1

 1 − Fl Al,k+1 + z k+1

 = · X l,k+1 , 1 − Fl Al,k+1 where Fl (t) is the lifetime distribution function of component l.

(2.5)

36

2 Basic Selective Maintenance Model

2.4.2 Typical Lifetime Distribution In the reliability engineering, three typical lifetime distributions, i.e., exponential, normal, and Weibull distributions, have widespread applications in characterizing the failure time of components. The characteristics of these baseline lifetime distributions are as follows: Exponential distribution: In probability theory, the exponential distribution is defined as the probability distribution of time between events in the homogenous Poisson process. The most important feature of the exponential distribution is the memoryless property. The baseline lifetime distribution function of an exponential distribution is (2.6) F (t) = 1 − e−λt , where λ is the constant failure rate. Normal distribution: It is also known as Gaussian distribution. A normal distribution is symmetric of its mean; that is, data close to the mean have more occurrence frequency than those far from the mean. The baseline lifetime distribution function of a normal distribution is

t −μ , (2.7) F (t) =  σ where μ is the mean; σ is the standard deviation; (t) is standard normal distribution, of which μ = 0 and σ = 1. Weibull distribution: One of the most widely used lifetime distributions in reliability engineering is Weibull distribution. It is a versatile distribution that can consider the characteristics of other types of distributions using the shape parameter. The baseline lifetime distribution function of the Weibull distribution is β

F (t) = 1 − e−( α ) , t

(2.8)

where α and β are the scale and shape parameters, respectively.

2.4.3 Probability of a System Successfully Completing the Next Mission Series system: Assuming that a system is connected in series, it is failed if a component is failed. The probability of a series system successfully completing the (k + 1)th mission is given by M  rl (z k+1 ). (2.9) R (k + 1) = l=1

2.4 Probability of a System Successfully Completing a Mission

37

Parallel system: Assuming that a system is connected in parallel, it is considered failed if and only if all the components are failed. The probability of a parallelconnected system successfully completing the (k + 1)th mission is given by R (k + 1) = 1 −

M 

(1 − rl (z k+1 )).

(2.10)

l=1

M1 -out-of-M: G system: The M1 -out-of-M: G systems are used when a system requires a high level of reliability with unreliable components. An M-component system that works if and only if at least M1 of the M components is called M1 -outof-M: G system, 1 ≤ M1 ≤ M. If the components in M1 -out-of-M: G system are identical, the probability of the system successfully completing the (k + 1)th mission is given by R (k + 1) =

M  M (rl (z k+1 ))i (1 − rl (z k+1 )) M−i , i

(2.11)

i=M1

M M! . = i!(M−i)! i Apparently, 1-out-of-M: G system is a parallel-connected system, and M-out-ofM: G system is a series-connected system.

where

Serial-parallel system: In real-world applications, components in a system are typically serial-parallel connected. Assuming that a system comprises N subsystems connected in series, Mi components are connected in parallel to form subsystem i, N Mi . The structure of the and the number of components in the system is M = i=1 system is shown in Fig. 2.2. The probability of a system successfully completing the next mission is given by R (k + 1) = φ (r1 (z k+1 ) , r2 (z k+1 ) , ..., r M (z k+1 )) ,

(2.12)

where R(k + 1) can be determined by the survival probabilities of all the components and the structure function φ(·) of the system. For instance, if two components in a system are connected in series, then

Fig. 2.2 Structure of serial-parallel system

Subsystem 1

Subsystem 2

Subsystem N

1

1

1

2

2

2

M1

M2

MN

38

2 Basic Selective Maintenance Model



R (k + 1) = φ (r1 (z k+1 ) , r2 (z k+1 )) =

rl (z k+1 ).

(2.13)

l=1,2

If the two components in the system are connected in parallel, the following is obtained  (2.14) R (k + 1) = φ (r1 (z k+1 ) , r2 (z k+1 )) = 1 − (1 − rl (z k+1 )). l=1,2

The structure of some complex systems (e.g., imperfect switch, common cause failure, and complex network systems) can be found in [2].

2.5 Selective Maintenance Modelling 2.5.1 Constraints of Selective Maintenance Problems All components in the system are desired to be recovered to “as good as new” condition during breaks. However, owing to limited maintenance resources between two adjacent missions, all the desirable maintenance actions cannot be performed during breaks. In general, maintenance time and cost can be regarded as constraints in selective maintenance problems [4]. The two constraints are as follows: Constraint of maintenance time: Let T (k) denote the total maintenance time for all selected maintenance actions during the kth break. T (k) comprises two portions corresponding to two types of maintenance actions: minimal repair and replacement. T (k) is given by (2.15) T (k) = TM (k) + TR (k) , where TM (k) and TR (k) represent the time durations of minimal repair and replacement during the kth break, respectively. The total time duration for minimal repair TM (k) is given by TM (k) =

M 

t M,l Wl (k),

(2.16)

l=1

where t M,l is the time duration of the minimal repair for component l. The total time duration of replacement, that is, TR (k), is given by TR (k) =

M   

t R,l Ul (k) Yl,k + t  R,l Ul (k) 1 − Yl,k , l=1

(2.17)

2.5 Selective Maintenance Modelling

39

where t R,l and t  R,l are the time durations for replacing component l in failed state and functioning state, respectively. Constraint of maintenance cost: In many situations, both time and cost are limited. Let C(k) denote the total maintenance cost of the kth break. Similarly, C(k) is written as C (k) = C M (k) + C R (k) ,

(2.18)

where C M (k) and C R (k) are the costs for minimal repairs and replacements, respectively. The total cost of minimal repairs is given by C M (k) =

M 

c M,l Wl (k),

(2.19)

l=1

where c M,l is the minimal repair cost for component l. The total cost of replacement is given by C R (k) =

M   

c R,l Vl (k) Yl,k + c R,l Vl (k) 1 − Yl,k ,

(2.20)

l=1

where c R,l and c R,l are the replacement time for component l in failed state and functioning state, respectively.

2.5.2 Optimization Models of Selective Maintenance Problems Suppose that the state of each component in the system is known in advance. Selective maintenance aims to identify which components are to be repaired with minimal maintenance and which are to be replaced during the break. By treating the maintenance time and/or budget as constraints, three selective maintenance optimization models can be formulated as follows: Optimization model with time constraint Maximize R (k + 1) ⎧ T (k) ≤ L k ⎪ ⎪ ⎪ ⎨ W (k) + U (k) ≤ 1, l ∈ 1, 2, ..., M , l l Subject to ⎪ Yl,k + Wl (k) ≤ 1, l ∈ 1, 2, ..., M ⎪ ⎪ ⎩ Wl (k) and Ul (k) are binary, l ∈ 1, 2, ..., M

(2.21)

40

2 Basic Selective Maintenance Model

Optimization model with budget constraint Maximize R (k + 1) ⎧ C (k) ≤ Ck ⎪ ⎪ ⎪ ⎨ W (k) + U (k) ≤ 1, l ∈ 1, 2, ..., M , l l Subject to ⎪ Yl,k + Wl (k) ≤ 1, l ∈ 1, 2, ..., M ⎪ ⎪ ⎩ Wl (k) and Ul (k) are binary, l ∈ 1, 2, ..., M

(2.22)

Optimization model with time and budget constraints Maximize R (k + 1) ⎧ T (k) ≤ L k ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ C (k) ≤ Ck , Subject to Wl (k) + Ul (k) ≤ 1, l ∈ 1, 2, ..., M ⎪ ⎪ ⎪ Yl,k + Wl (k) ≤ 1, l ∈ 1, 2, ..., M ⎪ ⎪ ⎪ ⎩ Wl (k) and Ul (k) are binary, l ∈ 1, 2, ..., M

(2.23)

where Ck is the maintenance budget for the kth break. Note that, in these scenarios, we assume that all selected minimal repairs and replacements can only be executed in a sequential manner; that is, maintenance actions are carried out by one repair team/channel. If multiple repair teams/channels are available, selected maintenance actions must be distributed to repair teams/channels and executed simultaneously. The total maintenance time of all repair teams/channels must not exceed the time duration of the kth break.

2.6 Illustrative Example The structure of the illustrative system is presented in Fig. 2.3. The system comprises two s-independent subsystems connected in series, and each subsystem comprises two parallel-connected components. The maintenance budget of the kth break was $6,000. The time duration of the kth break, that is, L k , equaled to 0.8 weeks, whereas that of the (k + 1)th mission, i.e., z k , was 5 weeks. Suppose that the cost and time required to maintain the failed and functioning components are identical. Component 3 is in its failed state, and

Fig. 2.3 System structure

1

3

2

4

2.6 Illustrative Example

41

Table 2.2 Parameter settings of the illustrative example (unit of time: day, unit of cost: $1000) ID αl βl t M,l t R,l c M,l c R,l Bl,k Yl,k 1 2 3 4

15 15 20 20

1.5 1.5 3 3

0.06 0.03 0.05 0.02

0.6 0.3 0.5 0.2

0.2 0.3 0.1 0.4

2 3 1 4

15 20 8 15

1 1 0 1

the remaining components are functioning. Suppose that the first time to failure of component l is governed by a Weibull distribution with the shape parameter αl and scale parameter βl . αl , βl , and the other related parameters are listed in Table 2.2. By Eqs. (2.12)–(2.14), the probability that the system successfully completes the next mission can be written as R (k + 1) = φ (r1 (z k+1 ) , r2 (z k+1 ) , r3 (z k+1 ) , r4 (z k+1 ))         = 1 − 1 − r1 (z k+1 ) 1 − r2 (z k+1 ) 1 − 1 − r3 (z k+1 ) 1 − r4 (z k+1 ) . Because the system has only four components, and each component has only two binary decisions, 28 = 256 possible selective maintenance strategies exist. During breaks, no more than one maintenance action can be performed on a component; thus, only 24 strategies are feasible. The feasible solutions are listed in Table 2.3. If all components in the system are replaced (solution #24 in Table 2.3) during the kth break, $10,000 dollars and 1.6 weeks are required. The probability of successfully completing the next mission, that is, R (k + 1), approaches 0.9691. Owing to the limited time duration of the kth break and maintenance budget, an optimal selective maintenance strategy should be sought to identify a subset of components to be repaired and the corresponding maintenance actions for these selected components. Although the system functions at the end of the kth mission, without any maintenance action during the kth break (solution #1 in Table 2.3), R(k + 1) is only 0.4539 for the next mission. Under the constraints of maintenance budget and time, the maximum probability of mission success is 0.9207, and the corresponding solution #18 is the optimal maintenance strategy, which replaces component 2 and component 3 during the kth break. The total cost and time for implementing the optimal strategy are $4,000 and 0.8 weeks, respectively. If only the cost constraint is omitted, the optimal strategy is the solution #19 in Table 2.3. This strategy indicates that Component 3 is minimally repaired, and Components 2 and 3 are replaced. Suppose that the time constraint is omitted, while the other settings remain unchanged, and the optimal strategy for the example is solution #22 in Table 2.3. This strategy indicates that Components 1, 2, and 3 must be replaced. These maintenance actions consume $91,000 and 1.15 weeks, and the corresponding R(k + 1) is 0.9628.

42

2 Basic Selective Maintenance Model

Table 2.3 Feasible solutions for the illustrative example (unit of time: day, unit of cost: $1000) Solutions Wl (k) Ul (k) R (k + 1) C (k) T (k) 1 2 3 4 1 2 3 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 1 0 1 1 1 0 0 0 1 0 0 1 0 0 0 1 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 1 0 1 0 0 0 0 0 1 0 1 1 0 1 1 1 0 0 0 1 1 1 1

0 0 1 1 0 0 0 0 0 0 1 1 0 1 0 0 0 1 1 1 1 1 1 1

0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1 1 0 1 0 1 0 1

0 0 0 0 0 1 0 1 1 0 0 0 1 1 0 1 1 0 1 1 1 0 1 1

0.4539 0.5160 0.5200 0.5438 0.7416 0.7966 0.8036 0.8067 0.8089 0.8432 0.8497 0.8885 0.9056 0.9126 0.9136 0.9172 0.9196 0.9207 0.9243 0.9268 0.9543 0.9628 0.9665 0.9691

0.0 2.0 3.0 5.0 0.1 4.0 1.0 4.1 5.0 2.1 3.1 5.1 6.0 7.0 3.0 6.1 7.0 4.0 7.1 8.0 9.0 6.0 9.1 10.0

0.00 0.60 0.30 0.90 0.05 0.20 0.50 0.25 0.70 0.65 0.35 0.95 0.80 0.50 1.10 0.85 1.30 0.80 0.55 1.00 1.10 1.40 1.15 1.60

Component 3 is the only failed component of the system. It is predicted that the component should be repaired during the next break. If solution #5 in Table 2.3 is adopted, that is, minimal repair is performed on Component 3, one has R(k + 1) = 0.7416, C(k) = $100 and T (k) = 0.05 weeks. Solution #5 in Table 2.3 can be considered the most cost-effective solution. Furthermore, to investigate the influence of the break duration and maintenance budget on the probability of the system successfully completing the next mission, we changed the time duration of the kth break from 0.4 to 1.6 weeks and the maintenance budget of the kth break from $4,000 to $10,000. The maximum probabilities of the system to successfully complete the next mission under these constraint settings are shown in Fig. 2.4. It can be seen that the maximum achievable R(k + 1) is a nondecreasing function with respect to the time duration and maintenance budget of a break. The greater the time duration and maintenance budget involved, the greater the probability of mission success.

References

43

Probability of successfully completing a mission R(k+1)

0.96 0.94 0.92 0.9 0.88 0.86

Dura

tion o

f brea

k Lk

Fig. 2.4 The probability of the system successfully completing the (k + 1) th mission vs. the settings of the time duration and maintenance budget of the kth break (unit of time: day, unit of cost: $1000)

2.7 Closure In this chapter, the basic mathematical models of the selective maintenance problem were introduced. Two optional repair actions, that is, minimal repair and replacement, were involved as decision variables in the selective maintenance problem. The probability of a system successfully completing the next mission was deduced for four typical system configurations based on the survival probabilities of all constituent components. Three selective maintenance models were formulated with different constraint settings. A four-component serial-parallel system was used to examine the selective maintenance model. The results of the example indicate that the more maintenance resources involved, the larger probability of a system successfully completing the next mission. The model introduced in this chapter was a basic and simple selective maintenance problem. The remaining chapters will expand the basic model from several perspectives as follows: selective maintenance model for multi-state systems, selective maintenance model with loading strategy, selective maintenance model under stochastic time durations, selective maintenance model with imperfect observations, and selective maintenance model executing multiple consecutive missions.

References 1. Alaswad S, Xiang Y (2017) A review on condition-based maintenance optimization models for stochastically deteriorating system. Reliab Eng Syst Saf 157:54–63 2. Birolini A (2017) Reliability & availability of repairable systems. Reliability engineering. Springer, Berlin

44

2 Basic Selective Maintenance Model

3. Cassady CR, Murdock WP, Pohl EA (2001) Selective maintenance for support equipment involving multiple maintenance actions. Euro J Oper Res 129(2):252–258 4. Cassady CR, Pohl EA, Murdock WP (2001) Selective maintenance modeling for industrial systems. J Qual Maintenance Eng 7(2):104–117 5. Iyoob IM, Cassady CR, Pohl EA (2006) Establishing maintenance resource levels using selective maintenance. Eng Econ 51(2):99–114 6. Liu Y, Huang HZ (2010) Optimal selective maintenance strategy for multi-state systems under imperfect maintenance. IEEE Trans Reliab 59(2):356–367 7. Maillart LM, Cassady CR, Rainwater C, Schneider K (2009) Selective maintenance decisionmaking over extended planning horizons. IEEE Trans Reliab 58(3):462–469 8. Pham H, Wang H (1996) Imperfect maintenance. Euro J Oper Res 94(3):425–438 9. Rice W, Cassady C, Nachlas J (1998) Optimal maintenance plans under limited maintenance time. In: Proceedings of the seventh industrial engineering research conference, pp 1–3 10. Schneider K, Cassady CR (2004) Fleet performance under selective maintenance. In: 2004 Annual reliability and maintainability symposium, pp 571–576

Chapter 3

Selective Maintenance for Multi-state Systems under Imperfect Maintenance

3.1 Introduction The traditional selective maintenance optimization reported in the literature only focuses on binary-state systems, which have only two possible states: working perfectly and failed completely. However, most real-world systems possess multiple distinguished performance capacities or damage severities between the two aforementioned extreme states. Systems with multi-state characteristics are termed multi-state systems (MSSs). For example, a system with multiple disks can continue operating if some of these disks fail; however, the storage capacity of the system deteriorates. A power generating system has more than two possible states with different power output levels (e.g., 0, 50, and 80 MW) [1]. Many engineered systems with the following two features can be viewed as MSSs [10]: • A system consisting of different units that have a cumulative performance effect on the entire system [9, 10]. • A system consisting of components with variable performance due to deterioration (fatigue, partial failures, etc.) and repair actions [7, 10]. Selective maintenance problems for MSSs have been investigated in literature. Meng et al. [12] developed a selective maintenance problem for parallel-series MSSs of which the system state is determined by number of functioning components in each subsystem. Liu and Huang [10] and Pandey et al. [14] studied selective maintenance problems for MSSs consisting of binary-state and multi-state components, respectively. Considering the dependence between components, selective maintenance problems for MSSs with economic dependence [6], stochastic dependence [4], and structural dependence [5] have been investigated. A deteriorating system/component can be restored through maintenance activities to ensure the success of subsequent missions. Based on the condition of the system/component after maintenance, most maintenance actions can be categorized as perfect, minimal, and imperfect maintenance. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_3

45

46

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance

• Perfect maintenance/repair: A system/component can be restored to “as good as new” condition via perfect maintenance. For example, replacing a deteriorating component by a brand-new one can be regarded as a perfect maintenance. Cassady et al. [3] proposed a basic selective maintenance model for the case of perfect maintenance. • Minimal maintenance/repair: A failed system/component can be restored to “as bad as old” condition via minimal maintenance (i.e., the system/component is restored to the functioning state and its condition is the same as just before the failure). Cassady et al. [2] introduced minimal maintenance to the selective maintenance problem. • Imperfect maintenance/repair: In many industrial or military environments, a system/component may not always be restored to “as good as new” or “as bad as old” condition, but to somewhere between these two extremes. Such a maintenance activity is referred to as imperfect maintenance. The selective maintenance problem with imperfect maintenance was explored by Liu and Huang [10], and the relation between age reduction coefficient and the consumed maintenance cost was formulated as an exponential function. In the context of selective maintenance problems for binary-state systems, Pandey et al. [15] introduced a hybrid imperfect maintenance model of which the failure intensity and effective age of a component can be changed by imperfect maintenance actions. In the context of selective maintenance problems for multi-state systems with multi-state components, Pandey et al. [14] assumed that a component can be restored to a better state by imperfect maintenance activities. A comprehensive review on selective maintenance models from the imperfect maintenance perspective was reported in [13]. This chapter provided a basic mathematical model for the selective maintenance of MSSs under imperfect maintenance. An MSS consists of multiple binary-capacitated components, and the performance capacity of the entire system is determined by the states of all the components. The Kijima type-II age reduction model was utilized to quantify the maintenance efficiency of imperfect maintenance activities. The probability of the system successfully completing the next mission was evaluated using the universal generating function (UGF) method. Two illustrative examples were presented to demonstrate the effectiveness of the proposed method. The remainder of this chapter was organized as follows. In Section 3.2, the selective maintenance problem for MSSs, together with some basic assumptions, was introduced. In Sect. 3.3, the Kijima type-II age reduction model was reviewed, and the relation between maintenance efficiency and cost was formulated. The probability of a system successfully completing the next mission was evaluated in Sect. 3.4. The resulting selective maintenance model was formulated as a constrained optimization problem in Sect 3.5. Two illustrative examples were presented in Sect. 3.6. Finally, a brief description of the closure was provided in Sect. 3.7.

3.2 Problem Statements and Model Assumptions

47

3.2 Problem Statements and Model Assumptions A multi-component system with binary-capacitated components intends to complete a sequence of missions with a break between two adjacent missions. The break after the kth mission is denoted by the kth break. After executing the kth mission, some components of the system fail or deteriorate at the end of the kth mission. To maximize the probability of the system successfully completing the next mission, that is, the (k + 1)th mission, the condition of the system/components can be restored via maintenance activities, which can be executed at the kth break. However, only a subset of maintenance actions can be selected, owing to the limited maintenance budget in the kth break. The basic assumptions of the selective maintenance model are as follows [2, 10, 11]: • A system consists of M s-independent components. Each component may propose two states, that is, working perfectly (with nominal performance capacity) and failed completely (with zero performance capacity). The performance capacities of component l are denoted by gl,1 = 0 and gl,2 = 0 for failed state and functioning state, respectively. The performance capacity of component l at any time instant t (≥ 0) is a random variable, denoted by G l (t), and one has G l (t) ∈ {gl,1 , gl,2 }. • The state of component l at the end of the kth mission is denoted by Yl,k which is a binary variable, and one has Yl,k

 1 if component l is functioning = . 0 if component l is failed

As maintenance activities cannot be conducted during a mission, the relation X l,k ≥ Yl,k always holds. The failure intensity function of component l at time instant t without any maintenance is denoted by λl (t), and the survival probability of component l at time instant t is denoted by Rl (t), where Rl (t) = t exp(− 0 λl (t)dt). • The performance capacity of a system at any time instant t is denoted by G(t), which is unambiguously determined by the performance capacities of its components and the system configuration, such as series-parallel, bridge, and network. The system configuration remains unchanged throughout all the missions. The performance capacity of the system at time instant t is determined by G(t) = φ(G 1 (t), G 2 (t), . . . , G M (t)),

(3.1)

where φ(·) is the structure function which represents the relation between the performance capacities of components and that of the system. For example, in the case of a flow transmission system with two components connected in series, the structure function of the system takes the form φ(G 1 (t), G 2 (t)) = min{G 1 (t), G 2 (t)}, and one has φ(G 1 (t), G 2 (t)) = G 1 (t) + G 2 (t) in the case of a system with two

48

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance

Table 3.1 The performance capacities of flow transmission systems with two components connected in series and parallel, respectively G 1 (t) G 2 (t) G(t) Series system Parallel system 0 tons/h (Failed) 0 tons/h (Failed) 500 tons/h (Functioning) 500 tons/h (Functioning)

0 tons/h (Failed) 500 tons/h (Functioning) 0 tons/h (Failed)

0 tons/h 0 tons/h

0 tons/h 500 tons/h

0 tons/h

500 tons/h

500 tons/h (Functioning)

500 tons/h

1000 tons/h

parallel components. Specifically, the performance capacities of a specific flow transmission system in these two scenarios are listed in Table 3.1, where the performance capacities of each component in failed state and functioning state are 0 tons/h and 500 tons/h, respectively. The set of performance capacities of a system is denoted by G = {g1 , g2 , . . . , g Ns }, where gi is the ith performance capacity in set G, and Ns is the number of possible performance capacities of the system. For example, Table 3.1 shows that the set of the performance capacities of the series system is 0, 500 tons/h, whereas that of the parallel system is 0, 500, 1000 tons/h. • During each break between two adjacent missions, the condition of a component can be restored by a maintenance action, which is selected from a set of optional maintenance actions, including doing nothing, minimal repair, corrective/preventive replacement, imperfect corrective/preventive maintenance. The break after the kth mission is denoted by the kth break, and the maintenance budget of the kth break is denoted by Ck . • The time duration of the kth mission is denoted by z k . The states and effective ages of all components at the end of the kth mission are known in advance. A failed component will not continue to degenerate in a mission (i.e., a components’effective age will immediately stop increasing if it fails in the middle of a mission). • The system is required to meet a pre-specified mission demand, and the demand of the kth mission is denoted by Wk . Hence, a mission is successfully completed if the performance capacity of the system at the end of the mission is greater than mission demand.

3.3 Imperfect Maintenance and Its Cost In the literature on maintenance modelling, a component is typically assumed to be restored to a condition either “as good as new” or “as bad as old” prior to failure. However, in most industry practices, such an assumption is unrealistic, and a

3.3 Imperfect Maintenance and Its Cost

49

component can be restored to a condition somewhere between the two extreme states. This type of maintenance is referred to as imperfect maintenance. Imperfect maintenance models with the age-reduction concept have been extensively utilized to quantify efficiency of maintenance actions. The condition of a binary-state component can be characterized using the concept of effective age. If the effective age of a component after maintenance is t, the condition of the component is exactly the same as the condition of a functioning component that has been working for the time duration of t. Therefore, the condition of the component is not determined by chronological time. Let Al,k and Bl,k represent the effective age of component l at the beginning and end of the kth mission, respectively, and one has Bl,k = Al,k + z k if component l continues to function throughout the kth mission. In this section, the Kijima type-II age reduction model [8] is used to quantify the efficiency of imperfect maintenance actions. A component may not be restored to “as good as new” or “as bad as old” conditions after imperfect maintenance, but it becomes younger. Based on the Kijima type-II age reduction model, the effective age of component l after maintenance in the kth break is given by Al,k+1 = bl,k Bl,k ,

(3.2)

where bl,k (0 ≤ bl,k ≤ 1) is the age reduction factor corresponding to the efficiency of the maintenance action for component l in the kth break. A smaller bl,k represents a greater improvement, as shown in Fig. 3.1, and three illustrative cases are given as follows In Case I, at the end of the first mission, the component sojourns in the functioning state and doing nothing is selected. Hence, age reduction factor bl,1 = 1.

Fig. 3.1 The chronological time versus the effective age in the Kijima type II model

50

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance

The component fails in the second mission and its effective age is reduced by performing an imperfect maintenance action with bl,2 = 0.6 in the second break. In Case II, the component broke down on day 4, and its effective age at the end of the first mission remained at 4 d. In the first break, minimal repair with an age reduction factor bl,1 = 1 is selected for the component. The component is only repaired to functioning state just before failure (i.e., “as bad as old”), and its effective age is not changed. In the second break of Case II, the component is replaced with bl,2 = 0; that is, it is restored to “as good as new” condition with effective age Al,3 = 0. In Case III, the component sojourns in the failure state at the end of the first mission, and doing nothing is selected in the first break; that is, bl,1 = 1. The component sojourns in the failure state and its effective age will not cumulate during the second mission. In the second break, the effective age of the component is reduced to one day by executing imperfect maintenance with bl,2 = 0.3. During the kth break, N L + 1 optional maintenance actions can be selected for each component, and the maintenance action selected for component l in the kth break is denoted by al,k (al,k ∈ {0, 1, . . . , N L }). Let al,k = 0 and al,k = N L represent replacement and doing nothing, respectively. Generally, maintenance actions with better maintenance efficiency consume more maintenance costs. The maintenance cost for component l in the kth break can be formulated as  0 al,k = 0 , (3.3) Cl,k = al,k = 1, 2, . . . N L cl (al,k , Yl,k ) + cl0 where cl0 is the minimal maintenance cost for component l, i.e., the maintenance cost of restoring component l to the functioning state prior to failure; cl (al,k , Yl,k ) represents the maintenance cost of improving the condition of component l, and it is determined by the selected maintenance action and the state of component l, i.e., al,k and Yl,k . rp Let clrf and cl represent the costs of corrective and preventive replacement for component l, respectively. Based on the state of a component before maintenance, the maintenance efficiency, that is, the age reduction factor in the Kijima type-II age reduction model, is defined as a function of the maintenance cost as follows ⎧ al,k = 0 ⎪ ⎪1  ⎪ 1 ⎨ cl (al,k ,0) mlf al,k > 0, Yl,k = 0 , bl,k = 1 − (3.4) clrf ⎪ 1  ⎪ p ⎪ ⎩1 − cl (al,krp ,1) ml a > 0, Y = 1 l,k l,k c l

where m lrf is a characteristic parameter that determines the relation between corrective maintenance cost and age reduction factor, and it can be estimated by history rp maintenance data and expert experience. Similarly, m l is a characteristic parameter with respect to preventive maintenance actions. The evolutions of the age reduction factor with respect to cl (al,k , 0)/clrf with different m lrf values is presented in

3.4 Probability of a System Successfully Completing a Mission

51

Fig. 3.2 The plot of bl,k versus cl (al,k , 0)/clrf for different m lf

Fig. 3.2. Specifically, if al,k > 0, Yl,k = 0, and cl (al,k , 0) = 0, the age reduction factor bl,k = 1 (i.e., component l) is restored to the functioning state, but its effective age is not reduced by the selected maintenance action. Hence, the selected maintenance action is minimal repair and the associated maintenance cost Cl,k = cl0 . Based on the selected maintenance action for each component, the maintenance cost of the entire system at the kth break can be evaluated as follows sys

Ck =

M

Cl,k .

(3.5)

l=1

3.4 Probability of a System Successfully Completing a Mission If component l functions at the beginning of the (k + 1)th mission, the probability of component l sojourning in the functioning state at the end of the (k + 1)th mission, that is, the conditional survival probability of component l at the end of the (k + 1)th mission, denoted by rl (z k+1 ), can be evaluated as follows rl (z k+1 ) =

Rl (Al,k+1 + z k+1 ) , Rl (Al,k+1 )

(3.6)

52

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance

where Al,k+1 is the effective age of component l at the beginning of the (k + 1)th mission; z k+1 is the duration of the (k + 1)th mission; Rl (·) is the reliability function of component l. For example, if the failure distribution of component l follows the Weibull distribution with scale parameter ηl and shape parameter βl , whereas the component functions at the beginning of the (k + 1)th mission with effective age Al,k+1 , the probability of component l surviving at the end of the (k + 1)th mission is Rl (Al,k+1 + z k+1 ) Rl (Al,k+1 ) 

 Al,k+1 βl Al,k+1 + z k+1 βl + = exp − . ηl ηl

rl (z k+1 ) =

(3.7)

The probability of the system successfully completing the (k + 1)th mission can be evaluated via the UGF method, which is an effective method for evaluating the probability distribution of the performance capacities of an MSS using the probability distribution of components. UGF of the binary-capacitated component l at the end of the (k + 1)th mission can be represented as u l,k+1 =

2

pl,k+1 (i) · z gl,i

i=1

(3.8)

= (1 − rl (z k+1 ))z gl,1 + rl (z k+1 )z gl,2 , where pl,k+1 (i) (i = 1, 2) represents the probability of component l sojourning in state i at the end of the (k + 1)th mission, wherein pl,k+1 (1) = 1 − rl (z k+1 ) and pl,k+1 (2) = rl (z k+1 ). The UGF of the system at the end of the (k + 1)th mission, denoted by Uk+1 , can be written as M  2 2



 φ(g1,i1 ,...,g M,i M ) Uk+1 = ... pl,k+1 (il ) · z i 1 =1

=

Ns

i M =1

l=1

sys pk+1 (i)

·z ,

(3.9)

gi

i=1 sys

where pk+1 (i) is the probability of the system performance capacity being equal to gi at the end of the (k + 1)th mission; φ(·) is the structure function of the system. For example, a flow transmission system consists of two parallel components, and the performance capacity of each component in the functional state is 500 tons/h. Given the probability of component l (l = 1, 2) in the functioning state at the end of the (k + 1)th mission is rl (z k+1 ) = 0.7, the UGF of component l can be written as u l,k+1 = (1 − rl (z k+1 ))z gl,1 + rl (z k+1 )z gl,2 = 0.3z 0 + 0.7z 500 .

(3.10)

3.5 Selective Maintenance Modelling

53

The structure function of the system is φ(G 1 (t), G 2 (t)) = G 1 (t) + G 2 (t) as shown in Sect. 3.2. The UGF of the system at the end of the (k + 1)th mission can be evaluated as  2  2

2

 φ(g1,i1 ,g2,i2 ) pl,k+1 (il ) · z Uk+1 = i 1 =1 i 2 =1

l=1

= 0.3 × 0.3z 0+0 + 0.3 × 0.7z 0+500 + 0.7 × 0.3z 500+0 + 0.7 × 0.7z 500+500 = 0.09z 0 + 0.42z 500 + 0.49z 1000 sys sys sys = pk+1 (1) · z g1 + pk+1 (2) · z g2 + pk+1 (3) · z g3 . (3.11) Hence, the state distribution of the system at the end of the (k + 1)th mission can be written as sys

sys

sys

pk+1 (1) = 0.09, pk+1 (2) = 0.42, pk+1 (3) = 0.49.

(3.12)

Based on the state distribution of the system at the end of the (k + 1)th mission, the probability of the system successfully completing the (k + 1)th mission can be evaluated by R(k + 1) =

Ns

sys

pk+1 (i) · 1(gi ≥ Wk+1 ),

(3.13)

i=1

where 1(gi ≥ W ), which is an indicator function, is defined by  1 gi ≥ Wk+1 1(gi ≥ Wk+1 ) = . 0 gi < Wk+1

(3.14)

The indicator function equals one if the system performance capacity satisfies the demand of the (k + 1)th mission.

3.5 Selective Maintenance Modelling At the kth break, based on the state and effective age of each component, that is, X l,k and Bl,k , the maintenance action for each component, that is, al,k , was identified to maximize the probability of the system successfully completing the (k + 1)th mission, that is, R(k + 1). Consequently, the selective maintenance of MSSs with imperfect maintenance can be formulated as follows

54

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance

max R(k + 1) = al,k

sys

s.t. Ck ≤ Ck

Ns  i=1

sys

pk+1 (i) · 1(gi ≥ Wk+1 )

.

(3.15)

al,k = 0, 1, . . . , N L where Ck is the maintenance budget in the kth break. The resulting selective maintenance problem is a nonlinear combinatorial optimization model. For small-scale systems with several components, the selective maintenance problem can be solved by enumerating all possible maintenance strategies. However, because N L + 1 optional maintenance actions can be selected for each component, the number of possible solutions for the entire system becomes (N L + 1) M , which increases exponentially with respect to the number of components in a system. To overcome this challenge, meta-heuristic algorithms (e.g., genetic algorithm (GA), ant colony optimization (ACO), particle swarm optimization (PSO), tabu search, and simulated annealing algorithms) can be utilized to search for the global optimal solution (or approximate global optimal solution). A GA was implemented to resolve the resulting problem.

3.6 Illustrative Examples 3.6.1 A Three-Component System A three-component flow transmission system is described in this section to illustrate a selective maintenance strategy with imperfect maintenance, as shown in Fig. 3.3. The performance capacity of the system is determined by G(t) = min{G 1 (t) + G 2 (t), G 3 (t)}. In the kth break, N L + 1 maintenance actions are available for each component and the associated maintenance costs are given in Eq. (3.3), where cl (al,k , Yl,k ) is the cost of maintenance action al,k for component l sojourning in state Yl,k is defined as follows  (a −1) l,k crf Yl,k = 0 N L −1 l cl (al,k , Yl,k ) = al,k , (3.16) rp c Yl,k = 1 NL l where Yl,k = 0 and Yl,k = 1 represent corrective and preventive maintenance, respectively. In this example, N L is set to three, that is, four optional maintenance actions including doing nothing are available for each component.

Fig. 3.3 A three-component system

1 3 2

3.6 Illustrative Examples

55

Table 3.2 Parameters of components (unit of time: week, unit of cost: $1000, unit of performance capacity: tons/hour) ID (l )

gl,2

ηl

βl

ml

p

m lf

cl

rp

clrf

cl0

Bl,k

Yl,k

1

55

50

1.9

2.5

2.5

10

20

2

30

1

2

48

58

2.4

2.2

2.2

15

30

2

20

0

3

45

48

2.6

0.7

0.8

26

60

5

15

0

The failure intensity function of component l with effective age t, that is, λl (t), is defined as λl (t) =

βl ηl



t ηl

βl −1

,

(3.17)

where ηl and βl are the scale and shape parameters of failure intensity function of component l, respectively. The probability that component l survives at the end of the (k + 1)th mission can be evaluated using Eq. (3.7). The parameters of each component, including the cost of corrective and preventive replacement, performance capacity, parameters of the failure intensity function, and state and effective age after the kth mission, are tabulated in Table 3.2. The time duration and demand of the (k + 1)th mission were 5 weeks and 60 tons/h, respectively. The maintenance budget in the kth break is $70,000. To maximize the probability of the system successfully completing the (k + 1)th mission, the optimal maintenance strategy in the kth break can be determined by enumerating all possible solutions, as shown in Table 3.3. The optimal maintenance strategy is Solution #45. Component #1 is subjected to preventive replacement, which renews the aged component. Components #2 and #3 are recovered to their functioning states through imperfect repairs. The probability of the system successfully completing the (k + 1)th mission is 0.948, and the corresponding maintenance cost of the kth break is $64,000. The maintenance strategies without imperfect maintenance activities, that is, the maintenance strategy that involves only doing nothing, minimal repair, and replacement, can be found in Table 3.3. For the optimal maintenance strategy without imperfect maintenance (Solution #47 in Table 3.3), Components #1 and #2 were replaced and restored to the best condition, and minimal repair was selected for Component #3. The probability of the system successfully completing the (k + 1)th mission is 0.933, which is less than that of the scenario with imperfect maintenance, and the maintenance cost of the kth break is $49,000. The optimal maintenance strategies, for which the maintenance budgets are $0, $10,000, and $110,000, respectively, are presented in Table 3.4. If the maintenance budget of the kth break is $10,000, the system will not be repaired in the kth break, and the system cannot complete the (k + 1)th mission. If the maintenance budget of the kth break is $10,000, the failed components (i.e., Components #2 and #3) are restored to functioning by minimal maintenance, and the probability of the system

56

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance

Table 3.3 Feasible maintenance strategies (unit of cost: $1000) sys

sys

ID

a1,k

a2,k

a3,k

Ck

R(k + 1)

ID

a1,k

a2,k

a3,k

Ck

R(k + 1)

1

0

0

0

0

0

25

1

3

1

42.33

0.892

2

0

0

1

5

0

26

2

0

0

8.67

0

3

0

0

2

35

0

27

2

0

1

13.67

0

4

0

0

3

65

0

28

2

0

2

43.67

0

5

0

1

0

2

0

29

2

1

0

10.67

0

6

0

1

1

7

0.788

30

2

1

1

15.67

0.868

7

0

1

2

37

0.810

31

2

1

2

45.67

0.892

8

0

1

3

67

0.830

32

2

2

0

25.67

0

9

0

2

0

17

0

33

2

2

1

30.67

0.905

10

0

2

1

22

0.821

34

2

2

2

60.67

0.930

11

0

2

2

52

0.844

35

2

3

0

40.67

0

12

0

3

0

32

0

36

2

3

1

45.67

0.915

13

0

3

1

37

0.830

37

3

0

0

12

0

14

0

3

2

67

0.854

38

3

0

1

17

0

15

1

0

0

5.33

0

39

3

0

2

47

0

16

1

0

1

10.33

0

40

3

1

0

14

0

17

1

0

2

40.33

0

41

3

1

1

19

0.885

18

1

1

0

7.33

0

42

3

1

2

49

0.910

19

1

1

1

12.33

0.847

43

3

2

0

29

0

20

1

1

2

42.33

0.870

44

3

2

1

34

0.923

21

1

2

0

22.33

0

45

3

2

2

64

0.948

22

1

2

1

27.33

0.882

46

3

3

0

44

0

23

1

2

2

57.33

0.907

47

3

3

1

49

0.933

24

1

3

0

37.33

0

Table 3.4 Optimal maintenance strategies with different maintenance budget (unit of cost: $1000) ID (l) Ck = 0 Ck = 10 Ck = 110 al,k rl (z k+1 ) al,k rl (z k+1 ) al,k rl (z k+1 ) 1 2 3 sys Ck R(k + 1)

0 (DN) 0 (DN) 0 (DN) 0 0

0.879 0 0

0 (DN) 1 (MR) 1 (MR) 7 0.788

0.879 0.947 0.947

3 (PR) 3 (CR) 3 (CR) 109 0.982

0.988 0.997 0.997

DN doing nothing, MR minimal repair, CR corrective replacement, PR preventive replacement

successfully completing the (k + 1)th mission is 0.788. In the scenario with a maintenance budget of $110,000, all components can be replaced, and the probability of the system successfully completing the (k + 1)th mission is 0.982. The probability of the system successfully completing the (k + 1)th mission increases with a higher maintenance budget.

3.6 Illustrative Examples

57

Table 3.5 Optimal maintenance strategy with different demand (unit of cost: $1000, unit of demand: tons/hour) ID (l) Wk+1 = 20 Wk+1 = 40 Wk+1 = 60 al,k rl (z k+1 ) al,k rl (z k+1 ) al,k rl (z k+1 ) 1 2 3 sys Ck R(k + 1)

0 (DN) 1 (MR) 3 (CR) 67 0.991

0.879 0.947 0.997

0 (DN) 3 (CR) 2 (ICR) 67 0.971

0.879 0.997 0.974

3 (PR) 2 (ICR) 2 (ICR) 60 0.922

0.988 0.986 0.974

DN doing nothing, MR minimal repair, ICR imperfect corrective repair, IPM imperfect preventive maintenance, CR corrective replacement, PR preventive replacement

By setting the demand for the (k + 1) mission to 20, 40, and 60 tons/h, the optimal maintenance strategies are presented in Table 3.5. Table 3.5 indicates that the probability of the system successfully completing the (k + 1)th mission decreases with an increase in demand.

3.6.2 A Multi-state Coal Transportation System In this section, a fourteen-component coal transportation system, as shown in Fig. 3.4, is exemplified to demonstrate the effectiveness of the proposed method for a largerscale problem. The system consists of five subsystems: Feeder #1, Conveyor #1, Stacker Reclaimer, Feeder #2, and Conveyor #2, and coals are transmitted through these five subsystems. The performance capacity of the system can be determined by the performance capacities of all the components, as follows G(t) = min{G 1 (t) + G 2 (t) + G 3 (t), G 4 (t) + G 5 (t), G 6 (t) + G 7 (t) + G 8 (t), G 9 (t) + G 10 (t), G 11 (t) + G 12 (t) + G 13 (t) + G 14 (t)}.

(3.18)

In the kth break, eight maintenance actions are available for each component (N L = 7), and the maintenance cost of each maintenance action is given by Eqs. (3.3) and (3.16). The failure intensity function of each component is given by Eq. (3.17). The parameters for each component are listed in Table 3.6. The time duration and demand of the (k + 1)th mission were 10 weeks and 65 tons/h, respectively. The maintenance budget for the kth break was $220,000. As eight optional maintenance actions can be selected for each component, the number of possible solutions without budget constraints is (N L + 1) M = 814 = 4.398×1012 , and it is impossible to enumerate all the possible solutions, as shown in Sect. 3.6.1. Alternatively, GA, as one of the most widely used meta-heuristic

58

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance 11 6

1 4

9

12

10

13

7

2 5

8

3

14 Feeder #1 (Subsystem 1)

Conveyor #1 Stacker-reclaimer Feeder #2 (Subsystem 2) (Subsystem 3) (Subsystem 4)

Conveyor #2 (Subsystem 5)

Fig. 3.4 The configuration of a multi-state coal transportation system Table 3.6 Parameters of components (unit of time: week, unit of cost: $1000, unit of performance capacity: tons/hour) p rp ID (l) gl,2 ηl βl ml m lf cl clrf cl0 Bl,k Yl,k 1 2 3 4 5 6 7 8 9 10 11 12 13 14

80 85 90 100 120 75 90 85 93 110 55 70 82 98

28 35 30 38 27 35 23 31 27 30 42 36 33 44

1.6 2.3 1.7 2.5 1.9 2.3 2.6 1.9 1.3 1.3 2.9 1.4 2.5 2.1

2.2 2.6 2.8 3.2 2.5 2 2.3 2.8 2.4 1.8 2.5 2.2 2.6 2.2

3 2.2 2.8 2.6 4 3.2 3 2.8 2.5 2.8 2.5 2 3 3.2

16 19 27 18 26 14 26 18 19 19 24 23 20 15

26 32 34 35 34 21 29 35 28 36 32 33 36 38

3.3 4.4 3.4 5.7 2.6 3.3 6.5 5.5 3.5 6.3 7.4 4.5 6.3 3.6

32 21 42 32 25 33 41 25 35 12 27 19 35 32

1 0 0 0 1 1 0 0 1 0 0 1 1 0

algorithms in solving combinatorial optimization problems, can be utilized to seek the optimal maintenance strategy. GA is inspired by natural selection and genetics [16]. A possible solution to a specific problem is represented/encoded by a vector (the so-called chromosome), and the elements in the vector are so-called genes. A possible solution represents an individual in the GA. At the beginning of the iterations, a population of possible solutions (individuals) was initialized randomly. Then, similar to the process of natural genetics, these individuals produce children (i.e., new possible solutions) by crossover and mutation to search for more possible solutions. The fitness function of the GA is defined based on the objective function of the specific problem, and an

3.6 Illustrative Examples

59

individual with a better fitness value has a greater chance of surviving in the population and of producing children. Such a process of producing children and choose “fitter” individuals is an iteration of the GA. Because of the Darwinian theory of “survival of the fittest,” the solutions keep “evolving” over iterations and the optimal or global optimal solution (or approximate global optimal solution) can be eventually identified. The solution representation, that is, encoding chromosomes, is a crucial procedure for implementing the GA for the selective maintenance problem. In this example, there are eight optional maintenance actions can be chosen for each component, so the chromosome of a possible solution can be represented by the vector of maintenance actions of all components, and the chromosome can be represented by an integral string ak = (a1,k , a2,k , . . . , a M,k ),

(3.19)

where al,k (al,k = 0, 1, . . . , N L ) is the maintenance action selected for component l in the kth break. The resulting selective maintenance problem was resolved using GA. The fitness function of the GA is defined as  sys R(k + 1) Ck ≤ Ck . (3.20) Fitness(ak ) = sys 0 Ck > Ck Population size is often in the range [50, 200]. Ten elite individuals, which have the best fitness values in each generation, are selected to survive in the next generation without crossover and mutation. In each generation, 80% of the population will be crossovered. GA is terminated if the average relative variation of the best fitness value over 50 generations is not greater than 10, or the number of generations reaches 1400 (100×the number of decision variables). In the mutation operation, each gene in the chromosomes adds a random value that follows a normal distribution with mean 0 and standard deviation, where I is the generation index. The pseudo-code of the GA in this section is provided in Algorithm 3.1. Table 3.7 presents the optimal maintenance strategy resolved by the GA. The maximum probability of the system successfully completing the (k + 1)th mission is 0.907, and the corresponding maintenance cost is $219,662, which is very close to the maintenance budget. Optimal maintenance strategies without imperfect maintenance activities can be resolved by the GA, as presented in Table 3.8. The probability of the system successfully completing the (k + 1)th mission is 0.899, which is less than that of the scenario with imperfect maintenance. The maintenance cost of the kth break is $217,300. The optimal maintenance strategies, for which the maintenance budgets is $150,000, $250,000, and $500,000, respectively, are presented in Table 3.9. If the maintenance budget of the kth break is $150,000, the probability of the system successfully completing the (k + 1)th mission is 0.860. In the scenario with a maintenance budget of $500,000, all components can be replaced, and the probability of

60

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance

Table 3.7 Optimal maintenance strategy (unit of time: week, unit of cost: $1000) ID (l) Bl,k Yl,k al,k Cl,k Al,k+1 X l,k+1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 sys Ck R(k + 1)

32 21 42 32 25 33 41 25 35 12 27 19 35 32 219.662 0.907

1 0 0 0 1 1 0 0 1 0 0 1 1 0

7 (PR) 4 (ICR) 3 (ICR) 7 (CR) 2 (IPM) 7 (PR) 4 (ICR) 1 (MR) 7 (PR) 3 (ICR) 0 (DN) 0 (DN) 7 (PR) 1 (MR)

19.3 20.4 14.73 40.7 10.03 17.3 21 5.5 22.5 18.3 0 0 26.3 3.6

0 5.676 13.631 0 9.854 0 8.458 25 0 3.895 27 19 0 32

1 1 1 1 1 1 1 1 1 1 0 1 1 1

rl (z k+1 ) 0.825 0.867 0.667 0.965 0.664 0.946 0.613 0.552 0.760 0.743 0 0.719 0.951 0.674

DN doing nothing, MR minimal repair, ICR imperfect corrective repair, IPM imperfect preventive maintenance, CR corrective replacement, PR preventive replacement Table 3.8 Optimal maintenance strategy without imperfect maintenance (unit of time: week, unit of cost: $1000) ID (l) Bl,k Yl,k al,k Cl,k Al,k+1 X l,k+1 rl (z k+1 ) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 sys Ck R(k + 1)

32 21 42 32 25 33 41 25 35 12 27 19 35 32 217.3 0.899

1 0 0 0 1 1 0 0 1 0 0 1 1 0

7 (PR) 7 (CR) 1 (MR) 7 (CR) 0 (DN) 7 (PR) 0 (DN) 1 (MR) 7 (PR) 7 (CR) 0 (DN) 0 (DN) 7 (PR) 1 (MR)

19.3 36.4 3.4 40.7 0 17.3 0 5.5 22.5 42.3 0 0 26.3 3.6

0 0 42 0 25 0 41 25 0 0 27 19 0 32

1 1 1 1 1 1 0 1 1 1 0 1 1 1

0.825 0.946 0.460 0.965 0.462 0.946 0 0.552 0.760 0.787 0 0.719 0.951 0.674

DN doing nothing, MR minimal repair, CR corrective replacement, PR preventive replacement

3.6 Illustrative Examples

61

Algorithm 3.1 The pseudo-code of the GA Inputs: Imax : Maximum number of generations; I : Index of generation; N E : Number of elitem; rc : Crossover rate. 1: Randomly initialize population; 2: For IGA = 1 to Imax do 3: Chose N E elite individuals; 4: Crossover rc of population; 5: Mutation operation, al,k = al,k + N L (Imax − IGA )/Imax ; 6: If the best fitness value over 50 generations ≤ 10−6 then 7: Break 8: End If 9: End For Output: The optimal individual. Table 3.9 Optimal maintenance strategies with different maintenance budget (unit of cost: $1000) ID (l )

Ck = 150

Ck = 250

Ck = 500

al,k

rl (z k+1 )

al,k

rl (z k+1 )

al,k

rl (z k+1 )

1

0 (DN)

0.509

7 (PR)

0.825

7 (PR)

0.825

2

4 (ICR)

0.867

7 (CR)

0.945

7 (CR)

0.945

3

2 (ICR)

0.609

1 (MR)

0.460

7 (CR)

0.857

4

7 (CR)

0.965

7 (CR)

0.965

7 (CR)

0.965

5

0 (DN)

0.462

6 (IPM)

0.824

7 (PR)

0.859

6

7 (PR)

0.946

7 (PR)

0.946

7 (PR)

0.945

7

0 (DN)

0

0 (DN)

0

7 (CR)

0.892

8

2 (ICR)

0.703

2 (ICR)

0.703

7 (CR)

0.890

9

7 (PR)

0.760

7 (PR)

0.760

7 (PR)

0.760

10

1 (MR)

0.695

7 (CR)

0.787

7 (CR)

0.787

11

0 (DN)

0

0 (DN)

0

7 (CR)

0.985

12

0 (DN)

0.719

0 (DN)

0.719

7 (PR)

0.847

13

4 (IPM)

0.848

7 (PR)

0.951

7 (PR)

0.951

14

1 (MR)

0.674

1 (MR)

0.674

7 (CR)

0.956

sys

Ck

148.929

248.019

455.3

R(k + 1)

0.860

0.919

0.942

DN doing nothing, MR minimal repair, ICR imperfect corrective repair, IPM imperfect preventive maintenance, CR corrective replacement, PR preventive replacement

the system successfully completing the (k + 1)th mission is 0.942. The probability of the system successfully completing the (k + 1)th mission increases with a higher maintenance budget. By setting the demand of the (k + 1) mission to 35, 65, and 95 tons/h, the optimal maintenance strategies are presented in Table 3.10. As shown in Table 3.10, the probability of the system successfully completing the (k + 1)th mission decreases with an increase in demand.

62

3 Selective Maintenance for Multi-state Systems under Imperfect Maintenance

Table 3.10 Optimal maintenance strategies with different maintenance budget (unit of cost: $1000) ID (l )

Ck = 150

Ck = 250

Ck = 500

al,k

rl (z k+1 )

al,k

rl (z k+1 )

al,k

rl (z k+1 )

1

0 (DN)

0.509

7 (PR)

0.825

7 (PR)

0.825

2

4 (ICR)

0.867

7 (CR)

0.945

7 (CR)

0.945

3

2 (ICR)

0.609

1 (MR)

0.460

7 (CR)

0.857

4

7 (CR)

0.965

7 (CR)

0.965

7 (CR)

0.965

5

0 (DN)

0.462

6 (IPM)

0.824

7 (PR)

0.859

6

7 (PR)

0.946

7 (PR)

0.946

7 (PR)

0.945

7

0 (DN)

0

0 (DN)

0

7 (CR)

0.892

8

2 (ICR)

0.703

2 (ICR)

0.703

7 (CR)

0.890

9

7 (PR)

0.760

7 (PR)

0.760

7 (PR)

0.760

10

1 (MR)

0.695

7 (CR)

0.787

7 (CR)

0.787

11

0 (DN)

0

0 (DN)

0

7 (CR)

0.985

12

0 (DN)

0.719

0 (DN)

0.719

7 (PR)

0.847

13

4 (IPM)

0.848

7 (PR)

0.951

7 (PR)

0.951

14

1 (MR)

0.674

1 (MR)

0.674

7 (CR)

0.956

sys

Ck

148.929

248.019

455.3

R(k + 1)

0.860

0.919

0.942

DN doing nothing, MR minimal repair, ICR imperfect corrective repair, IPM imperfect preventive maintenance, CR corrective replacement, PR preventive replacement

3.7 Closure The selective maintenance problem of MSSs considering imperfect maintenance was investigated in this chapter. Efficiency of imperfect maintenance actions was formulated using the Kijima type-II age reduction model. The relationship between the efficiency of a maintenance action and its cost was formulated as an exponential function. The probability of the system successfully completing the next mission was evaluated using the UGF method. The resulting selective maintenance problem was formulated as a nonlinear combinatorial optimization problem. As shown in the two illustrative examples, the proposed selective maintenance strategies with imperfect maintenance can yield better results than those without imperfect maintenance actions.

References 1. Billington R, Allan R (1996) Reliability evaluation of power systems. Plenum Press, London 2. Cassady CR, Murdock WP, Pohl EA (2001) Selective maintenance for support equipment involving multiple maintenance actions. Euro J Oper Res 129(2):252–258 3. Cassady CR, Pohl EA, Murdock WP (2001) Selective maintenance modeling for industrial systems. J Qual Maintenance Eng 7(2):104–117

References

63

4. Dao CD, Zuo MJ (2015) Selective maintenance for multistate series systems with s-dependent components. IEEE Trans Reliab 65(2):525–539 5. Dao CD, Zuo MJ (2017) Selective maintenance of multi-state systems with structural dependence. Reliab Eng Syst Saf 159:184–195 6. Dao CD, Zuo MJ, Pandey M (2014) Selective maintenance for multi-state series-parallel systems under economic dependence. Reliab Eng Syst Saf 121:240–249 7. Jiang T, Liu Y, Zheng YX (2019) Optimal loading strategy for multi-state systems: cumulative performance perspective. Appl Math Model 74:199–216 8. Kijima M (1989) Some results for repairable systems with general repair. J Appl Prob 26(1):89– 102 9. Lisnianski A, Frenkel I, Karagrigoriou A (2017) Recent advances in multi-state systems reliability: theory and applications. Springer, London 10. Liu Y, Huang HZ (2010) Optimal selective maintenance strategy for multi-state systems under imperfect maintenance. IEEE Trans Reliab 59(2):356–367 11. Liu Y, Chen Y, Jiang T (2020) Dynamic selective maintenance optimization for multi-state systems over a finite horizon: a deep reinforcement learning approach. Euro J Oper Res 283(1):166–181 12. Meng MH, Zuo M, et al (1999) Selective maintenance optimization for multi-state systems. In: 1999 IEEE Canadian conference on electrical and computer engineering, vol 3, pp 1477–1482 13. Nakamura S, Qian CH, Chen M (2013) Reliability modeling with applications: essays in Honor of Professor Toshio Nakagawa on his 70th birthday. World Scientific, Singapore 14. Pandey M, Zuo MJ, Moghaddass R (2013) Selective maintenance modeling for a multistate system with multistate components under imperfect maintenance. IIE Trans 45(11):1221–1234 15. Pandey M, Zuo MJ, Moghaddass R, Tiwari M (2013) Selective maintenance for binary systems under imperfect repair. Reliab Eng Syst Saf 113:42–51 16. Whitley D (1994) A genetic algorithm tutorial. Stat Comput 4(2):65–85

Chapter 4

Selective Maintenance for Multi-state Systems with Loading Strategy

4.1 Introduction In many practical scenarios, a load has a strong effect on the failure probability of systems and their components, especially for conveyers, computer processors, loadcarrying systems, and cutting tools [1, 6, 7]. Typically, health conditions of industrial applications degrade faster under heavy loading conditions. For instance, the speed of machinery can be set at different levels to achieve higher or lower production rates. When machines are overutilized at high speed, machine failure incidence is higher than that under normal speed. Another typical example is that production systems may be required to run under overload conditions when demand is high [9]. To accomplish the required production order, managers should adjust the load distribution imposed on a system’s components. The load distribution problem is an important research concern in system reliability engineering. Existing studies have been carried out based on the relationship between the failure rates of the components and load. Cox [2] first introduced the load-dependent failure rate in the analysis of survival data in biostatistics. Lanza et al. [4] supposed that the lifetime of a tool obeyed the Weibull cumulative damage model, and the load on each component has an influence on scale parameters. The system exhibited different failure rates under various load conditions. Nourelfath and Yalaoui [7] integrately optimized the load distribution and production planning in multi-state production systems. They suggested that the failure rate of a component depends on the load, following a power law. Dao and Zuo [3] employed the proportional hazards model (PHM) to characterize load-dependent degradation processes of multi-state components operating in a dynamic loading condition. A sequential preventive maintenance (PM) model for a system operating in a given loading profile was investigated. In this chapter, we investigated the joint optimization of the selective maintenance and load distribution problem for multi-state systems (MSSs). Failure rates of the components in the investigated system were determined not only by the inherent © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_4

65

66

4 Selective Maintenance for Multi-state Systems with Loading Strategy

characteristics but also by the external workload imposed on the components. To achieve the required system performance in the next mission, decision makers aim to maximize the probability of the system successfully completing the next mission by allotting the maintenance budget and adjusting the load distribution in a joint manner. The remainder of this chapter was organized as follows. Section 4.2 introduced the selective maintenance problem, some basic model assumptions, and the PHM. The imperfect maintenance model was introduced in Sect. 4.3. Further, the relationship between maintenance efficiency and cost was formulated. The probability of a system successfully completing the next mission was evaluated in Sect. 4.4. Section 4.5 provided the illustrative example and further discussions. Finally, closure was provided in Sect. 4.6.

4.2 Problem Statements and Model Assumptions In engineering practice, systems must complete a series of consecutive missions. Maintenance activities can be conducted with finite breaks between any two adjacent missions to restore the system to a better health condition. However, a potential risk is that systems may fail during a mission owing to component degradation. Hence, during breaks, maintenance activities should be scheduled within a limited time slot between two adjacent missions. Therefore, we must identify only a subset of components to be restored to enhance system performance and reliability. The following are the basic assumptions of the selective maintenance model: • A system comprises M s-independent components. Each component possesses only two states: functioning and failed (with zero performance rate) states. At the beginning of the kth mission, the state of component l is represented by a binary variable, denoted by X l,k . Thus, we have  1 if component l is functioning . (4.1) X l,k = 0 if component l is failed The state of component l at the end of the kth mission is denoted by Yl,k , which is a binary variable denoted by  1 if component l is functioning . (4.2) Yl,k = 0 if component l is failed As maintenance activities cannot be carried out during mission execution, the relation X l,k ≥ Yl,k always holds. • The performance rate of component l is represented as gl,0 (gl,0 = 0) and gl,1 (gl,1 = 0) for the failed and functioning states, respectively. The performance rate of component l at any time instant t (t ≥ 0) is a random variable, denoted by G l (t),

4.3 Imperfect Maintenance

67

and one has G l (t) ∈ {gl,0 , gl,1 }. In practice, the failure rate of components is related to the load imposed on the components. Several models have been proposed (e.g., PHM and accelerated failure-time model (AFTM)) to mathematically describe the relation between the failure rate and load [5]. • The performance rate of the entire MSS at time t is denoted by G(t), which is unambiguously determined by the performance rates of its components and the system configuration (e.g., series-parallel, bridge, and network). The system configuration remains unchanged throughout the mission. The performance rate of the system at time t is determined by G (t) = φ (G 1 (t), G 2 (t), . . . , G M (t)) ,

(4.3)

where φ(·) is the structure function. For example, in the case of a flow transmission system with two components connected in series, the structure function of the system takes the form φ(G 1 (t), G 2 (t)) = min{G 1 (t), G 2 (t)}. One has φ(G 1 (t), G 2 (t)) = G 1 (t) + G 2 (t) in the case of a system with two parallelconnected components. The set of performance capacities of a system is denoted by G = {g1 , g2 , . . . , g Ns }. Here, gi is the ith performance capacity in set G, and Ns is the number of possible performance capacities of a system. • During each break between two adjacent missions, the condition of a component can be restored through maintenance action, which is selected from a set of optional maintenance actions. These actions include doing nothing, corrective maintenance (CM) for failed components, and PM for functioning components. The break before the kth mission is denoted by the kth break, and the maintenance budget of the kth break is denoted by Ck . • The time duration of the kth mission is denoted by z k . The states and effective ages of all components at the end of the kth mission are exactly known in advance. • The system is required to meet a pre-specified mission demand, and the demand of the kth mission is denoted by Wk . Hence, a mission is successfully completed if the performance capacity of the system at the end of the mission is greater than the mission demand.

4.3 Imperfect Maintenance 4.3.1 Failure Rate with Load Distribution To assess the MSS reliability with a load strategy, the relationship between the load and failure rate should be defined first. In this section, two typical models, namely, PHM and AFTM, are used to characterize the relationship between the failure rate and load of a component.

68

4 Selective Maintenance for Multi-state Systems with Loading Strategy

PHM: In PHM, the failure rate of a component is written as λl (t, gl ) = λ0,l (t) exp{θl gl }.

(4.4)

Here, λ0,l (t) is the baseline failure intensity of component l, θl is a constant coefficient, and gl is the load on component l. The exponential function exp{θl gl } can be replaced by any other function fl (gl ). For example, if fl (gl ) is a linear function, then fl (gl ) = θl gl and λl (t, gl ) = θl gl λ0,l (t) .

(4.5)

For a component with a Weibull distribution lifetime, the failure intensity of the component under PHM is written as     γl t γl −1 (4.6) θl gl , λl (t, gl ) = ηl ηl where γl (γl > 0) and ηl (ηl > 0) are the shape and scale parameters of the Weibull distribution of component l, respectively. AFTM: In AFTM, the failure intensity of component l is formulated as λl (t, gl ) = ψ (gl ) λ0,l (tψ (gl )) ,

(4.7)

where λ0,l (·) is the baseline failure intensity function and ψ(gl ) is the link function. The commonly used forms of ψ(gl ) include • Power Law • Exponential Law

ψ (gl ) = gl θl .

(4.8)

ψ (gl ) = exp (θl gl ) .

(4.9)

If ψ(gl ) follows a power law, then the failure intensity of component l with Weibull distribution lifetime can be written as   γ −1  γl gl θl t l θl λl (t, gl ) = gl . (4.10) ηl ηl

4.3.2 Imperfect Maintenance Modelling Several imperfect maintenance models have been studied in the literature to characterize the maintenance efficiency for the cases that a repaired system or component cannot be restored to a completely new condition [8]. The Kijima type-II model is

4.4 Probability of a System Successfully Completing a Mission

69

one of the most popular imperfect maintenance models that uses an effective (virtual) age to represent the actual condition of a repaired component or system. Based on the definition of the Kijima type-II model, the effective age of component l after maintenance subsequent to the kth mission can be written as follows Al,k+1 = bl,k Bl,k ,

(4.11)

where Al,k+1 and Bl,k represent the effective age of component l at the beginning of the (k+1)th mission and the end of the kth mission, respectively. Further, bl,k (0 ≤ bl,k ≤ 1) is the age reduction factor corresponding to the efficiency of a maintenance action for component l in the kth break. A lower value of bl,k indicates more efficient maintenance action. In particular, bl,k = 0 indicates a perfect maintenance action, whereas bl,k = 1 represents the minimal repair action for the failed component l. Maintenance efficiency depends on the maintenance cost allocated to component l. If more maintenance costs are allocated to a component, a more efficient maintenance action can be performed, and a better health condition of the component will be generated. This relationship between the age reduction factor and maintenance cost allocated with a component can be characterized as follows: • For a failed component

 bl,k = 1 −

cl cre,l

αl

,

(4.12)

,

(4.13)

• For a functioning component  bl,k = 1 −

cl cre,l

βl

where cre,l is the replacement cost of component l and cl is the maintenance cost allotted to component l in the kth break. Further, αl and βl are the coefficients determined by inherent component characteristic and reflects the exact relationship between the age reduction factor and allotted maintenance cost. In general, a greater cl can restore the component to a better condition.

4.4 Probability of a System Successfully Completing a Mission The time duration of the kth mission, denoted by z k , was known in advance. If component l functions at the beginning of the kth mission and works under load gl during the kth mission, the survival probability of the component at the end of the kth mission can be deduced as

70

4 Selective Maintenance for Multi-state Systems with Loading Strategy

pl,k (1) = Pr { Yl (k) = 1| X l (k) = 1} Pr {T > Ak + z k } = Pr {T > Ak }      Ak +zk γl u γl −1 θl gl du exp − 0 ηl ηl 

   =  Ak γl u γl −1 θl gl du exp − 0 ηl ηl   θl gl γl γl = exp (Ak − (Ak + z k ) ) . ηl

(4.14)

The UGF of component l at the end of the kth mission can be represented as u l,k = pl,k (0)z gl,0 + pl,k (1)z gl,1 ,

(4.15)

where pl,k (0) = 1 − pl,k (1). The UGF of the entire MSS at the end of the kth mission is derived as follows Uk =

M

 u l,k l=1

⎛ ⎞ M 1  ⎝ pl,k (il ) · z gl,il ⎠ = l=1

=

il =0

1  1  i 1 =0 i 2 =0

=

Ns 

···

M 1  i M =0



(4.16)

pl,k (il ) · z φ (g1,i1 ,g2,i2 ,...,g M,i M )

l=1

sys

pk (i) · z gi .

i=1 sys

Here, pk (i) is the probability that the performance rate of the system equates to gi at the end of the kth mission, and G S (k) is the set of system performance rates at the end of the kth mission. If the demand during the kth mission is Wk , then the probability of the system successfully completing the mission can be deduced as follows R(k) =

Ns 

sys

pk (i) · 1(gi ≥ Wk ),

(4.17)

i=1

where 1(·) is an indicator function that is equal to 1 if the argument in the bracket is true and otherwise 0.

4.5 Selective Maintenance Modelling

71

4.5 Selective Maintenance Modelling In this section, multiple maintenance options can be adopted during the break between two consecutive missions. Decision makers should determine the maintenance action/cost and load on each component. The maintenance cost comprises two parts: CM and PM costs. The total maintenance cost spent in the break can be computed easily by (4.18) C (k) = CCM (k) + CPM (k) , where CCM (k) and CPM (k) represent the CM and PM costs, respectively. CCM (k) can be computed as M 

 cl · 1 − Yl,k−1 , (4.19) CCM (k) = l=1

and CPM (k) can be computed by CPM (k) =

M 

cl · Yl,k−1 .

(4.20)

l=1

Under the limited maintenance budget Ck , the joint selective maintenance and load distribution problem can be formulated as an optimization model with two missions: (1) determining the maintenance cost allotted to each component (i.e., cl ) and (2) determining the load distributed to each component (i.e., gl ). The optimization model can be formulated as max : R(k) (4.21) subject to C (k) ≤ Ck

(4.22)

cl ≥ 0

(4.23)

gmin,l ≤ gl ≤ gmax,l

(4.24)

 X l,k =

0 if Yl,k−1 = 0 1 otherwise



gl = 0 if X l,k = 0,

cl = 0

(4.25) (4.26)

where constraint (4.22) implies that the total maintenance cost should be less than the maintenance budget in the kth break. Constraints (4.23) and (4.24) represent the ranges of decision variables. gmin,l and gmax,l are the lower and upper bounds of the load on component l, respectively. Constraint (4.25) provides the state of component

72

4 Selective Maintenance for Multi-state Systems with Loading Strategy

l at the beginning of the kth mission. Constraint (4.26) implies that the load on a component is zero if the component fails at the end of a mission, and no maintenance cost is incurred on the component.

4.6 Illustrative Example The studied system comprises five binary-capacitated components. The structure of the system is shown in Fig. 4.1. Suppose that the baseline failure intensity is derived from the Weibull distribution. The corresponding parameters, for example, γl and ηl , are tabulated in Table 4.1. The PHM was adopted to characterize the relationship between the load and failure rate of the components. The failure intensity is given by Eq. (4.6). The parameters for each component include the bound of load gmin,l , gmax,l , coefficients of age reduction αl , βl , coefficient of failure intensity θl , replacement cost cre,l , effective age at the end of the (k − 1)th mission Bl,k , and state of a component at the end of the (k − 1)th mission Yl,k−1 . The maintenance budget Ck is $500, and the demand for the kth mission Wk is 85 tons/day. The selective maintenance optimization model is solved by the genetic algorithm (GA) with a population size of 200, a maximum iteration of 2000, and a crossover rate and mutation rate of 0.6 and 0.05, respectively. The pseudo-code of the GA is provided in Algorithm 4.1. The optimization results are presented in Table 4.2. In Scenario 1, the load distribution optimization is considered. We can observe that the maximum probability of

3 1 4 2 5

Fig. 4.1 Structure of the studied system Table 4.1 Parameter settings of all the components (time unit: hour, cost unit: US dollars, load unit: tons/day) l γl ηl gmin,l gmax,l αl βl θl cre,l Bl,k Yl,k−1 1 2 3 4 5

3.8 3.6 3.9 3.9 3.8

100 120 110 120 125

0 0 0 0 0

60 70 30 80 100

0.18 0.22 0.25 0.21 0.22

0.23 0.25 0.27 0.25 0.25

0.017 0.014 0.033 0.0125 0.01

200 200 300 150 350

80 50 60 60 90

0 1 1 0 1

4.6 Illustrative Example

73

Algorithm 4.1 The pseudo-code of GA Inputs: Imax : The maximum number of generations; I : The index of generation. 1: Initialize the genes with the maintenance cost cl and the performance rate gl of each unit; 2: Evaluate R(k) by Eq. (4.17); 3: For I ← 1 to Imax do 4: Parent selection: roulette wheel selection; 5: Crossover operation: whole arithmetic recombination with the probability of 0.6; 6: Mutation operation: bit flip mutation with the probability of 0.05; 7: Evaluate R(k) by Eq. (4.17); 8: End For Output: The optimal individual Table 4.2 Optimal results with/without load distribution (cost unit: US dollars, load unit: tons/day) ID Scenario 1 (with considering Scenario 2 (without considering load distribution optimization) load distribution optimization) cl gl Action cl gl Action 1 2 3 4 5 C(k) R(k)

76.13 181.36 91.13 85.78 65.60 500.00 0.9011

5.03 79.97 43.03 43.03 41.97

CM PM PM CM PM

199.96 123.24 58.22 56.35 62.23 500.00 0.8146

70 80 50 60 45

CM PM PM CM PM

the system successfully completing the next mission is 0.9011, the total maintenance cost is $500, and the maintenance and load allocated to each component are tabulated in columns cl and gl , respectively. In Scenario 2, the load on each component is a pre-specified value, that is, the maximum performance rate of component gmax,l . The probability of the system successfully completing the kth mission is 0.8146, with a total maintenance cost of $500. Compared with Scenario 2, the probability of the system successfully completing the next mission in Scenario 1 increased remarkably by 0.0865. To further demonstrate the advantage of the proposed joint optimization model, a comparative study between two strategies (with and without considering load distribution optimization) under different settings of the maintenance budget is conducted. The results are shown in Fig. 4.2. As observed from Fig. 4.2, the probabilities of the system successfully completing the next mission with the strategy considering considering load distribution optimization are always greater than that without considering load distribution optimization.

74

4 Selective Maintenance for Multi-state Systems with Loading Strategy

Probability of compelting mission

0.95

0.9

0.85

0.8 Strategy with load distribution 0.75

0.7 250

Strategy without load distribution

300

350

400 450 500 550 600 Maintenance budget(US dollars)

650

700

750

Fig. 4.2 Maximum probability of successfully completing the next mission versus maintenance budget

4.7 Closure This chapter investigated the effects of the loading strategy on selective maintenance optimization. Specifically, we investigated how to distribute the load and maintenance cost for each component to maximize the probability of the system successfully completing the next mission within a limited maintenance budget. The PHM was adopted to characterize the relationship between the load and failure rate of the components. The Kijima type-II model was used to characterize the imperfect maintenance efficiency. As observed from the case study, the selective maintenance strategy that considers load distribution optimization always yielded a better result.

References 1. Cassady CR, Pohl EA, Murdock WP (2001) Selective maintenance modeling for industrial systems. J Qual Maintenance Eng 7(2):104–117 2. Cox DR (1972) Regression models and life-tables. J Roy Stat Soc Ser B (Methodol) 34(2):187– 202 3. Dao CD, Zuo MJ (2017) Optimal selective maintenance for multi-state systems in variable loading conditions. Reliab Eng Syst Saf 166:171–180 4. Lanza G, Niggeschmidt S, Werner P (2009) Optimization of preventive maintenance and spare part provision for machine tools based on variable operational conditions. CIRP Ann 58(1):429– 432 5. Levitin G, Amari SV (2009) Optimal load distribution in series-parallel systems. Reliab Eng Syst Saf 94(2):254–260 6. Liu Y, Huang HZ (2010) Optimal selective maintenance strategy for multi-state systems under imperfect maintenance. IEEE Trans Reliab 59(2):356–367

References

75

7. Nourelfath M, Yalaoui F (2012) Integrated load distribution and production planning in seriesparallel multi-state systems with failure rate depending on load. Reliab Eng Syst Saf 106:138– 145 8. Pham H, Wang H (1996) Imperfect maintenance. Eur J Oper Res 94(3):425–438 9. Xiao H, Shi D, Ding Y, Peng R (2016) Optimal loading and protection of multi-state systems considering performance sharing mechanism. Reliab Eng Syst Saf 149:88–95

Chapter 5

Selective Maintenance under Stochastic Time Durations of Breaks and Maintenance Actions

5.1 Introduction The traditional selective maintenance models are based on the premise that the time durations of maintenance actions and breaks are constant [1, 2, 9, 10], that is, these time durations can be exactly known before the maintenance decision-making at a break between two adjacent missions. Unfortunately, the maintenance time duration and the beginning time of the next mission cannot always be determined in advance [6, 7]. On one hand, the maintenance duration cannot be quantified in a deterministic manner due to the uncertainties arised from the varying skill levels of repairmen, the condition of maintenance tools, the maintenance environment, and so forth. On the other hand, the arrival of the next mission cannot be accurately forecasted in many applications. For example, it is unrealistic to accurately predict the next attack in many military environments, and the breaks between two planned flights of a civil aircraft may be adjusted due to the sudden change of weather. Only a few works on selective maintenance investigated the time duration uncertainties of maintenance actions and breaks. From the perspective of binary-state systems, Khatab et al. [6, 7] developed selective maintenance problems by modelling the time durations of breaks, maintenance actions, and missions as random variables with Gamma distribution which has the additive property. Under the pre-determined probabilistic constraints of the system successfully completing the next mission, and the probabilistic constraints of all the planned maintenance actions being completed in a break, the maintenance actions were identified by minimizing the maintenance cost. This chapter introduced a new framework of selective maintenance optimization for multi-state systems with stochastic time durations of maintenance actions and breaks. The maintenance actions are executed sequentially in a break between two adjacent missions to ensure the success of the next mission. Due to the uncertainty of the time durations of maintenance actions and breaks, not all the planned maintenance actions in a break can be completed before the beginning of the next mission, and the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_5

77

78

5 Selective Maintenance under Stochastic Time Durations …

maintenance actions at the front of a planned maintenance sequence are more likely to be completed. The decision-makers should, therefore, not only select the maintenance actions from the set of all possible maintenance actions, but also determine the sequence of executing selected maintenance actions. To facilitate the computation of the probability distribution of the number of completed maintenance actions, the saddlepoint approximation was utilized to reduce the computational complexity of involved multi-dimensional convolution. Given the completed maintenance actions in a break, the probability of a system successfully completing the next mission was evaluated by the UGF method [8, 11]. A tailored ant colony optimization algorithm was developed for the cases that all feasible maintenance sequences cannot be enumerated. A four-component system and a multi-state coal transportation system were exemplified to demonstrate the effectiveness of the proposed method. The remainder of this chapter was rolled out as follows. The problem statement and basic assumptions of the selective maintenance under stochastic time durations of breaks and maintenance actions were represented in Sect. 5.2. Given a specified maintenance sequence, the probability of a system successfully completing the next mission was evaluated in Sect. 5.3. In Sect 5.4, the resulting selective maintenance problem was formulated, and a tailored ant colony optimization algorithm was introduced. Two illustrative examples are presented in Sect. 5.5. A brief closure was given in Sect. 5.6.

5.2 Problem Statements and Model Assumptions A system is required to execute a sequence of missions with a break between two adjacent missions, and the components of a system can be repaired in breaks to ensure the success of the subsequent missions. The break after the kth mission is denoted by the kth break. Due to limited maintenance resources, such as budget, time, and manpower, only a subset of available maintenance actions can be selected and executed in each break. Moreover, due to the uncertainty of the time durations of maintenance actions and breaks, a selected maintenance action may not be successfully performed before the end of a break. For example, as illustrated in Fig. 5.1, three maintenance actions, namely Actions #1, #2, and #3, are selected and sequentially executed in the kth break. In Sequence #1, the actions are executed in the order of Action #1, Action #2, and Action #3, whereas in Sequence #2, the three actions are executed in the order of Action #3, Action #2, and Action #1. If the time durations of maintenance actions and breaks are deterministic values, as shown in Scenario #1, the sequence of maintenance actions does not impact the completion of the three selected maintenance actions, and the number of completed maintenance actions in a break is a deterministic value too. Scenarios #2.1 and #2.2 illustrate two possible cases with stochastic time durations of maintenance actions and breaks. In Scenario #2.1, the first two actions in each sequence are successfully completed, whereas Action #3 in Sequence #1 and Action #1 in Sequence #2 cannot be completed. Nonetheless, Action #2 in Sequence #1 is not completed as the break

5.2 Problem Statements and Model Assumptions Scenario #1 (Deterministic Cases) Action #1 Sequence #1 Action #3

79 Break Ends Action #3

Action #2 Action #2

Action #1

Sequence #2 Scenario #2 (Stochastic Cases)

Scenario #2.1 Sequence #1

Sequence #2

Action #1 Action #3

Action #2 Action #2

Action #1

Break Starts Scenario #2.2 Sequence #1 Sequence #2

Action #3

Break Ends Action #1

Action #3

Action #2 Action #2

Break Starts

Action #3 Action #1

Break Ends

Sequence 1: Action #1→ Action #2→ Action #3

Maintenance action cannot be completed

Sequence 2: Action #3→ Action #2→ Action #1

Maintenance action be completed

Fig. 5.1 The chronological time versus the effective age in the Kijima type II model

in Scenario #2.2 ends earlier than that in Scenario #2.1, where the numbers of completed maintenance actions in Sequences #1 and #2 are one and two, respectively. As illustrated in Scenarios #2.1 and #2.2, the uncertainty of the time durations of maintenance actions and breaks do impact the completion of selected maintenance actions, and the maintenance action performed earlier is more likely to be completed. To maximize the probability of a system successfully completing the (k + 1)th mission, the maintenance actions in the kth breaks should be selected and ordered under the stochastic time durations of breaks and maintenance actions. The basic assumptions in the selective maintenance model are as follows: • A system consists of M s-independent multi-state components, and the states of a component are distinguished by its performance capacity, where component l can process n l possible states during its lifetime. The performance capacity of component l at state i is denoted by gl,i . Let gl,1 and gl,nl denote the worst and the best performance capacities of component l, respectively. The degradation behavior of each unit can be characterized by a homogenous continuous-time Markov chain, and the intensity of component l transiting from state i to j is denoted dy λli, j . The performance capacity and the state of component l at time instant t is denoted by G l (t) and sl (t), respectively, and one has G l (t) = gl,sl (t) .

80

5 Selective Maintenance under Stochastic Time Durations …

• The performance capacity of a system, denoted by G(t), is completely determined by the performance capacity of each component and the system configuration. The performance capacity of the system at time instant t is formulated by G(t) = φ(G 1 (t), G 2 (t), . . . , G M (t)),

(5.1)

where φ(·) is the structure function, representing the relation between the performance capacities of all the components and that of the system. For example, in the case of a flow transmission system, the structure function of the system with two parallel components takes the form of φ(G 1 (t), G 2 (t)) = G 1 (t) + G 2 (t), and in the case of a system with two series-connected components, the structure function of the system can be written as φ(G 1 (t), G 2 (t)) = min{G 1 (t), G 2 (t)}. The set of performance capacities of a system is denoted by G = {g1 , g2 , . . . , g Ns }, where gi is the ith performance capacity in set G, and Ns is the number of possible performance capacities of a system. • The time duration of the kth break is denoted by L k , which is a random variable with the probability density function f k (t). During the kth break, the performance capacity of a component can be recovered by maintenance actions. The maintenance action of restoring component l from state i to state j is denoted by ai,l j (i < j), and the corresponding maintenance cost is denoted by Ci,l j . The time duration of maintenance action ai,l j is denoted by a random variable Ti,l j with the probability density function of f i,l j (t). The maintenance action, which is not completed at the end of a break, cannot change the state of the corresponding component, and thus the component sojourns in the state before maintenance. The maintenance budget of the kth break is denoted by Ck . • The time duration of the kth mission is denoted by z k , which is a constant. The time instant at the beginning and the end of the kth mission is denoted by tk− and tk+ , respectively. The state of each component at the end of the kth mission (i.e., sl (tk+ )) is exactly known. In the kth mission, a system is required to meet a pre-specified mission demand, denoted by Wk , and hence, a mission is successfully completed if the performance capacity of the system at the end of the mission is greater than the mission demand. • Let decision variable X k,v,l,i, j represents that the vth maintenance action in the kth break is ai,l j (i < j), and one has X k,v,l,i, j = 0 if i ≥ j. Suppose that at most nlmaintenance action can be selected for each component, and one has  M one + v=1 j=1 X k,v,l,s(tk ), j ≤ 1 for any component l. Moreover, as a selected maintenance action can only be placed on a particular locationof a maintenance sequence, M n l + X the decision variables in the kth break are subject to l=1 j=1 k,v,l,s(tk ), j ≤ 1 l for any location v. If nj=1 X k,v,l,s(tk+ ), j = 0, component l will not be repaired in the kth break.

5.3 Probability of a System Successfully Completing a Mission

81

5.3 Probability of a System Successfully Completing a Mission 5.3.1 Probability Distribution of the Number of Completed Maintenance Actions The number of maintenance actions that can be completed in the kth break is denoted by Nk . Due to the stochastic nature of the time durations of breaks and maintenance actions, Nk is a discrete random variable rather than a constant, and its probability mass function is evaluated in this section. The time duration of the vth maintenance action in a particular maintenance sequence (i.e., X k,v,l,i, j ) is denoted by a random variable Tv , and one has Tv =

nl M  

X k,v,l,sl (tk+ ), j Tsll (t + ), j ,

(5.2)

k

l=1 j=1

where sl (tk+ ) is the state of component l at the end of the kth mission. Let h v (t) represent the probability density function of Tv , and it can be written as h v (t) =

nl M  

X k,v,l,sl (tk+ ), j f sll (t + ), j (t).

(5.3)

k

l=1 j=1

Given a specific maintenance sequence, the cumulative duration of completing the first m maintenance actions, denoted by TmC , can be formulated as TmC =

m  v=1

Tv =

nl m  M   v=1 l=1 j=1

X k,v,l,sl (tk+ ), j Tsll (t + ), j .

(5.4)

k

As shown in Eq. (5.4), TmC is a sum of the time durations of the first m maintenance actions, and hence, the probability density function of TmC , denoted by f TmC (t), can be evaluated by a multi-dimensional convolution as follows: t f TmC (t) =

C C C C (t h m (tmC − tm−1 ) f Tm−1 m−1 )dtm−1 0 C

t

tm−1 C C C C h m (tm − tm−1 ) h m−1 (tm−1 − tm−2 )...

= 0

0

C tm−1



C C h 2 (t2C − t1C )h 1 (t1C )dt1C . . . dtm−2 dtm−1 . 0

(5.5)

82

5 Selective Maintenance under Stochastic Time Durations …

Furthermore, the cumulative distribution function of random variable TmC , denoted by FTmC (t), can be formulated as t F (t) =

f TmC (tmC )dtmC .

TmC

(5.6)

0

Note that if Tv (v = 1, 2, . . . , m) are governed by the distributions with the additive property, such as Gamma distribution, the distribution type of TmC can be directly determined, and the corresponding parameters can be evaluated by the parameters of Tv . For example, if Tv follows a Gamma distribution with the shape parameter of kv and the scale parameter of θ , where the scale parameters of all Tv are the same, with the scale parameter of θ , and its shape TmC also follows a Gamma distribution  k parameter can be evaluated by m v=1 v . Let Nk (t) denote the number of completed maintenance actions at time t since the beginning of the kth break (i.e., Nk (t)) is the number of completed maintenance actions at time instant tk+ + t. The probability of at least m maintenance actions being completed at time instant tk+ + t can be represented as Pr{Nk (t) ≥ m} = FTmC (t).

(5.7)

where Pr{Nk (t) ≥ 0} = 1 and Pr{Nk (t) ≥ M + 1} = 0. Based on Eqs. (5.5)–(5.7), the probability distribution of the number of completed maintenance actions at time instant tk+ + t can be evaluated by Pr{Nk (t) = m} = Pr{N (t) ≥ m} − Pr{N (t) ≥ m + 1} C (t), = FTmC (t) − FTm+1

(5.8)

where Pr{Nk (t) = m} is the probability of the first m maintenance actions being completed by the time instant tk+ + t. Hence, the probability of the first m maintenance actions being completed at the end of the kth break, denoted by Pk,m , can be formulated as Pk,m = Pr{Nk = m} +∞ Pr{Nk (t) = m} f k (t)dt = 0 +∞



=

(5.9)  C (t) FTmC (t) − FTm+1 f k (t)dt,

0

where f k (t) is the probability density function of the time duration of the kth break, and Pk,m is the probability mass function of Nk .

5.3 Probability of a System Successfully Completing a Mission

83

5.3.2 Saddlepoint Approximation As presented in Sect. 5.3.1, the probability density function and cumulative distribution function of random variable TmC can be evaluated in a computationally efficient manner if the distributions of all Tv have additive property. However, as the time duration of maintenance actions may follow mixed distributions or follow a distribution without additive property, the nice additive property may not be always held in most engineering practices. Hence, computing the cumulative distribution function of TmC is extremely troublesome due to the multi-dimensional convolution, and the analytical solution may not exist in most cases. Alternatively, the saddlepoint approximation [3, 5] can be utilized to approximate the probability mass function of Nk , in lieu of the computationally inefficient multi-dimensional convolution. Suppose a random variable Y with the probability density function of f Y (y), the moment generating function of Y is defined as [5] M(ξ ) =

+∞ eξ y f Y (y)dy,

(5.10)

−∞

and the cumulant generating function of Y is defined as [5] K (ξ ) = ln (M(ξ )) .

(5.11)

The first derivative and the second derivative of K (ξ ) are denoted by K  (ξ ) and K  (ξ ), respectively. The cumulant generating functions of some typical distributions are tabulated in Table 5.1. Let (·) and ϕ(·) respectively represent the cumulant generating function and the probability density function of the standard normal random variable. Let sign(x) denote a sign function defined as

Table 5.1 The cumulant generating functions of some typical distributions Distribution type f Y (y) K (ξ )   (y−μ)2 1 √ Normal exp 2σ 2 μξ + 21 σ 2 ξ 2 2πσ   ϕ y−μ σ

Truncated normal

σ (β)−σ (α) b−μ a−μ σ ;α = σ

β=

Uniform

1 b−a

Gamma

1 (k)θ k

Exponential

β exp(−βy)

y k−1 exp

 −y θ

μξ + 21 σ 2 ξ 2 − ln [(β) − (α)] + ln [(β − σ ξ ) − (α − σ ξ )] β = b−μ ; α = a−μ σ σ

ln exp(bξ ) − exp(aξ ) − ln(b − a) − ln(ξ ) −k ln(1 − θξ )   − ln 1 − βξ

84

5 Selective Maintenance under Stochastic Time Durations …

⎧ ⎨1 x > 0 sign(x) = 0 x = 0 . ⎩ −1 x < 0

(5.12)

The saddlepoint approximation of the cumulative distribution function of arbitrary random variable Y can be formulated as [5]   1 1 − , (5.13) FY (y) = Pr{Y ≤ y} = (w) + ϕ(w) w u where w and u are defined as  1 w = sign(ξ0 ){2 [ξ0 y − K (ξ0 )]} 2 ,

21  u = ξ0 K (ξ0 )

(5.14)

where ξ0 is the saddlepoint solved by K  (ξ0 ) = y.

(5.15)

Therefore, the cumulative distribution function of a random variable can be evaluated by the saddlepoint approximation if its cumulant generating function can be computed. To implement the saddlepoint approximation in the evaluation of the probability mass function of Nk (i.e., Pk,m in Eq. (5.9)) is further formulated as Pk,m = Pr{Nk = m} C = Pr{TmC ≤ L k } − Pr{Tm+1 ≤ Lk}  m  m+1    = Pr Tv − L k ≤ 0 − Pr Tv − L k ≤ 0 . v=1

(5.16)

v=1

 m Let Y = m v=1 Tv − L k , and Pr{ v=1 Tv − L k ≤ 0} can be written as FY (0) = Pr{Y ≤ 0}. The original multi-dimensional integration in Eqs. (5.5)–(5.9) can be, therefore, transformed into an equivalent problem of evaluating a particular value of the cumulative distribution function of a random variable (i.e., Y ). As Y is a linear function of Tv and L k , based on the definition of cumulant generating functions, the cumulant generating function of Y is also the linear function of the cumulant generating functions of Tv and L k [3, 5]. The cumulant generating function of Y can be, therefore, evaluated as K Y (ξ ) =

m  v=1

K Tv (ξ ) + K L k (−ξ ),

(5.17)

5.3 Probability of a System Successfully Completing a Mission

85

where K Tv (ξ )(i ∈ {1, 2, . . . , m}) and K L k (ξ ) are the cumulant generating functions of Tv and L k , respectively, and the cumulant generating functions of most typical distributions are readily derived as shown in Table 5.1. Based on Eqs. (5.12)–(5.15), m Ti − L k ≤ 0} = FY (0) can be evaluated, and the probability mass function Pr{ i=1 of Nk (i.e., Pk,m ) can be evaluated by the saddlepoint approximation without the multi-dimensional convolution.

5.3.3 Probability of a System Successfully Completing the Next Mission If the first m maintenance actions are completed in the kth break (i.e., Nk = m) the probability of component l sojourning in state i at the end of the kth break, that is, at the beginning of the (k + 1)th mission can be written as − ) = i|Nk = m} = Pr{sl (tk+1

m  v=1

X k,v, l, sl (tk+ ), i .

(5.18)

l Under the condition that Nk = m, let pi,k,m (t) represent the probability of coml (t) = Pr{sl (t) = i|Nk = m} ponent l sojourning in state i at time t, that is, pi,k,m − − − l (t ≥ tk+1 ), and one has pi,k,m (tk+1 ) = Pr{sl (tk+1 ) = i|Nk = m}. Under the condition that that Nk = m, the probability of component l at the end of the (k + 1)th + − l l (tk+1 ) = pi,k,m (tk+1 + z k+1 ), where z k+1 is the time mission can be denoted by pi,k,m duration of the (k + 1)th mission. As the stochastic degradation behavior of each + l (tk+1 ) can component follows a homogenous continuous-time Markov chain, pi,k,m be evaluated by the corresponding Kolmogorov differential equations as

− l dpi,k,m (tk+1 + τ)

dτ

=

nl  j=i+1

− − l λlj, i plj,k,m (tk+1 + τ ) − pi,k,m (tk+1 + τ)

i−1 

λli, j , (5.19)

j=1

− l where pi,k,m (tk+1 + τ ) is the probability of component l sojourning in state i at time − l (tk+1 + τ since the beginning of the (k + 1)th mission given that Nk = m, and pi,k,m − 0) = Pr{sl (tk+1 ) = i|Nk = m} are presented in Eq. (5.18). Under the condition that Nk = m, the probability of a system performance capacity being equal to gi (gi ∈ G) at the end of the (k + 1)th mission is denoted by + ) = gi |Nk = m}. Based on the state distribution of each component and Pr{G(tk+1 the structure function φ(·), the distribution of the performance capacity of a system can be derived via the UGF method [9]. Given that Nk = m, the UGF of component l at the end of the (k + 1)th mission can be written as

86

5 Selective Maintenance under Stochastic Time Durations …

u l,k+1 =

nl 

+ l pi,k,m (tk+1 )z gl,i .

(5.20)

i=1

Hence, the UGF of the system at the end of the (k + 1)th mission can be formulated as Uk+1 =

n1 

···

i 1 =1

=

Ns 

M nM   i M =1

 + pill ,k,m (tk+1 )

·z

φ(g1,i1 ,...,g M,i M )

l=1

+ Pr{G(tk+1 )

(5.21)

= gi |Nk = m} · z , gi

i=1

where Ns is the number of all possible performance capacities of the system. il is a state of component l. The (k + 1)th mission can be successfully completed if the performance capacity of the system at the end of (k + 1)th mission is greater than the mission demand, and the probability of a system successfully completing the (k + 1)th mission, given that Nk = m, can be expressed as

R(k + 1|Nk = m) =

Ns 

+ Pr{G(tk+1 ) = gi |Nk = m} · 1(gi ≥ Wk+1 ),

(5.22)

i=1

where 1(·) is an indicator function, and one has 1(gi ≥ Wk+1 ) = 1 if gi ≥ Wk+1 , otherwise 1(gi ≥ Wk+1 ) = 0. Overall, the total probability of the system successfully completing the (k + 1)th mission can be written as R(k + 1) =

M 

Pr{Nk = m} · R(k + 1|Nk = m).

(5.23)

m=0

5.4 Selective Maintenance Optimization 5.4.1 Selective Maintenance Optimization Modelling Given a particular maintenance sequence in the kth mission (i.e., X k,v,l,i, j ) the total maintenance cost in the kth break, denoted by C(k), can be written as C(k) =

nl M  M   v=1 l=1 j=1

X k,v, l, sl (tk+ ), j · Csll (t + ), j . k

(5.24)

5.4 Selective Maintenance Optimization

87

The probability of a system successfully completing the (k + 1)th mission (i.e., R(k + 1)) can be evaluated via Sect. 5.3. The resulting selective maintenance problem can be formulated as a constrained combinational optimization problem as following: R(k + 1)

(5.25)

s.t. C(k) ≤ Ck

(5.26)

max

nl M   v=1 j=1 nl M  

X k,v,l,s(tk+ ), j ≤ 1

(5.27)

X k,v,l,sl (tk+ ), j ≤ 1

(5.28)

l=1 j=1

X k,v,l,sl (tk+ ), j = 0 or 1

(5.29)

where Ck is the limited maintenance budget in the kth break. As an NP-hard combinational optimization problem, the resulting selective maintenance problem can be resolved by enumerating all possible solutions. If the maintenance budget is unlimited, the number of possible solutions of the resulting selective maintenance problem is   M   M! + n l − sl (tk ) , (5.30) (M − m)! l∈V m=0 where M!/(M − m)! represents the m permutations of M, and V = {l|n l > sl (tk+ )} is the set of components which can be repaired in the kth break. As shown in (5.30), due to the “curse of dimensionality,” enumerating all possible solutions is irrational for a large-scale system. To mitigate the “curse of dimensionality,” meta-heuristic algorithms, such as genetic algorithms, simulated annealing algorithms, and ant colony optimization (ACO) algorithms, can be implemented to solve the problem. Inspired by the process of ants successively visiting multiple nodes in the ACO algorithm [4], the process of components being repaired sequentially is analogous to the travel of ants, and the ACO algorithm can be tailored to search the optimal maintenance sequence of the resulting selective maintenance problem in a computationally efficient manner.

5.4.2 Tailored Ant Colony Optimization Algorithm The ACO algorithm is motivated by the foraging behavior of ant species. Ants leave pheromones to guide subsequent ants to find foods quickly. By multiple iterations,

88

5 Selective Maintenance under Stochastic Time Durations …

ants can find the optimal foraging path by the pheromone on each path. A tailored ACO algorithm for the resulting selective maintenance optimization problem is introduced as follows: Solution construction: An individual solution is mapped by the trajectory of an ant among the nodes with specific definitions. Let a node represent an optional maintenance action in the kth break, and an ant passing through a node indicates that the corresponding maintenance action is performed. Therefore, a trajectory with the nodes that an ant sequentially visited represents a maintenance sequence as illustrated in Fig. 5.2. As shown in Fig. 5.2, each ant starts from a start point. Ant #1 sequentially visits 1 2 3 , a1,3 , and a2,4 , and it corresponds to sequentially repairing the nodes of actions a1,2 component 1 from state 1 to state 2, repairing component 2 from state 1 to state 3, and repairing component 3 from state 2 to state 4. The maintenance sequence of Ant 3 1 2 , a1,4 , and a1,4 . If a mainte#2 is to sequentially execute maintenance actions a2,3 nance action is selected for a component, all optional maintenance actions for the component will be added to the tabu list of the ant, which is the set of nodes that 1 1 , the nodes of a1,2 , the ant will not visit. For example, Ant #1 visits the node of a1,2 1 1 a1,3 , and a1,4 will be included into the tabu list of Ant #1, so component 1 will not be repaired twice. Movement probability: An ant in a node will randomly visit the next node to search for the optimal path, that is, the optimal solution of the selective maintenance problem. If an ant stays in node δ1 , the probability of the ant visiting node δ2 is denoted by Pδ1 ,δ2 , and it is formulated as

Fig. 5.2 An illustration of the possible paths of ants

Component 1 1 1,2

a Pheromone Trails

Component 2

Ant #1

1 a1,3

2 a1,2 2 a1,3

1 a1,4

2 a1,4

Start 3 a2,3 3 a2,4

Ant #2

Component 3

5.4 Selective Maintenance Optimization

Pδ1 , δ2 =

⎧ ⎨ ⎩0

89

(τδ1 , δ2 )α (ηδ1 , δ2 )β (τδ1 , δ2 )α (ηδ1 , δ2 )β

δ2 ∈Tabu /

δ2 ∈ / Tabu otherwise

,

(5.31)

where Tabu is the tabu list of the ant; τδ1 , δ2 is the quantity of pheromone on the edge of nodes δ1 and δ2 ; ηδ1 , δ2 denotes the heuristic information which offers ants the prior knowledge of travel; α and β(α, β ≥ 0) are two pre-specified weights of the ACO algorithm, balancing the importance of pheromone and heuristic information. In this study, the heuristic information is identified as  ηδ1 , δ2 =

Rδ1 , δ2 (k + 1) − Rδ1 (k + 1) 0

if Rδ1 , δ2 (k + 1) > Rδ1 (k + 1) . otherwise

(5.32)

where Rδ1 , δ2 (k + 1) and Rδ1 (k + 1) are the probabilities of a system successfully completing the (k + 1)th mission if the maintenance action corresponding to δ2 is taken or not, respectively. Hence, ηδ1 , δ2 is the increment of the probability of a system successfully completing the (k + 1)th mission if the maintenance action corresponding to δ2 is completed in the kth break. Pheromone tail updating rules: At the beginning of the tailored ACO algorithm, the pheromone on each edge of two nodes are initialized as follows  1 δ 1  = δ2 τδ1 , δ2 = . (5.33) 0 δ1 = δ2 In each search iteration, several independent ants begin traveling from the start point with a maintenance budget, and the maintenance budget is reduced when they visit a node, that is, performing maintenance actions. The travel of an ant ends if all nodes are included in the tabu list of the ant, or the remaining maintenance budget cannot offer one more maintenance action. At the end of a search iteration, the pheromone on each edge is updated by = (1 − ρ)τδ1 , δ2 + τδnew 1 , δ2

D 

τδ(d) , 1 , δ2

(5.34)

d=1

is the updated quantity of pheromone on edge (δ1 , δ2 ); ρ (0 < ρ ≤ 1) where τδnew 1 , δ2 is the evaporation rate of pheromone; D is the total number of ants in each search is the amount of pheromone laid on edge (δ1 , δ2 ) by ant d which iteration; τδ(d) 1 , δ2 is defined as  Q · Rd (k + 1) Ant d passes through edge (δ1 , δ2 ) (d) τδ1 , δ2 = , (5.35) 0 otherwise

90

5 Selective Maintenance under Stochastic Time Durations …

where Q is a pre-specified parameter of the ACO algorithm, and Rd (k + 1) is the probability if the system successfully completing the (k + 1)th mission with the maintenance sequence searched by ant d, that is, the path/trajectory of ant d. The optimization iteration will terminate once its iteration count reaches Nmax , and the optimal maintenance sequence among all iterations is the solution searched by the tailored ACO algorithm. The pseudocode of the tailored ACO algorithm is provided in Algorithm 5.1. Algorithm 5.1 The pseudocode of the tailored ACO algorithm Inputs: D: The amount of ants in each iteration; d: The index of each ant; n: The iteration counter; Nmax : The maximum number of iterations; C (d) : The remaining budget carried by each ant. 1: For every edge (δ1 , δ2 ) in which δ1 is not equal to δ2 , set the pheromone τδ1 ,δ2 = 1; 2: For n = 1 to Nmax do 3: Set τδ1 ,δ2 = 0 for each edge (δ1 , δ2 ); 4: For d = 1 to D do 5: Set Tabu = ∅, C (d) = Ck ; 6: Set the state node as node δ1 ; 7: Add the start node to Tabu; 8: While at least one node is outside the Tabu do 9: Choose next node δ2 by Eq. (5.31); 10: Ant d move from node δ1 to node δ2 ; 11: Add node v and all the nodes whose components are the same as node δ2 to Tabu; 12: C (d) = C (d) − Csl (t + ), j ; l k

13: Add all the nodes whose maintenance cost greater than C (d) to Tabu; 14: δ1 = δ 2 ; 15: End While 16: End For 17: For d = 1 to D do 18: Evaluate R(k + 1) based on the path passed by ant d; 19: End For 20: Record the optimal feasible solution of this iteration; 21: Update the pheromone of every edge (δ1 , δ2 ) by Eq. (5.34); 22: End For Output: The optimal feasible solution among all the iterations.

5.5 Illustrative Examples 5.5.1 A Four-Component System In this section, a multi-state four-component flow transmission system is exemplified to illustrate the selective maintenance strategy under stochastic time durations of breaks and maintenance actions. The structure of the system is exhibited

5.5 Illustrative Examples

91

in Fig. 5.3, where the performance capacity of the system can be written as G(t) = min{G 1 (t), G 2 (t) + G 3 (t), G 4 (t)}. At the end of the kth mission, the state of each component is perfectly observed, and the performance capacity of each component can be restored in the kth break to maximize the probability of the system successfully completing the (k + 1)th mission. The parameters of each component, including the state at the end of the kth mission, possible performance capacities, state transition intensities during the (k + 1)th mission, are tabulated in Table 5.2. Components 1 and 4 have three possible performance capacities, whereas Components 2 and 3 are binary-capacitied. The time duration and demand of the (k + 1)th mission are 1 month and 50 tons/h, respectively. The maintenance budget in the kth break was $15,000. The time duration of each maintenance action follows a gamma distribution. The distribution of each maintenance action is shown in Table 5.3, where (k , θ ) represents a gamma distribution with shape parameter k and scale parameter θ . The time duration of the kth break follows a uniform distribution in the range of [15, 40] hours. By enumerating all the possible solutions as given in Table 5.4, the optimal maintenance actions and the corresponding maintenance sequence in the kth break can be identified. As shown in Table 5.4, the sequence of maintenance action has a significant impact on the probability of the system successfully completing the (k + 1)th mission, even though the same maintenance actions are selected. For example, both 1 2 and a1,2 , but the probability of Solutions #5 and #8 select maintenance action a2,3 the system successfully completing the (k + 1)th mission in Solutions #5 (0.579) is less than Solutions #8 (0.670). In the optimal maintenance strategy (Solution #11 in Table 5.4), component 3 is firstly repaired from state 1 to state 2, and then the state of component 1 is restored to state 3. The associated maintenance cost is $11,000, which is less than the maintenance budget. The corresponding probability of the system successfully completing the (k + 1)th mission is 0.716. The details of the optimal maintenance sequence with 2 1

4 3

Fig. 5.3 A four-component system Table 5.2 Parameter settings of each component (unit of state transition intensities: month-1, unit of cost: $1000, unit of performance capacities: tons/hour) l l l ID (l) gl,1 gl,2 gl,3 λl2,1 λl3,1 λl3,2 C1.2 C1.3 C2,3 sl (tk+ ) 1 2 3 4

0 0 0 0

60 60 50 50

90 – – 100

0.2 0.09 0.07 0.2

0.05 – – 0.1

0.1 – – 0.15

3 5 6 6

7 – – 10

5 – – 5

2 1 1 2

92

5 Selective Maintenance under Stochastic Time Durations …

Table 5.3 Maintenance duration of each component (unit of time: hours) l l l ID (l) a1,2 a1,3 a2,3 1 2 3 4

(20, 0.2) (50, 0.2) (30, 0.2) (50, 0.2)

(60, 0.2) – – (80, 0.2)

(50, 0.2) – – (60, 0.2)

the maintenance budget constraint are shown in Table 5.5, where both convolution and the saddlepoint approximation (SPA) are implemented to evaluate the probability distribution of the number of completed maintenance actions in the kth break (i.e., Pk,m ). As presented in Table 5.5, the values of Pk,m evaluated the saddlepoint approximation are close to the true value from convolution. On a PC with an Intel Core(TM) i5-4590, 3.30 GHz CPU and 16 GB RAM, the runtime of convolution and saddlepoint approximation are 0.012 seconds and 0.014 seconds, respectively, and they are close in the cases that only two maintenance actions are selected. If the maintenance budget is unlimited, that is, without the constraint of the maintenance budget, the optimal maintenance sequence is executing maintenance actions 3 1 4 2 , a2,3 , a2,3 , and a1,2 in order, that is, Solution #51 in Table 5.4. The associated a1,2 maintenance cost is $21,000, and the corresponding probability of the system successfully completing the (k + 1)th mission is 0.755. The details of the optimal maintenance sequence are presented in Table 5.6. All the selected maintenance actions cannot always be completed, and the probability of all selected maintenance actions being completed in the kth break is 0.0957. The values of Pk,m evaluated by the saddlepoint approximation are close to the true value from convolution. The runtime of convolution and saddlepoint approximation are 0.113 seconds and 0.020 seconds, respectively. The runtime of the convolution extremely increases with respect to the number of selected maintenance actions, even though the time duration of each maintenance action follows a Gamma distribution with additive property, whereas the increased runtime of the saddlepoint approximation is much less. The saddlepoint approximation can evaluate Pk,m in a computationally efficient manner.

5.5.2 A Multi-state Coal Transportation System A multi-state ten-component coal transportation system is exemplified in this section. The configuration of the system is represented in Fig. 5.4, where coal is conveyed through Feeder #1, Conveyor, Stacker-reclaimer, and Feeder #2 in succession. Based on the performance capacities of all components, the performance capacity of the system is expressed as G(t) = min{G 1 (t) + G 2 (t) + G 3 (t), G 4 (t) + G 5 (t), G 6 (t) + G 7 (t) + G 8 (t), G 9 (t) + G 10 (t)}.

(5.36)

5.5 Illustrative Examples

93

Table 5.4 Feasible solutions (unit of cost: $1000) ID Sequence C(k) Rk+1 ID Sequence 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

1 a2,3 2 a1,2 3 a1,2 4 a2,3 1 ,a 2 a2,3 1,2 1 ,a 3 a2,3 1,2 1 ,a 4 a2,3 2,3 2 ,a 1 a1,2 2,3 2 ,a 3 a1,2 1,2 2 ,a 4 a1,2 2,3 3 ,a 1 a1,2 2,3 3 ,a 2 a1,2 1,2 3 ,a 4 a1,2 2,3 4 ,a 1 a2,3 2,3 4 ,a 2 a2,3 1,2 4 ,a 3 a2,3 1,2 1 ,a 2 ,a 3 a2,3 1,2 1,2 1 ,a 2 ,a 4 a2,3 1,2 2,3 1 ,a 3 ,a 2 a2,3 1,2 1,2 1 ,a 3 ,a 4 a2,3 1,2 2,3 1 ,a 4 ,a 2 a2,3 2,3 1,2 1 ,a 4 ,a 3 a2,3 2,3 1,2 2 ,a 1 ,a 3 a1,2 2,3 1,2 2 ,a 1 ,a 4 a1,2 2,3 2,3 2 ,a 3 ,a 1 a1,2 1,2 2,3 2 ,a 3 ,a 4 a1,2 1,2 2,3 2 ,a 4 ,a 1 a1,2 2,3 2,3 2 ,a 4 ,a 3 a1,2 2,3 1,2 3 ,a 1 ,a 2 a1,2 2,3 1,2 3 ,a 1 ,a 4 a1,2 2,3 2,3 3 ,a 2 ,a 1 a1,2 1,2 2,3 3 ,a 2 ,a 4 a1,2 1,2 2,3

5 5 6 5 10 11 10 10 11 10 11 11 11 10 10 11 16 15 16 16 15 16 16 15 16 16 15 16 16 16 16 16

0 0.613 0.625 0 0.579 0.683 0 0.670 0.664 0.657 0.716 0.664 0.679 0 0.500 0.608 0.614 0.600 0.710 0.716 0.233 0.378 0.725 0.711 0.722 0.695 0.688 0.685 0.743 0.750 0.723 0.695

33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

3 ,a 4 ,a 1 a1,2 2,3 2,3 3 ,a 4 ,a 2 a1,2 2,3 1,2 4 ,a 1 ,a 2 a2,3 2,3 1,2 4 ,a 1 ,a 3 a2,3 2,3 1,2 4 ,a 2 ,a 1 a2,3 1,2 2,3 4 ,a 2 ,a 3 a2,3 1,2 1,2 4 ,a 3 ,a 1 a2,3 1,2 2,3 4 ,a 3 ,a 2 a2,3 1,2 1,2 4 ,a 3 ,a 2 ,a 1 a2,3 1,2 1,2 2,3 4 ,a 4 ,a 1 ,a 2 a2,3 2,3 2,3 1,2 4 ,a 2 ,a 3 ,a 1 a2,3 1,2 1,2 2,3 4 ,a 2 ,a 1 ,a 3 a2,3 1,2 2,3 1,2 4 ,a 1 ,a 3 ,a 2 a2,3 2,3 1,2 1,2 4 ,a 1 ,a 2 ,a 3 a2,3 2,3 1,2 1,2 3 ,a 4 ,a 2 ,a 1 a1,2 2,3 1,2 2,3 3 ,a 4 ,a 1 ,a 2 a1,2 2,3 2,3 1,2 3 ,a 2 ,a 4 ,a 1 a1,2 1,2 2,3 2,3 3 ,a 2 ,a 1 ,a 4 a1,2 1,2 2,3 2,3 3 ,a 1 ,a 4 ,a 2 a1,2 2,3 2,3 1,2 3 ,a 1 ,a 2 ,a 4 a1,2 2,3 1,2 2,3 2 ,a 4 ,a 3 ,a 1 a1,2 2,3 1,2 2,3 2 ,a 4 ,a 1 ,a 3 a1,2 2,3 2,3 1,2 2 ,a 3 ,a 4 ,a 1 a1,2 1,2 2,3 2,3 2 ,a 3 ,a 1 ,a 4 a1,2 1,2 2,3 2,3 2 ,a 1 ,a 4 ,a 3 a1,2 2,3 2,3 1,2 2 ,a 1 ,a 3 ,a 4 a1,2 2,3 1,2 2,3 1 ,a 4 ,a 3 ,a 2 a2,3 2,3 1,2 1,2 1 ,a 4 ,a 2 ,a 3 a2,3 2,3 1,2 1,2 1 ,a 3 ,a 4 ,a 2 a2,3 1,2 2,3 1,2 1 ,a 3 ,a 2 ,a 4 a2,3 1,2 1,2 2,3 1 ,a 2 ,a 4 ,a 3 a2,3 1,2 2,3 1,2 1 ,a 2 ,a 3 ,a 4 a2,3 1,2 1,2 2,3

C(k)

Rk+1

16 16 15 16 15 16 16 16 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21

0.729 0.701 0.233 0.378 0.531 0.528 0.659 0.630 0.641 0.664 0.538 0.537 0.383 0.239 0.711 0.734 0.706 0.729 0.755 0.750 0.696 0.694 0.705 0.729 0.717 0.732 0.383 0.239 0.721 0.717 0.606 0.621

The maintenance cost of repairing component l from state i to j (i.e., Ci,l j ) is defined as   gl, j − gl, i × clr + clfix , (5.37) Ci,l j = gl, nl

94

5 Selective Maintenance under Stochastic Time Durations …

Table 5.5 Optimal solution with the maintenance budget constraint Sequence ID (m) ai,l j Pk,m Convolution SPA Error 1 2 R(k + 1)

3 a1,2 1 a2,3

0.0524 0.9476 0.7161

0.0532 0.9468 0.7160

R(k + 1|Nk = m)

0.0008 −0.0008 −0.0001

0.6250 0.7211 –

Table 5.6 Optimal solution with unlimited maintenance budget Sequence ID (m) ai,l j Pk,m Convolution SPA Error 1 2 3 4 R(k + 1)

3 a1,2 1 a2,3 4 a2,3 2 a1,2

0.0524 0.4676 0.3843 0.0957 0.7549

Fig. 5.4 The configuration of a multi-state coal transportation system

0.0532 0.4696 0.3854 0.0919 0.7544

R(k + 1|Nk = m)

0.0008 0.0019 0.0010 −0.0037 −0.0005

1

0.6250 0.7211 0.7915 0.8439 –

6 9

4 2

7 5

10

3

8

Feeder #1 (Subsystem 1)

Conveyor Stacker-reclaimer Feeder #2 (Subsystem 2) (Subsystem 3) (Subsystem 4)

where clfix and clr are the fixed maintenance cost and additional replacement cost of component l, respectively. The time duration of each maintenance action follows a non-negative truncated normal distribution, that is, the time duration of action ai,l j follows a normal distribution with mean μli, j and variance σi,l j truncated in the range [0, +∞), and its parameters are formulated as ⎧   ⎨ μl = gl, j −gl,i × t r + t fix i, j l l gl,nl , (5.38) ⎩ σ l = μl(i, j) −μl(i, j−1) i, j

3

where tlfix and tlr are the fixed maintenance duration and additional replacement time of component l, respectively. The parameters of each component, including the possible performance capacities, state at the end of the kth mission, and the parameters of maintenance cost and maintenance duration, are listed in Table 5.7. The state transition intensities of each component are tabulated in Table 5.8.

5.5 Illustrative Examples

95

Table 5.7 The state transition intensities of each component (month−1 ) ID (l) λl2,1 λl3,1 λl3, 2 λl4, 1 λl4, 2 1 2 3 4 5 6 7 8 9 10

0.5 0.3 0.2 0.5 0.2 0.4 0.3 0.2 0.4 0.3

0.2 0.2 0.4 0.3 0.2 0.2 0.15 0.1 0.2 0.2

0.3 0.3 0.3 0.2 0.2 0.25 0.12 0.18 0.1 0.15

0.25 0.15 0.2 – – 0.3 0.3 0.15 0.2 0.2

λl4, 3

0.2 0.3 0.4 – – 0.4 0.2 0.3 0.2 0.2

0.2 0.2 0.3 – – 0.3 0.4 0.5 0.4 0.3

Table 5.8 Parameter settings of each component (unit of time: hours, unit of cost: $1000, unit of performance capacity: tons/hour) ID (l) gl,1 gl,2 gl,3 gl,4 clfix clr tlfix tlr sl (tk+ ) 1 2 3 4 5 6 7 8 9 10

0 0 0 0 0 0 0 0 0 0

20 26 15 35 44 17 20 27 15 20

31 45 40 67 70 43 36 35 25 40

54 53 50 – – 52 49 50 40 60

1.2 1 1.1 1.1 1.2 0.75 1.15 0.8 1.4 1

16 19 27 18 20 10 23 12 17 15

0.25 0.25 0.25 0.30 0.30 0.15 0.30 0.10 0.40 0.20

20 15 20 12 20 10 15 12 20 15

2 1 3 1 2 2 3 2 2 3

The time duration of the kth break follows a uniform distribution in the range of [20, 40] hours. The maintenance budget in the kth break is set to be $40,000. The time duration and demand of the (k + 1)th mission are 0.5 months and 30 tons/h, respectively. In this example, enumerating all feasible solutions is impracticable. Hence, the tailored ACO algorithm introduced in Sect. 5.4.2 is utilized to search the optimal solution, where the parameters of the tailored ACO algorithm are set to D = 10, α = 1.0, β = 0.9, ρ = 0.1, Q = 1, and Nmax = 50. The optimal maintenance sequences of the selective maintenance problem, with and without the constraint of maintenance budget, are presented in Table 5.9. With the constraint of maintenance budget Ck = $40,000, the probability of the system successfully completing the (k + 1)th mission is 0.878, and the corresponding maintenance cost is $36,207. Only five of the ten components are selected for maintenance. If the maintenance budget in the kth break is unlimited, the probabil-

96

5 Selective Maintenance under Stochastic Time Durations …

Table 5.9 Optimal solutions with and without the maintenance budget constraint Sequence Limited budget Unlimited budget ID (m) ai,l j Pk,m R(k + 1| ai,l j Pk,m Nk = m) 1 2 3 4 5 6

1 a2,3 4 a1,2 9 a2,4 8 a2,3 3 a3,4 –

0 0.194 0.090 0.220 0.495 –

0.750 0.811 0.875 0.889 0.900 –

1 a2,3 4 a1,2 9 a2,4 8 a2,3 2 a1,2 3 a3,4

0 0.194 0.090 0.220 0.339 0.114

R(k + 1| Nk = m) 0.750 0.811 0.875 0.889 0.915 0.923

ity of the system successfully completing the (k + 1)th mission is 0.882 which is resolved by the tailored ACO algorithm, and the corresponding maintenance cost is $46,528. It is worth noting that, in the scenarios of stochastic time durations of breaks and maintenance actions, not all components are repaired to their best states in the optimal solution, even though maintenance resources are unlimited. In this example, the runtime of the saddlepoint approximation is less than 0.06 seconds, even though at most 10 maintenance actions can be selected in a break. Hence, compared to the runtime in Sect. 5.5.1, the runtime of the saddlepoint approximation will not extremely increase with respect to the number of selected maintenance actions, and Pk,m can be evaluated in a computationally efficient manner. By changing the maintenance budget from $5,000 to $50,000, the optimal maintenance sequences, as well as the worst maintenance sequences of the selected maintenance actions are presented in Table 5.10, and the corresponding probability of the system successfully completing the (k + 1)th mission are presented in Fig. 5.5. As shown in Table 5.10 and Fig. 5.5, with the increase of maintenance budget, more maintenance actions are selected by the optimal maintenance sequences. If the maintenance budget is less than $20,000, the probability of the system successfully completing the (k + 1)th mission will not be affected by the sequence of selected maintenance actions. If the maintenance budget is greater than $25,000, the probability of the system successfully completing the (k + 1)th mission of the optimal maintenance sequences is greater than the worst maintenance sequences, and the gap between the optimal and worst maintenance sequences increases with respect to maintenance budget. To investigate the influence of the maintenance budget and the duration of the kth break, the maintenance budget is changed from $5,000 to $50,000, and the average time duration of the kth break varies from 10 hours to 50 hours, where the width of the uniform distribution remains 20 hours. For various combinations of the maintenance budget and the average time duration of the kth break, the maximum achievable probabilities of the system successfully completing the (k+1)th mission are depicted in Fig. 5.6. The probability of the system successfully completing the (k+1)th mission is a non-decreasing function of the maintenance budget and the average time duration of the kth break.

5.5 Illustrative Examples

97

Table 5.10 The optimal maintenance sequences and the worst maintenance sequences of the maintenance actions which selected by the optimal maintenance sequences Budget Maintenance sequence R(k + 1) $5000 $10,000 $15,000 $20,000 $25,000 $30,000 $35,000 $40,000 $45,000 $50,000

Optimal Worst Optimal Worst Optimal Worst Optimal Worst Optimal Worst Optimal Worst Optimal Worst Optimal Worst Optimal Worst Optimal Worst

Fig. 5.5 The probability of the system successfully completing the (k+1)th mission of the optimal and worst maintenance sequences with respect to different settings of maintenance budget

1 a2,3 1 a2,3 1 , a8 a2,3 2,3 8 , a1 a2,3 2,3 1 , a4 a2,3 1,2 4 , a1 a1,2 2,3 1 , a4 , a8 a2,3 1,2 2,3 8 4 , a1 a2,3 , a1,2 2,3 1 4 , a8 , a3 a2,3 , a1,2 2,3 3,4 8 , a4 , a3 , a1 a2,3 1,2 3,4 2,3 1 , a4 , a9 , a8 a2,3 1,2 2,4 2,3 8 9 , a4 , a1 a2,3 , a2,4 1,2 2,3 1 4 , a9 , a8 a2,3 , a1,2 2,4 2,3 8 , a9 , a4 , a1 a2,3 2,4 1,2 2,3 1 , a4 , a9 , a8 , a3 a2,3 1,2 2,4 2,3 3,4 8 , a9 , a4 , a3 , a1 a2,3 2,4 1,2 3,4 2,3 1 , a4 , a9 , a8 , a2 a2,3 1,2 2,4 2,3 1,2 8 9 , a4 , a2 , a1 a2,3 , a2,4 1,2 1,2 2,3 1 4 , a9 , a8 , a2 , a3 a2,3 , a1,2 2,4 2,3 1,2 3,4 8 , a9 , a4 , a3 , a2 , a1 a2,3 2,4 1,2 3,4 1,2 2,3

0.750 0.750 0.761 0.761 0.811 0.811 0.824 0.824 0.834 0.833 0.872 0.842 0.872 0.842 0.878 0.831 0.881 0.825 0.882 0.819

5 Selective Maintenance under Stochastic Time Durations …

Probability of A System Successfully Completing The (k+1)th Mission

98

Fig. 5.6 The probability of the system successfully completing the (k+1)th mission versus the average duration of the kth break and the maintenance budget

5.6 Closure Under the scenarios with stochastic time durations of breaks and maintenance actions, a selective maintenance problem for multi-state systems was introduced in this chapter. Not all the planned maintenance actions can be completed in a break, and the maintenance actions performed earlier were more likely to be completed than the later ones. Hence, the maintenance actions in a break were selected and ordered to ensure the success of the next mission. The probability distribution of the number of completed maintenance actions in a break was formulated to evaluate the probability of a system successfully completing the next mission, where the saddlepoint approximation was utilized to address the involved multi-dimensional convolution computation. The resulting selective maintenance problem was formulated as an NP-hard combinational optimization problem, and a tailored ACO algorithm was introduced to mitigate the computational burden arised from full enumeration. An illustrative example with a four-component system was given to exemplified the impact of maintenance sequences, and a ten-component coal transportation system was presented to show the solution of the tailored ACO algorithm for large-scale systems.

References

99

References 1. Cassady CR, Murdock WP, Pohl EA (2001) Selective maintenance for support equipment involving multiple maintenance actions. Eur J Oper Res 129(2):252–258 2. Cassady CR, Pohl EA, Murdock WP (2001) Selective maintenance modeling for industrial systems. J Qual Maintenance Eng 7(2):104–117 3. Daniels HE (1954) Saddle point approximations in statistics. Ann Math Stat 25(4):631–650 4. Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern Part B (Cybern) 26(1):29–41 5. Du X, Sudjianto A (2004) First order saddle point approximation for reliability analysis. AIAA J 42(6):1199–1207 6. Khatab A, Aghezzaf EH, Diallo C, Djelloul I (2017) Selective maintenance optimisation for series-parallel systems alternating missions and scheduled breaks with stochastic durations. Int J Prod Res 55(10):3008–3024 7. Khatab A, Aghezzaf EH, Djelloul I, Sari Z (2017) Selective maintenance optimization for systems operating missions and scheduled breaks with stochastic durations. J Manuf Syst 43:168–177 8. Levitin G, Lisnianski A (2000) Optimization of imperfect preventive maintenance for multistate systems. Reliab Eng Syst Saf 67(2):193–203 9. Liu Y, Huang HZ (2010) Optimal selective maintenance strategy for multi-state systems under imperfect maintenance. IEEE Trans Reliab 59(2):356–367 10. Meng MH, Zuo M et al (1999) Selective maintenance optimization for multi-state systems. In: 1999 IEEE Canadian conference on electrical and computer engineering, vol 3, pp 1477–1482 11. Ushakov I (1986) A universal generating function. Sov J Comput Syst Sci 24(5):118–129

Chapter 6

Robust Selective Maintenance under Imperfect Observations

6.1 Introduction The decision-making of selective maintenance substantially relies on the condition of all the components in a repaired system at the end of the last mission, the limited maintenance resource constraints, and the reliability or survival probability requirement of the next mission. The traditional models for selective maintenance problems assume that the conditions of all components in the system can be accurately measured. In the real-world applications, the condition of a component can be indirectly detected by various monitoring techniques, such as vibration or ultrasonic. However, in most industrial applications, limited accuracy of sensors or inspection instruments, poor diagnostic tools or algorithms, and non-rigorous interpretation varying from person to person are inevitable. The observations from sensors or experts’ judgments often contain noise and uncertainty. Therefore, these observations cannot completely reveal the true condition of components and the entire repairable system [3, 4]. Such observations are oftentimes referred to as “indirect condition monitoring” [10], “partial observations” [7], or “imperfect observations” [3, 4]. This chapter introduces the selective maintenance optimization problem involving imperfect observations. The imperfection of observations introduces additional uncertainties into the condition (both the state and effective age) of each component. Such uncertainties will be propagated to the evaluation of system performance and mission completion, and eventually, produce a non-negligible impact on the decision-making of selective maintenance. It may be untrustworthy to choose a selective maintenance strategy while ignoring the unavoidable uncertainties associated with imperfect observations, as the evaluation result of system performance and mission completion under such the identified strategy may be inconsistent with its expected value. This chapter was therefore motivated to incorporate the uncertainties associated with imperfect observations into selective maintenance optimization. A multi-objective selective maintenance optimization model was put forth to identify the best selective maintenance strategy that can achieve a high system performance in the next mission while guaranteeing the robustness of the strategy under uncertainty from imperfect observations. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_6

101

102

6 Robust Selective Maintenance under Imperfect Observations

The rest of this chapter was organized as follows. Section 6.2 introduced the specific selective maintenance problem. Section 6.3 briefly reviewed the Kijima type-II imperfect maintenance model. The uncertainty quantification and propagation were conducted in Sect. 6.4, and then the conditional survival probability of a component was obtained. In Sect. 6.5, the expectation and variance of the probability of a system successfully completing a mission were derived. Section 6.6 formulated the robust selective maintenance optimization model. Two illustrative examples and some fundamental analysis results were presented in Sect. 6.7. Section 6.8 was the closure.

6.2 Problem Statements and Model Assumptions A repairable system consisting of multiple repairable components is required to operate a sequence of consecutive missions with breaks between two adjacent missions. When the system has completed the last mission (i.e., the kth mission), it will be maintained during the following break (i.e., the kth break). Due to the limitation of maintenance resources and the imperfection of observations, a selective maintenance strategy with the uncertainties raised from imperfect observations must be determined after the kth mission to ensure the success of the next mission (i.e., the (k + 1)th mission). Three different scenarios of selective maintenance decision processes are delineated in Fig. 6.1. In the traditional selective maintenance optimization models, the condition of each component is assumed to be perfectly observed, and in most cases, the objective is to maximize the probability of the system successfully completing the next mission. Therefore, an optimal and unique selective maintenance strategy (Strategy 1) can be identified for such a certain scenario (Scenario 1). However, the condition of components cannot be perfectly observed due to the limited inspection techniques. Under such a circumstance, if the imperfect observations are treated as true condition of components by decision-makers, an optimal strategy (Strategy 2) can also be identified by the traditional selective maintenance optimization models (Scenario 2). A difference between the two results (i.e., “Maintenance Result 1” under “Strategy 1” and “Maintenance Result 2” under “Strategy 2”) may exist unsurprisingly. To eliminate the discrepancy and achieve a better maintenance result, a robust selective maintenance optimization model is developed in this chapter with the consideration of the uncertainties associated with imperfect observations. An “optimal” selective maintenance strategy (Strategy 3) and corresponding maintenance result (Maintenance Result 3) can be obtained. In this chapter, we expect to reveal that “Strategy 3” is superior to “Strategy 2,” and “Maintenance Result 3” is closer to “Maintenance Result 1” than “Maintenance Result 2” to “Maintenance Result 1.” The assumptions pertaining to the repairable system in question and the related selective maintenance problem in this chapter are summarized below.

6.2 Problem Statements and Model Assumptions

103

Fig. 6.1 Selective maintenance decisions for three scenarios

• The system is composed of M repairable and s-independent binary-state (either working perfectly or failed completely) components that can be configurated arbitrarily. • The states of component i at the beginning and at the end of the kth are represented by  1 if component i is functioning X i,k = 0 if component i is failed 

and Yi,k =

1 0

if component i is functioning , if component i is failed

respectively. Obviously, the relationship X i,k ≥ Yi,k always holds. • The time duration of the kth mission is denoted by a constant z k . The failure time of component i from the beginning of the kth mission is denoted by Ti,k (Ti,k ∈ [0, +∞)), whereas the working time of the component in the kth mission can be w w w (Ti,k ∈ [0, z k ]). If X i,k = 0, then Ti,k = 0; if represented by a random variable Ti,k w w = zk the component fails within the kth mission, then Ti,k ∈ (0, z k ); otherwise, Ti,k as the component remains functioning throughout the kth mission. • At the end of the kth mission, the state and working time of each component are observable. The observation errors associated with the state and working time are characterized by the observation probability matrix and conditional probability density function, respectively. • The observed state of component i at the end of the kth mission is represented by o Yi,k

 1 = 0

if component i is observed in the functioning state . if component i is observed in the failure state

The observed value of the working time of component i in the kth mission is w,o denoted by Ti,k .

104

6 Robust Selective Maintenance under Imperfect Observations

6.3 Imperfect Maintenance Model In this chapter, the Kijima type-II model [6] is utilized to characterize the imperfect maintenance efficiency of a maintenance action. The Kijima type-II model assumes an age-reduction mechanism in which an imperfect maintenance action can reduce the physical age (therein call effective age) of a maintained component proportionally just before maintenance. The effective age of component i at the beginning of the kth mission is denoted by Ui,k ; the true and observed values of the effective age at the end of the kth mission o , respectively. Therefore, we can have are denoted by Vi,k and Vi,k w Vi,k = Ui,k + Ti,k

(6.1)

w,o o = Ui,k + Ti,k . Vi,k

(6.2)

and

During the kth break, three maintenance options can be chosen for each component as following: Option #1 Do Nothing (DN). DN means no maintenance action is chosen to be performed. Therefore, the condition of component i remains unchanged after the break, and we have X i,k+1 = Yi,k and Ui,k+1 = Vi,k . Option #2 Component Replacement (CR). CR, as a sort of perfect maintenance, replaces a component by a brand-new one. Obviously, we have X i,k+1 = 1 and Ui,k+1 = 0. Option #3 Imperfect Preventive/Corrective Maintenance (IPM/ICM). IPM/ICM refers to a preventive/corrective maintenance action performed on a functioning/failed component. Based on the Kijima type-II model, the effective age of a functioning/failed component i after an IPM/ICM action in the kth break is formulated by Ui,k+1 = bi,k Vi,k ,

(6.3)

where age reduction factor bi,k ∈ [0, 1] is associated with the maintenance efficiency. A smaller value of bi,k indicates a more efficient maintenance action, while consuming more maintenance resources. For a functioning component i, bi,k = 1, bi,k = 0, and bi,k ∈ (0, 1) correspond to the cases of DN, preventive CR, and IPM, respectively. For a failed component i, bi,k = 1, bi,k = 0, and bi,k ∈ (0, 1) correspond to the cases of minimal repair (MR), corrective CR, and ICM, respectively. Particularly, an ICM action for a failed component can be separated into two progressive stages [8]: (1) the failed component is first restored to the functioning state by MR; (2) an additional maintenance action is subsequently performed to further reduce the effective age.

6.4 Survival Probability of a Component under Imperfect Observations

105

The value of bi,k is closely related to the maintenance cost allotted to component i in the kth break. The maintenance cost of component i in the kth break is given by Ci,k = ci0 + ci,k ,

(6.4)

where ci0 and ci,k are the fixed maintenance cost and the variable preventive/corrective maintenance (PM/CM) cost for component i. Additionally, the variable maintenance costs of component i for MR and CR are denoted by cimin and cicr , respectively. Consequently, the relation between bi,k and ci,k can be formulated by [8] ⎧   1

⎪ ⎨1 − ci,kcr ζi,k Yi,k = 1, ci,k ∈ 0, cicr ci bi,k = (6.5)  ζ1   min cr , min ⎪ i,k i ⎩1 − ci,k −c Y = 0, c ∈ c , c i,k i,k i i ccr i

where ζi,k represents a characteristic parameter of which the value can be evaluated by the function of the effective age and mean residual life of the component [8].

6.4 Survival Probability of a Component under Imperfect Observations 6.4.1 State and Effective Age under Imperfect Observations Due to the uncertainties associated with the imperfect observations, there exists a difference between the observed and true values of the condition of each component, that is, the state and effective age. Hence, it necessitates the uncertainty quantification of the state and effective age. The observation probability matrix of component i is introduced to quantify the stochastic relation between the observed and true values of states, and it is denoted by

1 − δiI δiI , Oi = δiII 1 − δiII o o where δiI = Pr{Yi,k = 0 | Yi,k = 1} and δiII = Pr{Yi,k = 1 | Yi,k = 0}. The survival probability of component i in the kth mission conditional on Ui,k = u can be derived as

Ri,k (t | u) = Pr{Ti,k > t | Ui,k = u} =

Ri,0 (u + t) , Ri,0 (u)

(6.6)

where Ri,0 (·) is the unconditional survival function of component i. Therefore, the prior state probabilities of component i at the end of the kth mission are given by Pr{Yi,k = 1} = Ri,k (z k | u) and Pr{Yi,k = 0} = 1 − Pr{Yi,k = 1}. Consequently, the

106

6 Robust Selective Maintenance under Imperfect Observations

o Table 6.1 Relation between Vi,k and Vi,k

Case

Yi,k

o Yi,k

w,o Ti,k

o Vi,k

1 2 3 4

1 1 0 0

1 0 1 0

zk t zk t

Ui,k Ui,k Ui,k Ui,k

Vi,k + zk + t + zk + t

Ui,k + z k Ui,k + z k w Ui,k + Ti,k o (v | u, v  ) f Vi,k |Ui,k ,Vi,k

posterior state distribution of component i can be formulated based on the Bayes rule as o }= Pr{Yi,k | Yi,k

o Pr{Yi,k , Yi,k } o Pr{Yi,k }

=

o Pr{Yi,k | Yi,k } Pr{Yi,k } Yi,k

o Pr{Yi,k | Yi,k } Pr{Yi,k }

.

(6.7)

The relation between the observed and true values of the effective age can be tabulated in Table 6.1. o Case 1: Yi,k = 1 and Yi,k =1 In this case, the true value of the effective age is equal to the observed value, that is, o = Ui,k + z k . Vi,k = Vi,k o Case 2: Yi,k = 1 and Yi,k =0 o = Ui,k + t  , In this case, even though the observed value of the effective age is Vi,k its true value equates to Vi,k = Ui,k + z k . o Case 3: Yi,k = 0 and Yi,k =1 In this case, component i fails within the kth mission, and thus the failure time is w w ∈ (0, z k ). The truncated probability density function (PDF) of Ti,k truncated and Ti,k conditional on Ui,k = u can be given by

f Ti,kw |Ui,k (t | u) =

f Ti,k |Ui,k (t | u) , 1 − Ri,k (z k | u)

(6.8)

where f Ti,k |Ui,k (t | u) = −dRi,k (t | u)/dt. The relation between f Ti,kw |Ui,k (t | u) and f Ti,k |Ui,k (t | u) is illustrated in Fig. 6.2a. The true value of the effective age is Vi,k = w ; therefore, the PDF of Vi,k conditional on Ui,k = u is given by Ui,k + Ti,k f Vi,k |Ui,k (v | u) = f Ti,kw |Ui,k (v − u | u).

(6.9)

o =0 Case 4: Yi,k = 0 and Yi,k In this case, component i fails before the kth break and the failure is detected. A conditional PDF f Ti,kw,o |Ti,kw (t  | t) is introduced to quantify the stochastic relation between the observed and true values of the working time. For instance, a truncated conditional normal distribution can be formulated by

6.4 Survival Probability of a Component under Imperfect Observations

107

Fig. 6.2 Relation among PDFs

exp{−(t  − t)2 /(2σi2 )} f Ti,kw,o |Ti,kw (t  | t) = √ , 2π σi [(z k ) − (0)]

(6.10)

where (t) is the cumulative distribution function (CDF) of a normal distribution N (t, σi2 ). The variance σi2 is used to quantify the uncertainty of the working time. o conditional on Ui,k = u and Vi,k = v is given by As a result, the PDF of Vi,k f Vi,ko |Ui,k ,Vi,k (v  | u, v) = f Ti,kw,o |Ti,kw (v  − u | t).

(6.11)

o = v  can be The posterior PDF of the effective age conditional on Ui,k = u and Vi,k calculated based on the Bayes rule as following:

f Vi,k |Ui,k ,Vi,ko (v | u, v  ) = 

f Vi,ko |Ui,k ,Vi,k (v  | u, v) f Vi,k |Ui,k (v | u) f Vi,ko |Ui,k ,Vi,k (v  | u, v) f Vi,k |Ui,k (v | u)dv

.

(6.12)

Vi,k

To sum up, in Cases 1 and 2, component i remains in the functioning state with o }, and the true value of the effective age takes Vi,k = probability Pr{Yi,k = 1 | Yi,k Ui,k + z k . In Cases 3 and 4, component i fails before the kth break with probability o }, and the true value of the effective age follows a distribution with Pr{Yi,k = 0 | Yi,k the following conditional PDF  o f Vi,k |Ui,k (v | u) Yi,k =1 f V∗i,k |Ui,k (v | u) = . (6.13) o  o f Vi,k |Ui,k ,Vi,k (v | u, v ) Yi,k = 0

108

6 Robust Selective Maintenance under Imperfect Observations

6.4.2 State and Effective Age after Maintenance 1 0 For notational convenience, let Ui,k+1 and Ui,k+1 denote the effective ages at the beginning of the (k + 1)th mission for the cases of Yi,k = 1 and Yi,k = 0, respectively. The state and effective age of a component after maintenance can be categorized into two cases.

Case 1: Yi,k = 1 In this case, component i remains functioning at the beginning of the (k + 1)th 1 = Ui,k + z k . If CR is chosen, mission (i.e., X i,k+1 = 1). If DN is chosen, then Ui,k+1 1 1 then Ui,k+1 = 0. If IPM is chosen, then Ui,k+1 = bi,k Vi,k = bi,k (Ui,k + z k ). Case 2: Yi,k = 0 In this case, component i fails before the end of the kth mission. If DN is chosen, then 0 = Vi,k . However, we can have X i,k+1 = 1 once a maintenance X i,k+1 = 0 and Ui,k+1 0 action is conducted. Likewise, if CR is chosen, then Ui,k+1 = 0. If ICM is chosen, 0 0 the effective age changes to be Ui,k+1 = bi,k Vi,k , and the PDF of Ui,k+1 conditional on Ui,k = u is given by



f V∗i,k |Ui,k u  /bi,k | u 

0 fUi,k+1 |Ui,k u | u = bi,k  



◦ f Vi,k |Ui,k u /bi,k | u /bi,k Yi,k =1 



= . ◦  f Vi,k |Ui,k ,Vi,k0 u /bi,k | u, v /bi,k Yi,k = 0

(6.14)

o 0 Particularly, if Yi,k = 1, the relationship between the PDFs of Ui,k+1 and Vi,k is illustrated in Fig. 6.2b.

6.4.3 Survival Probability of a Component The survival probability of component i at the end of the (k + 1)th mission conditional on Ui,k+1 = u  can be computed by Ri,0 (u  + z k+1 ) Ri,0 (u  )  1 Ri,k+1 (z k+1 | u  ) Yi,k = 1 . = 0 (z k+1 | u  ) Yi,k = 0 Ri,k+1

Ri,k+1 (z k+1 | u  ) =

(6.15)

Consequently, the expectation and variance of the survival probability of component i are formulated as

6.5 Probability of a System Successfully Completing a Mission

109

  m i,k+1 (z k+1 ) = E Ri,k+1 (z k+1 | u  ) o 1 = Pr{Yi,k = 1 | Yi,k } · Ri,k+1 (z k+1 | u  )  o 0   0 }· Ri,k+1 (z k+1 | u  ) fUi,k+1 + Pr{Yi,k = 0 | Yi,k |Ui,k (u | u)du . 0 Ui,k+1

(6.16) and





Di,k+1 (z k+1 ) = Var Ri,k+1 (z k+1 | u  )   . 2 = E Ri,k+1 (z k+1 | u  ) − m i,k+1 (z k+1 )2

(6.17)

respectively, where   2 2 o 1 } · [Ri,k+1 (z k+1 | u  )] E Ri,k+1 (z k+1 | u  ) = Pr{Yi,k = 1 | Yi,k o + Pr{Yi,k = 0 | Yi,k }  2 0   0 [Ri,k+1 (z k+1 | u  )] fUi,k+1 · |Ui,k (u | u)du .

(6.18)

0 Ui,k+1

To mitigate the computational burden in solving the integrals in Eqs. (6.16) and (6.18), the Riemann sum is utilized to calculate their approximations (see the book by Stewart [9]).

6.5 Probability of a System Successfully Completing a Mission Let Ys,k+1 and φ(·) represent the system state at the end of the (k + 1)th mission and the system structure function, respectively. The system state can be, therefore, derived by the composition of the states of all the M components, and we have Ys,k+1 = φ(Y1,k+1 , . . . , Y M,k+1 ). As a result, the probability of the system successfully completing the (k + 1)th mission conditional on Uk+1 = u can be expressed by Rs,k+1 (z i,k+1 | u) = Pr{Ys,k+1 = 1 | Uk+1 = u},

(6.19)

where Uk+1 = (U1,k+1 , . . . , U M,k+1 ). Taking account of the uncertainty associated with the effective ages, the expectation of the probability of the system successfully completing the (k + 1)th mission can be evaluated by a multiple integral as following:

110

6 Robust Selective Maintenance under Imperfect Observations

  m s,k+1 (z k+1 ) = E Rs,k+1 (z i,k+1 | u)    = ··· Rs,k+1 (z i,k+1 | u)du 1 du 2 · · · du M . U1,k+1 U2,k+1

(6.20)

U M,k+1

The computation of the multiple integral in Eq. (6.20) is oftentimes tedious. For a serial or parallel system with n components {i 1 , i 2 , . . . , i n }, the expectation of the probability of the system successfully completing the (k + 1)th mission can be readily derived by m series,k+1 (z k+1 ) =

n 

m i j ,k+1 (z k+1 )

(6.21)

j=1

and m parallel,k+1 (z k+1 ) = 1 −

n 

1 − m i j ,k+1 (z k+1 ) ,

(6.22)

j=1

respectively. Therefore, based on Eqs. (6.16), (6.21) and (6.22), as well as the system structure function, the expectation of the probability of the system successfully completing the (k + 1)th mission can be evaluated. Likewise, for a serial or parallel system with n components {i 1 , i 2 , . . . , i n }, the variance of the probability of the system successfully completing the (k + 1)th mission can be derived by [2, 5]

Dseries,k+1 (z i,k+1 ) =

n  j=1

n

 Di j (z k+1 ) + m i j (z k+1 )2 − m i j (z k+1 )2

(6.23)

j=1

and Dparallel,k+1 (z i,k+1 ) =

n  

n

2  

2 Di j (z k+1 ) + 1 − m i j (z k+1 ) − 1 − m i j (z k+1 ) ,

j=1

j=1

(6.24) respectively. Based on Eqs. (6.16), (6.17), (6.23) and (6.24), and the system structure function, the variance of the probability of the system successfully completing the (k + 1)th mission, denoted by Ds,k+1 (z k+1 ), can be evaluated.

6.6 Robust Selective Maintenance Modelling In the traditional maintenance optimization models, decision-makers desire to find an optimal maintenance strategy that can maximize the expectation of the probability of a repairable system successfully completing the (k + 1)th mission. However, only

6.6 Robust Selective Maintenance Modelling

111

maximizing the expectation is insufficient when uncertainties associated with observations are involved, because a selective maintenance strategy with the maximum expectation may not be credible if the result contains a huge uncertainty [2, 11]. m s,k+1 (z k+1 ) represents the performance of a selective maintenance strategy, while Ds,k+1 (z k+1 ) reflects its robustness. Therefore, a robust selective optimization model is formulated in this chapter as a bi-objective optimization problem. Suppose that the maintenance budget is C0 , then the robust selective optimization model can be formulated as following   maximize m s,k+1 (z k+1 ), −Ds,k+1 (z k+1 ) Ck

(6.25)

subject to M 

Ci,k ≤ Ck ,

(6.26)

i=1

ci,k ≤ cicr ,

(6.27)

ci,k ≥ 0, i ∈ {1, . . . , M}.

(6.28)

The resulting optimization model provides a multi-objective perspective on selective maintenance under imperfect observations. In this chapter, the Pareto-optimal set composed of multiple non-dominated solutions that correspond to multiple optional selective maintenance strategies is to be identified rather than a unique “optimal” solution. Furthermore, decision-makers can choose a best-compromised solution from the Pareto-optimal set based on their preferences or requirements. The final decision in question relies on the relative importance between the expectation and variance. To reduce the computational burden, the decision variables, i.e., maintenance costs ci,k , are simplified to discretized maintenance levels li,k that are decimal integers. Therefore, each solution can be represented by a string s = (l1,k , l2,k , . . . , l M,k ), where li,k ∈ {0, 1, . . . , NL } and NL is the maximum maintenance level. The relation among the maintenance action, cost, and level are tabulated in Table 6.2. Meta-heuristic algorithms can be utilized to efficiently resolve the resulting optimization problem. In this chapter, the multi-objective particle swarm optimization (MOPSO) [1] is customized to seek for the Pareto-optimal set. The pseudo-code of the customized MOPSO is given in Algorithm 6.1. More details of the original MOPSO can be found in [1]. In the selective maintenance optimization problem, an non-dominated solution in the Pareto-optimal set can exactly be on the constraint bound or sufficiently close to the constraint bound because of the discretized maintenance costs. We can therefore improve the particle via such a consideration.

112

6 Robust Selective Maintenance under Imperfect Observations

Algorithm 6.1 Pseudo-code of the customized MOPSO Inputs: Nite : Maximum number of iteration; Npop : Population size; Nrep : Repository size; w: Inertia weight; rw : Inertia weight damping rate; c1 : Personal learning coefficient; c2 : Global learning coefficient; n grid : Number of Grids per Dimension; αinf : Inflation rate; βlead : Leader selection pressure; γdel : Deletion selection pressure; pm : Mutation rate. 1: For each paticle in the swarm do 2: Initialize the position and velocity randomly; 3: IF the index of the particle is less than Npop /2 then 4: Improve the particle; 5: End If 6: Calculate the objectives, including the expectation and variance; 7: Update the personal best; 8: End For 9: Find feasible particles; 10: Determine the domination of the feasible particles; 11: Update repository (R E P) with all the non-dominated feasible particles; 12: Create the grid and find the grid index of each particle in R E P; 13: For i ite < Nite do 14: For each paticle in the swarm do 15: Select a particle leader from R E P; 16: Update the velocity and position; 17: If the index of the particle is less than Npop /2 then 18: Improve the particle; 19: End If 20: Calculate the objectives, including the expectation and variance; 21: If rand < (1 − (i ite − 1)/(Nite − 1))1/ pm then 22: Do the Mutation operation for the (old) particle, and calculate the objectives of the new particle; 23: If the old particle is dominated by the new one then 24: Replace the old particle by the new one; 25: Else If the new particle is not dominated by the old one & rand < 0.5 then 26: Replace the old particle by the new one; 27: End If 28: End If 29: End For 30: Find feasible particles; 31: Determine the domination of the feasible particles; 32: Add all the non-dominated feasible particles to R E P; 33: Determine the domination of the new R E P, and update R E P with all the non-dominated feasible particles; 34: Update the grid and grid index of each particle in R E P; 35: If the size of R E P is greater than Nite then 36: Delete the extra particles; 37: End If 38: Update the inertia weight w = w × rw ; 39: End For Output: The Pareto-optimal set.

6.7 Illustrative Examples

113

Table 6.2 Relation among the maintenance action, cost, and level li,k Ci,k Action Yi,k = 1 0 1 .. .

0 ci0 + cimin .. .

li,k .. . NL

ci0 + cimin + (li,k − 1) .. . ci0 + cicr

cicr −cimin NL −1

Yi,k = 0

DN IPM .. .

DN MR .. .

IPM .. . Preventive CR

ICM .. . Corrective CR

6.7 Illustrative Examples 6.7.1 A Five-Component System A five-component system, as shown in Fig. 6.3, is studied in this section as an illustrative example to examine the proposed robust selective maintenance optimization model. The system consists of two subsystems in series, and subsystems 1 and 2 have two and three components configurated parallelly, respectively. The failure time of component i is assumed to follow the Weibull distribution with scale parameter θi

Fig. 6.3 Configuration of the five-component system Table 6.3 Parameters of components (unit of cost: $1,000, unit of time: day) w,o o ID θi βi ci0 cimin cicr Yi,k Ui,k Ti,k 1 2 3 4 5

26 30 14 18 24

1.8 1.7 2.4 2.2 1.9

2.5 2.5 1.8 1.8 1.8

2.5 3 2 2 2

40 48 30 32 35

1 0 0 1 0

3 1 3 4 2

1 0 0 1 0

o Vi,k

5 10 5 10 4

114

6 Robust Selective Maintenance under Imperfect Observations

Fig. 6.4 Optimization results

and shape parameter βi . The associated parameters of each component are listed in Table 6.3. The time durations of both the kth and (k + 1)th missions are assumed to take z k = z k+1 = 10 days. Ten maintenance levels (i.e., NL = 10) can be chosen for each component. The observation errors for each component are assumed to be identical and take δiI = δiII = 0.2 and σi = 2 (i ∈ {1, 2, 3, 4}). For the simple five-component system, the enumeration method is utilized to identify the Pareto-optimal set. Given the maintenance budget Ck = $70,000, five non-dominated solutions can be found in the feasible domain. The Pareto-optimal set of all non-dominated solutions is delineated in Fig. 6.4a. A best-compromised selective maintenance strategy can be chosen for the five non-dominated solutions based on the requirements and preferences of decision-makers. For instance, the best-compromised strategy can be determined based on the weighted sum of the expectation and standard deviation, denoted by σs,k+1 (z k+1 ), of the probability of the system successfully completing the next mission. In this chapter, three optional criteria that can be utilized to choose the best-compromised strategy are listed as follows. Criterion 1 The non-dominated solution with the maximum value of m s,k+1 (z k+1 ). Criterion 2 The non-dominated solution with the minimum value of Ds,k+1 (z k+1 ). Criterion 3 The non-dominated solution with the maximum value of m s,k+1 (z k+1 ) − 3σs,k+1 (z k+1 ). The allotted maintenance costs, m s,k+1 (z k+1 ), Ds,k+1 (z k+1 ), and m s,k+1 (z k+1 ) − 3σs,k+1 (z k+1 ) for each non-dominated solution are listed in Table 6.4. The expectation and confidence interval for each solution are also depicted in Fig. 6.4b. Consequently, the best-compromised selective maintenance strategies under the three criteria, as highlighted in Table 6.4, are the fourteen, first, and eleven solutions, respectively. The criterion for choosing a best-compromised strategy is generally applicationdependent and may vary with different scenarios even for a same engineering system. For instance, a power supply system in a critical military base is extremely risk-averse,

6.7 Illustrative Examples

115

Table 6.4 Five non-dominated solutions (unit of cost: $1,000) ID Maintenance Criterion 1 Criterion 2 levels m s,k+1 (z k+1 ) Ds,k+1 (z k+1 ) 1 2 3 4 5 6 7 8 9 10 11 12 13 14

(5, 1, 0, 1, 10) (1, 4, 0, 1, 10) (3, 3, 0, 1, 10) (1, 4, 1, 1, 10) (1, 1, 0, 10, 6) (1, 1, 2, 10, 5) (1, 4, 1, 10, 1) (1, 3, 2, 10, 1) (4, 2, 1, 10, 1) (1, 3, 3, 10, 1) (2, 3, 2, 10, 1) (1, 3, 1, 10, 3) (1, 3, 2, 10, 2) (1, 2, 4, 10, 2)

0.8082 0.8101 0.8139 0.8306 0.8427 0.8532 0.8552 0.8560 0.8589 0.8592 0.8603 0.8622 0.8631 0.8633

1.6766 × 10−4 1.6998 × 10−4 1.9352 × 10−4 2.1772 × 10−4 2.4374 × 10−4 2.5221 × 10−4 2.5744 × 10−4 2.9028 × 10−4 2.9278 × 10−4 2.9841 × 10−4 2.9972 × 10−4 3.3356 × 10−4 3.3933 × 10−4 3.6642 × 10−4

Criterion 3 m s,k+1 (z k+1 ) ± 3σs,k+1 (z k+1 ) [0.7693, 0.8470] [0.7710, 0.8493] [0.7722, 0.8557] [0.7863, 0.8749] [0.7959, 0.8896] [0.8055, 0.9008] [0.8070, 0.9033] [0.8048, 0.9071] [0.8076, 0.9102] [0.8074, 0.9111] [0.8084, 0.9123] [0.8074, 0.9170] [0.8078, 0.9184] [0.8058, 0.9207]

decision-makers would choose a relatively robust strategy as an unexpected blackout may incur a huge risk to the base. In this regard, the strategy with a maximal lower bound under Criterion 3 may be preferable. However, it is risk-neutral to some extent for the power supply system in a public school, and therefore the strategy that can achieve a maximal expected value may be preferable. Based on the above analysis, three questions arise naturally: • Is the best strategy obtained by the proposed method credible if the observations are imperfect? • Is the best selective maintenance strategy considering the observation errors superior to the optimal strategy without considering the observation errors (as mentioned in Sect. 6.2)? • How sensitive is the best selective maintenance strategy concerning the observation errors? To answer these questions, we carry out a simulation test as shown in Fig. 6.5. In accordance with Fig. 6.1, in Scenario 1, the true values of the state and effective age of each component are known, and then the optimal selective maintenance strategy can be identified. For each sample i, the probability of the system successfully completing the (k + 1)th mission, denoted by R(1, i), can then be calculated. It is noteworthy the maintenance results in Scenario 1 are the true optimal selective maintenance strategies and will serve as benchmarks for comparisons. In Scenario 2, we can find the optimal selective maintenance strategy without considering the observation

116

6 Robust Selective Maintenance under Imperfect Observations

Fig. 6.5 Flowchat of the simulation test

errors. Analogously, the probability of the system successfully completing the (k + 1)th mission, denoted by R(2, i), can then be obtained. In Scenario 3, we can obtain the Pareto-optimal set with the proposed robust selective maintenance method in which the observation errors are taken into account, and then choose the best selective maintenance strategy based on our preferences, say Criterion 3 in this section. In the simulation, we can also pick out the optimal selective maintenance strategy from the Pareto-optimal set. The probabilities of the system successfully completing the (k + 1)th mission under the best and optimal solutions, denoted by R1 (3, i) and R2 (3, i), respectively, can then be obtained. The deviation between the maintenance result under the optimal solution in Scenario 1 and the maintenance results under the optimal solution in Scenarios 2 and 3 are e(2, i) = R(2, i) − R(1, i) and e(3, i) = R2 (3, i) − R(1, i), respectively. Corresponding to the first question, the percentage of R1 (3, i) in the confidence interval of m s,k+1 (z k+1 ) ± 3σs,k+1 (z k+1 ) can be computed to validate the credibility of the proposed robust selective maintenance method. Corresponding to the second question, the mean values of e(2, i) and e(3, i) are utilized to compare the accuracy the selective maintenance methods in Scenarios 2 and 3. The simulation results are tabulated in the row with “No.” equal to zero in Table 6.5. We can see that the

6.7 Illustrative Examples

117

Table 6.5 Five non-dominated solutions (unit of cost: $1000) No. (δiI , δiII , σi ) Percentage Deviation Scenario 2 0 1 2 3 4 5 6 7 8 9

(0.2, 0.2, 2) (0.1, 0.1, 1) (0.1, 0.2, 3) (0.1, 0.3, 2) (0.2, 0.1, 2) (0.2, 0.2, 1) (0.2, 0.3, 3) (0.3, 0.1, 3) (0.3, 0.2, 2) (0.3, 0.3, 1)

99.22 98.64 99.07 99.14 99.41 98.77 99.24 99.32 99.20 98.74

−0.0178 −0.0088 −0.0180 −0.0244 −0.0097 −0.0171 −0.0256 −0.0118 −0.0184 −0.0268

Scenario 3 −0.0060 −0.0048 −0.0061 −0.0067 −0.0052 −0.0061 −0.0061 −0.0059 −0.0063 −0.0056

Fig. 6.6 Main effect of each observation error

percentage of R1 (3, i) in the confidence interval of m s,k+1 (z k+1 ) ± 3σs,k+1 (z k+1 ) is 99.22, which indicates the actual maintenance result is consistent with the theoretical one by the proposed method. The mean values of e(2, i) and e(3, i) are −0.0178 and −0.0060, respectively, which suggest the selective maintenance strategy in Scenario 3 by the proposed method is superior to the one in Scenario 2 without considering the observation errors. Furthermore, a one-third fraction of the 33 experimental design is implemented to examine the sensitivity of the observation errors. The maintenance results with respect to different settings of the observation errors are listed in Table 6.5. The positive responses to the first and second questions in the experiments number from 1 to 9 are also presented. Additional, the main effects of each observation error in Scenarios 2 and 3 are depicted in Fig. 6.6. We can observe that there is an apparent tendency that the maintenance results can be improved with decreased observation errors, and δiII has the greatest impact on the maintenance results.

118

6 Robust Selective Maintenance under Imperfect Observations

11 6

1 4

12

10

13

7

2 5 3

9

8 14

Feeder #1 (Subsystem 1)

Conveyor #1 Stacker-reclaimer Feeder #2 (Subsystem 2) (Subsystem 3) (Subsystem 4)

Conveyor #2 (Subsystem 5)

Fig. 6.7 Configuration of studied coal transportation system

6.7.2 A Coal Transportation System A coal transportation system is exemplified to further examine the effectiveness of the proposed robust selective maintenance method for large-scale systems. The system, which is composed of five subsystems, is used to supply a boiler in a power station as shown in Fig. 6.7. Feeder 1 transfers the coal from the bin to conveyor 1. Conveyor 1 transports the coal to the stacker-reclaimer that lifts the coal up to the burner level. Feeder 2 loads conveyor 2 that supplies the burner feeding system of the boiler. The system consists of 14 s-independent components, and the failure time of each component follows the Weibull distribution. The parameters of each component are tabulated in Table 6.6. The time durations of both the kth and (k + 1)th missions are assumed to take z k = z k+1 = 10 days. Ten maintenance levels, i.e., NL = 10, can be chosen for each component. The observation errors for each component are assumed to be identical and take δiI = δiII = 0.2 and σi = 2 (i ∈ {1, . . . , 14}). Because of the complexity of this relatively large-scale system, the MOPSO algorithm is executed to identify the Pareto front. Given the maintenance budget C0 = 200 units, the Pareto front of all non-dominated solutions is presented in Fig. 6.8. Meanwhile, the Pareto front of the case “Without Observation” is also provided in Fig. 6.8 for comparison. In this case, the states and operating times of all the components are not observed, hence the prior distributions of states and the conditional PDFs of effective ages are used to evaluate m s,k+1 (z k+1 ) and Ds,k+1 (z k+1 ). We can see that all the solutions without observations are dominated by, at least, one of the solutions with imperfect observations, which means that the maintenance results with imperfect observations are superior to the maintenance results without observations. The runtime for evaluating the expectation and variance of the probability of the system successfully completing the next mission is approximately 0.0033 seconds on a PC with an Intel Core i7 2.60 GHz CPU and

6.8 Closure

119

Table 6.6 Parameters of components (unit of cost: $1,000, unit of time: day) w,o o ID θi βi ci0 cimin cicr Yi,k Ui,k Ti,k 1 2 3 4 5 6 7 8 9 10 11 12 13 14

25 32 28 34 28 34 26 28 32 35 22 20 24 25

1.5 2.2 1.6 2.6 1.8 2.4 2.5 2.0 1.8 2.1 2.8 1.5 2.4 2.2

1.8 2.5 2.5 3.0 2.2 1.8 2.4 2.6 2.0 2.2 2.6 2.6 2.2 1.8

2.0 3.0 3.5 3.6 3.0 2.0 3.2 3.5 3.0 2.8 3.0 3.2 2.5 2.2

36 42 48 51 44 32 46 49 40 45 45 48 40 35

1 1 0 1 1 0 0 1 1 1 0 0 1 1

3.0 4.0 5.0 3.0 3.5 4.0 5.0 2.6 3.8 4.2 5.6 5.0 2.0 3.2

10.0 10.0 7.0 10.0 10.0 5.5 5.5 10.0 10.0 10.0 4.2 5.1 10.0 10.0

o Vi,k

13.0 14.0 12.0 13.0 13.5 9.5 10.5 12.6 13.8 14.2 9.8 10.1 12.0 13.2

8.0 GB RAM. The total runtime for identifying a Pareto front by using the MOPSO is approximately 7 minutes when the maximum iteration number is set to be 200. Intuitively, the maintenance results could be improved with the increase of the maintenance budget. The Pareto fronts versus different maintenance budgets are also plotted in Fig. 6.8 for the two cases “Without Observation” and “With Imperfect Observation.” We can see that, under each setting of the maintenance budget, the maintenance results with imperfect observations always dominate the maintenance results without observations. Moreover, by increasing the maintenance budget, both the performance and robustness of the non-dominated solutions can be improved in the two cases “Without Observation” and “With Imperfect Observation.” Meanwhile, with the increase in the maintenance budget, the difference between the two cases “Without Observation” and “With Imperfect Observation” declines. If all the components can be replaced by new ones, the maintenance results of the two cases “Without Observation” and “With Imperfect Observation” will be the same, and the variances of the two cases are zero.

6.8 Closure In this chapter, by considering the uncertainties raised from imperfect observations, a robust selective maintenance optimization model was developed. Due to the imperfection of observations, uncertainties were introduced into the state and effective age of each component. The uncertainties were incorporated and characterized by the posterior distributions of the state and effective age of each component based on the

120

6 Robust Selective Maintenance under Imperfect Observations

Fig. 6.8 Pareto fronts versus different maintenance budgets

Bayes rule. The expectation and variance of the probability of a system successfully completing the subsequent mission were computed to quantify the uncertainties propagated from imperfect observations. A multi-objective selective optimization model was then constructed with the aims of maximizing the expectation and simultaneously minimizing the variance to guarantee the robustness of a selective maintenance strategy under uncertainties. A five-component system and a coal transportation system were analyzed to validate the advantage of the proposed robust selective maintenance optimization model. A Pareto-optimal set consisting of non-dominated solutions was identified, and decision-makers can choose a best-compromised selective maintenance strategy based on their requirements or preferences. Comparative experimental results showed the superiority of the proposed robust selective maintenance method.

References 1. Coello CAC, Pulido GT, Lechuga MS (2004) Handling multiple objectives with particle swarm optimization. IEEE Trans Evol Comput 8(3):256–279 2. Coit DW, Jin TD, Wattanapongsakorn N (2004) System optimization with component reliability estimation uncertainty: a multi-criteria approach. IEEE Trans Reliab 53(3):369–380 3. Ghasemi A, Yacout S, Ouali MS (2010) Evaluating the reliability function and the mean residual life for equipment with unobservable states. IEEE Trans Reliab 59(1):45–54 4. Ghasemi A, Yacout S, Ouali MS (2010) Parameter estimation methods for condition-based maintenance with indirect observations. IEEE Trans Reliab 59(2):426–439 5. Jiang T, Liu Y (2020) Robust selective maintenance strategy under imperfect observations: a multi-objective perspective. IISE Trans 52(7):751–768 6. Kijima M (1989) Some results for repairable systems with general repair. J Appl Probab 26(1):89–102 7. Makis V, Jiang X (2003) Optimal replacement under partial observations. Math Oper Res 28(2):382–394 8. Pandey M, Zuo MJ, Moghaddass R, Tiwari MK (2013) Selective maintenance for binary systems under imperfect repair. Reliab Eng Syst Saf 113:42–51

References

121

9. Stewart J (2014) Calculus: early transcendentals, 8th edn. Cengage Learning, Boston, USA 10. Wang W, Christer AH (2000) Towards a general condition based maintenance model for a stochastic dynamic system. J Oper Res Soc 51(2):145–155 11. Wu SM, Coolen FPA, Liu B (2017) Optimization of maintenance policy under parameter uncertainty using portfolio theory. IISE Trans 49(7):711–721

Chapter 7

Selective Maintenance and Inspection Optimization for Partially Observable Systems

7.1 Introduction Selective maintenance is inherently a sort of condition-based maintenance for multicomponent systems under limited resources. The decision-making of selective maintenance relies heavily on the components’ states after the last mission. Most of the reported works on selective maintenance assumed that all component health conditions after the previous mission are precisely known before the maintenance optimization. However, inspection activities like vibration analysis and ultrasonic analysis are necessary to be conducted to identify the component health conditions [4]. On the one hand, the component conditions can be better revealed by inspection activities, leading to a more effective maintenance strategy than the case without inspection. On the other hand, the inspection activities will consume resources, resulting in fewer remaining time resources for maintenance activities than the case without inspection. Therefore, to resolve the selective maintenance and inspection problem, it necessitates a joint maintenance and inspection optimization to trade off the effectiveness and resource consumption of the two activities. The imperfect maintenance which restores a failed/derated component to somewhere between the “as good as new” and “as bad as old” conditions is incorporated in this chapter [14]. Due to the variation of repairperson skills and the differences among maintenance materials, the outcomes of imperfect maintenance are often stochastic [10]. Moreover, owing to the measurement error and poor diagnostic algorithms, the data collected from inspections often contain noise and thus cannot fully reveal the true component states [2]. The imperfect maintenance and inspection actions exert a significant impact on maintenance strategy and should be taken into account in the maintenance optimization. Most of the research on maintenance and inspection optimization either optimized the maintenance policy with the pre-determined inspection strategy or vice versa. Shi et al. [12] studied an optimal maintenance policy to reduce the system’s long-run average operational cost, where the system was periodically and perfectly inspected. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_7

123

124

7 Selective Maintenance and Inspection Optimization …

Liu et al. [6] proposed an optimal maintenance and inspection policy to minimize the average cost, in which each inspection action corresponds to a determined maintenance action. Some studies focused on the joint maintenance and inspection optimization. Corotis et al. [1] investigated a complicated maintenance and inspection policy considering multiple maintenance and inspection techniques. Based on this work, Papakonstantinou and Shinozuka [8] and Papakonstantinou et al. [9] leveraged several point-based algorithms to resolve the optimal value function more efficiently. Nevertheless, the aforementioned joint maintenance and inspection optimization were designed for single-component systems, and cannot be straightforwardly extended to the selective maintenance which is formulated for multi-component systems. This chapter provided a joint maintenance and inspection optimization model to resolve the selective maintenance and inspection problem. A finite-horizon mixed observability Markov decision process (MOMDP) was formulated as the remaining time resource was fully observable and the component states were partially observable. In the MOMDP model, to maximize the probability of the system successfully completing the next mission, the maintenance and inspection actions can be dynamically adjusted based on the remaining time resource and the state distribution of all components in the system. The remainder of this chapter was rolled out as follows. In Sect. 7.2, the selective maintenance and inspection problem, together with some basic assumptions, was introduced. In Sect. 7.3, the probability of a system successfully completing the next mission was evaluated and the optimization problem was formulated as a finitehorizon MOMDP. Two illustrative examples were given in Sect. 7.4. A brief closure was given in Sect. 7.5.

7.2 Problem Statements and Model Assumptions 7.2.1 Problem Statement Figure 7.1 illustrates the selective maintenance and inspection problem for a threecomponent system. The true states of the three components are unknown after the mission. If the first action is to maintain component 3 without any inspections (denoted by Scenario 1), the maintenance action would be useless if component 3 is perfectly functioning. Alternatively, if the first action is to inspect component 3, the subsequent action can be dynamically adjusted based on the inspection results. For example, if component 3 is perfectly functioning, the subsequent action is to maintain component 1 (denoted by Scenarios 2). However, if component 3 is failed, the subsequent action is to maintain component 3 (denoted by Scenarios 3). Therefore, the component conditions can be better revealed by inspection and thus lead to a more effective maintenance strategy (see Scenarios 2 and 3 compared to Scenario 1). Without loss of generality, the basic assumptions of the joint selective maintenance and inspection problem are defined as follows:

7.2 Problem Statements and Model Assumptions

125 Scenario 1

Maintaining component 3

Maintaining component 1

Inspecting component 3

kth mission

Maintaining component 3

kth break Component 1

Scenario 2

Component 2

Scenario 3 (k+1)th mission

Component 3

Fig. 7.1 Selective maintenance and inspection for a system that completed a mission

• A system consists of M s-independent components. Each component can operate in n l possible states. The performance capacity of component l in state i is denoted by gl,i (i ∈ l = {1, . . . , n l }). The performance capacity of component l at any time t is denoted by G l (t) ∈ {gl,1 , . . . , gl,nl }, where gl,1 and gl,nl are the minimum and maximum performance capacity of component l, respectively. • The degradation of components can be characterized by homogeneous discretetime Markov models. The state transition probability matrix is introduced to quantify the relation between the component states before and after a mission. The state transition probability matrix of component l after executing the (k + 1)th mission, denoted by Pl (k + 1), is as follows ⎡ ⎤ pl,(1,1) (k + 1) 0 ··· 0 ⎢ pl,(2,1) (k + 1) pl,(2,2) (k + 1) · · · ⎥ 0 ⎢ ⎥ (7.1) Pl (k + 1) = ⎢ ⎥, .. .. . .. .. ⎣ ⎦ . . . pl,(nl ,1) (k + 1) pl,(nl ,2) (k + 1) · · · pl,(nl ,nl ) (k + 1)

where pl,(i, j) (k + 1) is the probability of component l degrading from state i to state j at the end of the (k + 1)th mission. During the mission phase, the component state cannot transit to a better state. Hence, one has pl,(i, j) (k + 1) = 0 for i < j (i, j ∈ l ). • At any time instant t, the performance capacity of a system, denoted by G s (t), can be generated from the performance capacities of all the components and the system structure function φ(·), that is, G s (t) = φ(G 1 (t), . . . , G M (t)). The set of the system performance capacities is denoted by gs = {g1 , . . . , g N }, where N is the number of possible performance capacities of the system, and g1 and g N are the minimum and maximum performance capacities of the system, respectively. • After the kth mission, maintenance and inspection actions can be sequentially performed during the kth break. The mth action during the kth break is denoted by a(k, m). If a(k, m) is an inspection action, the observation value from a(k, m) is denoted by O(k, m). The remaining time resource of the kth break before conducting the mth action is T (k, m). The number of maintenance and inspection actions performed during the kth break is denoted by M(k).

126

7 Selective Maintenance and Inspection Optimization …

• The state of component l before the mth action during the kth break is denoted by Sl (k, m). Let bl (k, m) = (bl,1 (k, m), . . . , bl,nl (k, m)) represent the state probability distribution of component l, namely the belief state in the partially observable multi-state systems [11], where bl,i (k, m) = Pr{Sl (k, m) = i} (i ∈ l ). The set of belief states of all the components before the mth action during the kth mission is represented as B(k, m) = (b1 (k, m), . . . , b M (k, m)). • The kth mission can be successfully completed if the system performance capacity at the end of the kth mission is greater than a pre-specified demand Wk . The probability of the system successfully completing the kth mission is denoted by R(k). In this chapter, the time duration of the break is viewed as the resource constraint and the time duration of the kth break is denoted by L k .

7.2.2 Imperfect Maintenance Model The imperfect maintenance can recover a component condition to somewhere between the “as good as new” and “as bad as old.” In general, component l can rep rep be maintained with Nl optional maintenance levels. Let al (u) denote the mainterep nance action recovering component l with maintenance level u (u ∈ {1, . . . , Nl }). rep rep The maintenance time of al (u) is denoted by tl (u). The state transition probability matrix is introduced to characterize the stochastic results of the maintenance action. rep The state transition probability matrix of maintenance action al (u) is as follows ⎤ ⎡ rep rep rep pl,(1,1) (u) pl,(1,2) (u) · · · pl,(1,nl ) (u) rep rep ⎢ 0 pl,(2,2) (u) · · · pl,(2,nl ) (u) ⎥ ⎥ ⎢ rep (7.2) Pl (u) = ⎢ ⎥, .. .. .. . . ⎦ ⎣ . . . . rep 0 0 · · · pl,(nl ,nl ) (u) rep

where pl,(i, j) (u) denotes the probability of restoring component l from state i to  rep rep j when maintenance action al (u) is performed. One has j∈l pl,(i, j) (u) = 1 (i ∈ l ). rep Without loss of generality, if a(k, m) = al (u), the belief state of component l rep after the mth action is denoted by bl (k, m + 1) = (bl,1 (k, m + 1|al (u)), . . . , bl,nl rep rep (k, m + 1|al (u))), where bl, j (k, m + 1|al (u)) ( j ∈ l ) can be evaluated by rep

bl, j (k, m + 1|a(k, m) = al (u)) rep

= Pr {Sl (k, m + 1) = j|bl (k, m), a(k, m) = al (u)} rep bl,i (k, m) · pl,(i, j) (u). =

(7.3)

i∈l rep

The transition probability of remaining time resource after a(k, m) = al (u), rep denoted by Pr{T (k, m + 1)|T (k, m), a(k, m) = al (u)}, can be calculated as

7.2 Problem Statements and Model Assumptions

127 rep

Pr{T (k, m + 1)|T (k, m), a(k, m) = al (u)}

rep 1 T (k, m + 1) = T (k, m) − tl (u) = . rep 0 T (k, m + 1) = T (k, m) − tl (u)

(7.4)

rep

After conducting the action a(k, m) = al (u), the set of the belief states of all the components is B(k, m + 1) = (b1 (k, m + 1), . . . , bl (k, m + 1), . . . , b M (k, m + 1)). As only the belief state of component l is changed, B(k, m + 1) can be calculated by B(k, m) and bl (k, m + 1) as B(k, m + 1) = (b1 (k, m), . . . , bl (k, m + 1), . . . , b M (k, m)).

(7.5)

7.2.3 Imperfect Inspection Model Unlike the perfect inspection, imperfect inspection only provides partial information about the true state of a component. In general, component l can be inspected with Nlins optional inspection levels. Let alins (v) represent the action of inspecting component l with level v (v ∈ {1, . . . , Nlins }). The inspection time of alins (v) is denoted by tlins (v). The observation probability matrix is introduced to quantify the imperfection of the observed component state as reported in Ghasemi et al. [2]. The observation probability matrix of component l when alins (v) is performed, denoted by Plins (v), is written as follows ⎡ ins ⎤ ins ins pl,(1,1) (v) pl,(1,2) (v) · · · pl,(1,n (v) l) ins ins ins ⎢ pl,(2,1) (v) pl,(2,2) (v) · · · pl,(2,n (v) ⎥ l) ⎢ ⎥ (7.6) Plins (v) = ⎢ ⎥, .. .. .. .. ⎣ ⎦ . . . . ins ins ins (v) pl,(n (v) · · · pl,(n (v) pl,(n l ,1) l ,2) l ,n l ) ins where pl,(i,o) (v) represents the probability that the observed state from inspection  ins action alins (v) is o when component l is in state i. One has o∈l pl,(i,o) (v) = 1 (i ∈ l ). Without loss of generality, if a(k, m) = alins (v), the probability that the component l is observed in state o from inspection action alins (v) denoted by Pr{O(k, m) = o|bl (k, m), a(k, m) = alins (v)}, can be calculated by

Pr{O(k, m) = o|bl (k, m), a(k, m) = alins (v)} bl,i (k, m) · Pr{O(k, m) = o|Sl (k, m) = i, a(k, m) = alins (v)} = i∈l

=



i∈l

bl,i (k, m) ·

ins pl,v (i, o).

(7.7)

128

7 Selective Maintenance and Inspection Optimization …

The belief state of component l after conducting the inspection action a(k, m) = alins (v) and getting the inspected state o is denoted by bl (k, m + 1) = (bl,1 (k, m + 1|a(k, m) = alins (v), O(k, m) = o), . . . , bl,nl (k, m + 1|a(k, m) = alins (v), O(k, m) = o)), where bl, j (k, m + 1|a(k, m) = alins (v), O(k, m) = o) ( j ∈ l ) can be updated via the Bayes formula as following bl, j (k, m + 1|a(k, m) = alins (v), O(k, m) = o) = Pr{Sl (k, m + 1) = j|bl (k, m), a(k, m) = alins (v), O(k, m) = o} =

Pr{Sl (k, m + 1) = j, bl (k, m), a(k, m) = alins (v), O(k, m) = o} Pr{bl (k, m), a(k, m) = alins (v), O(k, m) = o}

Pr{O(k, m) = o|Sl (k, m + 1) = j, a(k, m) = alins (v)} · bl, j (k, m) = Pr{O(k, m) = o|bl (k, m), a(k, m) = alins (v)} =

ins pl,v ( j, o) · bl, j (k, m)

Pr{O(k, m) = o|bl (k, m), a(k, m) = alins (v)}

(7.8)

.

The transition probability of the remaining time resources after a(k, m) = alins (v), denoted by Pr{T (k, m + 1)|T (k, m), a(k, m) = alins (v)}, is Pr{T (k, m + 1)|T (k, m), a(k, m) = alins (v)}

1 T (k, m + 1) = T (k, m) − tlins (v) = , 0 T (k, m + 1) = T (k, m) − tlins (v)

(7.9)

and the set of belief states of all components (i.e., B(k, m + 1)) can be also calculated by Eq. (7.5).

7.3 Joint Selective Maintenance and Inspection Optimization 7.3.1 Probability of a System Successfully Completing a Mission After all the selected maintenance and inspection actions are performed in the kth break, the system would execute the (k + 1)th mission. The belief state of component l at the beginning of the (k + 1)th mission is the same as that at the end of the kth break (i.e., bl (k, M(k) + 1)). The belief state of component l at the end of the (k + 1)th mission is bl (k + 1, 1) = (bl,1 (k + 1, 1), . . . , bl,nl (k + 1, 1)). Based on the Markov property of component degradation, bl, j (k + 1, 1) ( j ∈ l ) can be computed by

7.3 Joint Selective Maintenance and Inspection Optimization

129

bl, j (k + 1, 1) = Pr{Sl (k + 1, 1) = j|bl (k, M(k) + 1)} = bl,i (k, M(k) + 1) · Pr{Sl (k + 1, 1) = j|Sl (k, M(k) + 1) = i} i∈l

=



(7.10)

bl,i (k, M(k) + 1) · pl,(i, j) (k + 1).

i∈l

Based on the set of the belief states of all components at the end of the (k + 1)th mission and the system configuration, the probability distribution of the system can be calculated by UGF [5]. The UGF of component l at the end of the (k + 1)th mission, denoted by u l,k+1 , is written as u l,k+1 =



bl,i (k + 1, 1) · z gl,i .

(7.11)

i∈l

The UGF of the system at the end of the (k + 1)th mission, denoted by Uk+1 , can be calculated by Uk+1 =

M

bl,i (k + 1, 1) · z gl,i

l=1 i∈l



=

S1 (k+1,1)∈1

=

N

...

M

S M (k+1,1)∈ M

bl,Sl (k+1,1) · z φ(g1,S1 (k+1,1) ,...,g M,SM (k+1,1) )

(7.12)

l=1

bs,i (k + 1) · z gi ,

i=1

where bs,i (k + 1) = Pr{G s (k + 1, 1) = gi }. The probability of the system successfully completing the (k + 1)th mission, denoted by R(k + 1), can be written as R(k + 1) =

N

bs,i (k + 1) · 1{gi ≥ Wk+1 },

(7.13)

i=1

where 1{·} is an indicator function that is equal to one if gi is not less than Wk+1 and zero otherwise.

7.3.2 Mixed Observability Markov Decision Process For notation convenience, we define

a(k, m) a(k, m) is a maintenance action y(k, m) = . (a(k, m), O(k, m)) otherwise

130

7 Selective Maintenance and Inspection Optimization …

Due to partial observations, some sequential decision problems may not be formulated as Markov decision processes. In this chapter, the Markov property of the sequential decision problem is demonstrated as follows. On the one hand, the remaining time resource T (k, m + 1) can be calculated from T (k, m) and resource consumption of the action in y(k, m). On the other hand, the belief state of the inspected and maintained component can be updated based on the belief state of itself and the selected action. Thus, one has Pr {T (k, m + 1)|B(k, 1), T (k, 1), y(k, 1), . . . , B(k, m), T (k, m), y(k, m)} (7.14) = Pr {T (k, m + 1)|T (k, m), y(k, m)} , Pr {B(k, m + 1)|B(k, 1), T (k, 1), y(k, 1), . . . , B(k, m), T (k, m), y(k, m)} (7.15) = Pr {B(k, m + 1)|B(k, m), y(k, m)} . The Markov property is, therefore, held in the studied problem. Furthermore, as the true component states are partially observable and the remaining time resource is fully observable, the selective maintenance and inspection optimization problem can be modeled as a finite-horizon mixed observability Markov decision process (MOMDP) [9]. The MOMDP can be defined as an 8-tuple T, S, A, PT , PS , , Q, R. In the tuple, T and S, respectively indicate fully and partially observable state variables, that is, the remaining time resource and the true states of all components. A is the action set containing all possible actions as rep

rep

rep

rep

rep

rep

A = {a1 (1), . . . , a1 (N1 ), . . . , a M (1), . . . , a M (N M ), ins ins a1ins (1), . . . , a1ins (N1ins ), . . . , a ins M (1), . . . , a M (N M )}.

(7.16)

PT and PS represent the transition probabilities of the remaining time resource and the set of the belief states of all components, respectively. PT (T (k, m), y(k, m), T (k, m + 1)) can be respectively evaluated by Eqs. (7.4) and (7.9) for maintenance and inspection actions. While PS (B(k, m), y(k, m), B(k, m + 1)) can be calculated by Eqs. (7.3) and (7.8) for maintenance and inspection actions, respectively.  is the set of possible observations composed of all component states. Let Q represent the observation function and Q(B(k, m), y(k, m)) is the probability that the inspected component is in state o from a(k, m) if the set of belief states of all components is B(k, m). Q(B(k, m), y(k, m)) can be calculated by Eq. (7.7). R is the reward function modelling the effectiveness of the selected action. To resolve MOMDP, the state space, the action space, and the reward should be specified first. The state space contains all the possible combinations of the set of the belief states of all components and the remaining time resource. Given the present remaining time resource T (k, m), the action space A(k, m) is written as A(k, m) = Arep (k, m) ∪ Ains (k, m),

(7.17)

7.3 Joint Selective Maintenance and Inspection Optimization rep

131

rep

where Arep (k, m) = {al (u)|tl (u) ≤ T (k, m)} and Ains (k, m) = {alins (v)| tlins (v) ≤ T (k, m)}. To ensure the cumulative rewards being equal to the probability of the system successfully completing the next mission, the reward of the executing an action in A(k, m), denoted by r (k, m), is defined as

0 A(k, m) = ∅ r (k, m) = . (7.18) R(k + 1) A(k, m) = ∅ The value function of the MOMDP, denoted by V ∗ (k, m), represents the maximum expectation of the cumulative rewards. Combined with the definition of the reward function in Eq. (7.18), the value function V ∗ (k, m) equals to the maximum probability of the system successfully completing the next mission. Hence, one has V ∗ (k, 1) = max R(k + 1).

(7.19)

Let a denote a maintenance/inspection action and a ∈ A(k, m). If a∈Arep (k, m), V (k, m + 1|a) represent the value function after a. If a∈ Ains (k, m), let po|B,T,a (k, m) denote the probability that the inspected component is in state o from a, and V ∗ (k, m + 1|a, o) represent the value function after getting the observation o from the inspection action a. To resolve the selective maintenance and inspection optimization problem and maximize the probability of the system successfully completing the next mission, the Bellman equation of the MOMDP is formulated as follows ∗

V ∗ (k, m)

⎧ ⎪ r (k, m) A(k, m) = ∅ ⎪ ⎨ ∗ (k, m + 1|a) a ∈ Arep (k, m) r (k, m) + V = max  a∈A(k,m) ⎪ ∗ ⎪ po|B,T,a (k, m) · V (k, m + 1|a, o) a ∈ Ains (k, m) ⎩r (k, m) + o∈

⎧ ⎪ (k + 1) A(k, m) = ∅ ⎪ ⎪ ⎪ rep ⎨ max V ∗ (k, m + 1|a rep (u)) rep a l l (u) ∈ A (k, m) = max alrep (u) , ⎪  ⎪ ∗ (k, m + 1|a ins (v), o) a ins (v) ∈ Ains (k, m) ⎪ p (k, m) · V max ⎪ ins o∈l o|B,T,al (v) ⎩ ins l l al (v)

(7.20) where po|B,T,alins (v) (k, m) = Pr{O(k, m) = o|bl (k, m), a(k, m) = alins (v)} as the remaining time resource T (k, m) does not affect the inspected state and the inspection result is only based on inspected component l and the inspection action alins (v). Given the optimal value function, the optimal selective maintenance and inspection policy, mapping from the set of belief states of all components and the remaining time resources to the selected action, can be calculated based on the Bellman equation as following

132

7 Selective Maintenance and Inspection Optimization …

π ∗ (k, m)

⎧ ⎪ R(k + 1) A(k, m) = ∅ ⎪ ⎨ ∗ a ∈ Arep (k, m) . = arg max V(k, m + 1|a) ⎪ ∗ a∈A(k,m) ⎪ po|B,T,a (k, m) · V (k, m + 1|a, o) a ∈ Ains (k, m) ⎩

(7.21)

o∈l

7.3.3 Dynamic Programming Algorithm The dynamic programming (DP) algorithm can be implemented to resolve the Bellman equation and obtain the optimal value function accurately. It uses the structure of the Bellman equation to search for the optimal value function and policy function. As we can see from Eq. (7.20), the value function before the mth action V ∗ (k, m) can be calculated by the value function before the (m + 1)th action V ∗ (k, m + 1). However, the value of V ∗ (k, m + 1) is also unknown and needed to be optimized. By replacing m by m + 1 in Eq. (7.20), V ∗ (k, m + 1) can be calculated by the value function before the (m + 2)th action V ∗ (k, m + 2). By repeating the above process until all inspection and maintenance action are performed (i.e., m = M(k) + 1 and A(k, m) = ∅) the value function V ∗ (k, M(k) + 1) is equal to R(k + 1) by Eq. (7.20). Thereafter, V ∗ (k, m) can be obtained through repeating the above process reversely. The pseudo-code of the DP algorithm is given in Algorithm 7.1. Algorithm 7.1 Pseudo-code of the DP algorithm Inputs: B(k, 1): The initial set of belief states of all components; T (k, 1): The time duration of the kth break. 1: Initialize the memory E to store the value function. 2: For m ← 1 to M(k) + 1 do 3: Calculate the action space A(k, m) by Eq. (7.17); 4: For a(k, m) in A(k, m) do 5: Update the set of the belief states of all the components and the remaining time resource after completing a(k, m): B(k, m + 1) and T (k, m + 1); 6: Store (B(k, m + 1), T (k, m + 1)) in memory E; 7: End For 8: End For 9: For m ← M(k) + 1 to 1 do 10: For (B(k, m), T (k, m)) in E do 11: Calculate the value function V ∗ (k, m) by Eq. (7.20); 12: End For 13: End For Output: Optimal value function V ∗ (k, m).

7.3 Joint Selective Maintenance and Inspection Optimization

133

7.3.4 Deep Reinforcement Learning Algorithm The DP algorithm can obtain the optimal value function accurately. However, as the DP algorithm needs to enumerate all possible subsequent states, it is practically intractable with the increase of action space and state space. Alternatively, the deep reinforcement learning (DRL) algorithm offers a promising framework to resolve the high-dimensional state space and action space problems. DRL has been widely utilized in operation and maintenance management recently [3, 7, 13]. Particularly, the deep Q-learning (DQN) algorithm leverages an artificial neural network (ANN) to approximate the action-value function, which is suitable for resolving continuous state space and finite action space problems. In the spirit of the DQN algorithm, a DVN algorithm is customized to approximate the value function V ∗ (k, m). The pipeline of the customized DVN algorithm is shown in Fig. 7.2. As shown in Fig. 7.2, In the customized DVN algorithm, the agent is represented by a value network V (·), whose input is the set of the belief states of all components B(k, m) and the remaining time resource T (k, m), and output is the approximated value function V (k, m). The detailed procedures are summarized as follows: Step 1: Given the set of the belief states of all components and the remaining time resource after the (m − 1)th action that is, B(k, m) and T (k, m). The agent follows the policy function in Eq. (7.21) to select a maintenance or inspection action a(k, m). Step 2: The reward of performing a(k, m) given B(k, m) and T (k, m) can be evaluated by Eq. (7.18) and is returned to the agent.

Fig. 7.2 Pipeline of the customized DVN algorithm

134

7 Selective Maintenance and Inspection Optimization …

Step 3: The set of the belief states of all components and the remaining time resource after executing action a(k, m), that is, B(k, m + 1) and T (k, m + 1), will be randomly simulated by the environment. After the hth loop of Steps 1–3, the experience of the agent and the environment, denoted by eh (k, m) = (Bh (k, m), Th (k, m), ah (k, m), rh (k, m), Bh (k, m + 1), Th (k, m + 1)), can be recorded for offline training. For eh (k, m), the evaluated value is the output of the value network, denoted by Vh (k, m). The target value yh can be deduced by the Bellman equation as

rh (k, m) A(k, m) = ∅ yh = rh (k, m) + Vh (k, m + 1) A(k, m) = ∅

(7.22) rh (k, m) A(k, m) = ∅ = . Vh (k, m + 1) A(k, m) = ∅ The loss function L h (θV ) for eh (k, m) is defined as L h (θV ) = (yh − Vh (k, m))2 .

(7.23)

The correlation between the evaluated value and the target value influences the stability of the training process. To decouple the correlation, the target network and experience replay techniques are implemented. On the one hand, in the target network technique, the parameter of the value (·) with parameter  network is periodically cloned to a target network V θV and fre(·) when quency C. After evolving the target network, V (·) should be replaced by V computing the target value in Eq. (7.22). On the other hand, in the experience replay technique, the recorded experience eh (k, m) is stored in replay memory D = {e1 (k, m), . . . , e N D (k, m)}, where N D is the capacity of the replay memory. In each iteration of the customized DVN algorithm, a set of experiences with size N M (N M < N D ) is randomly sampled from the replay memory to train the value network. Through the two techniques, the parameters of the value network can be updated to minimize the sum of the loss functions θV = arg min θV

NM h=1

L h (θV ) = arg min θV

NM

(yh − Vh (k, m))2 .

(7.24)

h=1

Furthermore, the ε-greedy policy is utilized to tackle the “exploration and exploitation” dilemma. The agent selects the action by the policy function with a probability of ε, while randomly performing an action with a probability of 1 − ε. ε can be generated by

7.4 Illustrative Examples

ε=

135

⎧ ⎪ ⎨0

0.99(I −0.1Imax ) ⎪ 0.6Imax

⎩

0.99

0 < I ≤ 0.1Imax 0.1Imax < I ≤ 0.7Imax , 0.7Imax < I ≤ Imax

(7.25)

where I is the index of iterations and Imax is the maximum number of iterations. The pseudo-code of the customized DVN algorithm is given in Algorithm 7.2. Algorithm 7.2 Pseudo-code of the customized DVN algorithm Inputs: Imax : The maximum number of iterations; M(k): The maximum number of actions; N M : The size of minibatch; Nmax : The maximum number of iterations; C: The target network updating frequency. 1: Initialize replay memory and minibatch; 2: Initialize the value network with random parameter θV ; 3: Initialize the target network with parameter  θV = θV . 4: For I ← 1 to Imax do 5: Initialize the set of the belief states of all the components and the break time at the beginning of the kth break: B(k, 1) and T (k, 1); 6: Compute ε by Eq. (7.25); 7: For m ← 1 to M(k) do 8: Select an action ah (k, m) by ε-greedy policy; 9: Execute ah (k, m) and obtain a reward rh (k, m); 10: Update the set of the belief states of all the components and the remaining time resource after completing ah (k, m): Bh (k, m + 1) and Th (k, m + 1); 11: Store the experience record (Bh (k, m), Th (k, m), ah (k, m), rh (k, m), Bh (k, m + 1), Th (k, m + 1)) in memory D; 12: End For 13: Sample a random minibatch of N M experience records from memory; 14: Calculate the target value yh from the target network by Eq. (7.22); 15: Perform the Adam algorithm on Eq. (7.22) with respect to θV ; 16: Every C iterations, reset the target network by  θV = θV ; 17: End For Output: The optimal value function and policy function.

7.4 Illustrative Examples 7.4.1 A Five-Component System A five-component system is presented to illustrate the effectiveness of the proposed method. The configuration of the five-component multi-state system is delineated in Fig. 7.3. Each component in the system possesses three possible states, i.e., completely failed state (State 1), degraded state (State 2), and perfectly functioning state

136

7 Selective Maintenance and Inspection Optimization …

Fig. 7.3 The configuration of the five-component system Table 7.1 Belief states of each component at the end of the kth mission ID bl,1 (k, 1) bl,2 (k, 1) bl,3 (k, 1) 1 2 3 4 5

0.1 0.2 0.1 0.6 0.3

0.3 0.4 0.2 0.2 0.5

0.6 0.4 0.7 0.2 0.2

(State 3). The performance capacities of each component in States 1, 2, and 3 are 0, 10, and 20 tons/h, respectively. The structure function of the system is defined as G S (t) = min{G 1 (t) + G 2 (t), G 3 (t) + G 4 (t) + G 5 (t)}. Thereby, the system possesses 5 possible performance capacities. The belief state of all the components at the beginning of the kth break is presented in Table 7.1. The time duration of the kth break is set to be L k = 11 hours. The system is regarded as successfully completing the (k + 1)th mission if its performance capacity is not less than Wk+1 = 30 tons/h. The state transition probabilities of each component after executing the (k + 1)th mission are tabulated in Table 7.2. Two maintenance and two inspection levels for each component can be selected. The state transition matrices of maintenance and observation probability matrices of inspection for component l (l ∈ {1, 2, 3, 4, 5}) are Table 7.2 The state transition probabilities of each component executing the (k + 1)th mission ID pl,(2,1) (k + 1) pl,(2,2) (k + 1) pl,(3,1) (k + 1) pl,(3,2) (k + 1) pl,(3,3) (k + 1) 1 2 3 4 5

0.1 0.15 0.25 0.1 0.15

0.9 0.85 0.75 0.9 0.85

0 0 0.1 0 0

0.1 0.15 0.25 0.1 0.15

0.9 0.85 0.75 0.9 0.85

7.4 Illustrative Examples

137

⎡

⎤ ⎡ ⎤ 0.8 0.2 0 100 Plins (1) = ⎣ 0.1 0.7 0.2⎦ , Plins (2) = ⎣0 1 0⎦ , 0.05 0.15 0.8 001 ⎡

⎤ ⎡ ⎤ 0.3 0.5 0.2 001 rep rep Pl (1) = ⎣ 0 0.3 0.7⎦ , Pl (2) = ⎣0 0 1⎦ . 0 0 1 001 The corresponding maintenance and inspection time of component l (l ∈ {1, 2, 3, rep rep 4, 5}) are tl (1) = 3 hours, tl (2) = 6 hours, tlins (1) = 1 hour, and tlins (2) = 2 hours. By implementing the DP algorithm, the probability of the system successfully completing the next mission can be precisely resolved. After enumerating all possible subsequent sets of belief states of all components and remaining time resource, the probability of the system successfully completing the next mission is 0.6911. The runtime of the DP algorithm is 2.161 × 105 seconds. Although the DP algorithm can resolve the probability of the system successfully completing the next mission accurately, it takes too long to run. The customized DVN algorithm is more efficient than DP algorithm. In the customized DVN algorithm, the value network and target network both consist of 3 hidden layers, with each hidden layer composed of 64 neurons. The sizes of the memory and minibatch are set to be N D = 2000 and N M = 32, respectively. The parameters of the ε-greedy are set to be εmin = 0 and εmax = 0.99. The target network is updated every C = 100 iterations. The maximum number of iterations is set to be Imax = 1 × 104 . Consequently, the runtime of the customized DVN algorithm is 1.489 × 103 seconds. The probability of the system successfully completing the next mission evaluated by the value network is 0.6961. A Monte Carlo simulation with 106 sample size is served as a benchmark to demonstrate the accuracy of the customized DVN algorithm. By the Monte Carlo simulation, the probability of the system successfully completing the next mission is 0.6896. We can see that compared with the Monte Carlo simulation, the relative error of the customized DVN algorithm is 0.9426%. Moreover, compared with the DP algorithm, the relative error of the customized DVN algorithm is 0.7235%. However, the runtime of the DP algorithm is almost two hundred times greater than that of the customized DVN algorithm. Therefore, the customized DVN algorithm can achieve an accurate estimation of the probability of the system successfully completing the next mission efficiently. To look more into details of the maintenance and inspection policies of the fivecomponent systems, the optimal policy with different inspection results is illustrated through three possible scenarios, named Scenarios 1, 2, and 3. As shown in Fig. 7.4, the belief state of each component and the remaining time resource are represented by its bar chart and the length of the bottom rectangle, respectively. The abbreviations, i.e., PM #l, IM #l, PI #l, and II #l (l ∈ {1, 2, 3, 4, 5}), represent perfectly maintaining, imperfectly maintaining, perfectly inspecting, and imperfectly inspecting component l, respectively. At the beginning of the kth break, the belief state of all the components and the remaining time resource are known. In decision epoch 1, the optimal policy

138

7 Selective Maintenance and Inspection Optimization … 0.7 0.1 0.2

0.6 0.3

0.1

0.6 0.2 0.2

0.2

0.4 0.4 0.3

0.5 0.2

15

PI #2 Scenario 1

Scenario 2

0.1 0.2

0.6 0.1

0.1

0.6 0.2 0.2

0 0 0.3

2

0

0 0.3

2

0.1

0.6 0.2 0.2

0.3

1

0.3

3

5

0.2

6

PM #4

0.3

1

0.380.44 0.18

0.4

0.7 0.8

0.1 0.2

0.6

0.3

0.2 11

0.1 0.2

0.17 0.03

1 1

0 0

0.6

0

0.5

0.5

0.7

0.1

11

0.3

0.2

IM #2

0.3

0.3

0.2 0.2

0 0

0.5

0.7

0 0

0.6 1

8

0.1 0.2

1

0.1 0.2

0.17 0…

0 0 0

3

0.6

0.7

0.8 1

IM #4

0.1

9

0.1 0.2

0.2

8

0.2

IM #1

0.3

0

0.5

0.5

0.7 0.6

0.3

0 0

0.3

0.2

9

0.1 0.2

1

0.2 0.2

0 0

0.5

0.7 0.6

0.6 1

PM #4

PM #2

0.1

0.3

0.2 0.2

2

9

0.1

0.6

0.2

0.1 0.2

0.6

0.3

1

0.5

0.7

0.1 0.2

0.6

0.3

1

Scenario 3 0.7

0.7

0 0

0 0

0.5

0.3

0.2 11

Fig. 7.4 Different optimal maintenance and inspection policies of different scenarios

0.5 0.2

7.4 Illustrative Examples

139

is to fully inspect component 2. Then, three different scenarios are formed depending on the inspection results. In Scenario 1, the inspection reveals that component 2 is completely failed and the subsequent actions are to perfectly maintain component 2 and imperfectly maintain component 4 due to their poor conditions. In Scenario 2, the inspection reveals that component 2 is in the degraded state, The subsequent actions are to perform imperfect maintenance on component 2 and perfect maintenance on component 4 successively. In Scenario 3, as component 2 is perfectly functioning by the inspection, it is, therefore, unnecessary to maintain component 2. The remaining time resource is utilized to conduct imperfect maintenance on component 1 and perfect maintenance on component 4. From the above analysis, it can be seen that the maintenance and inspection actions can be dynamically performed to achieve the maximum probability of the system successfully completing the next mission. To demonstrate the merits of incorporating inspection strategy into selective maintenance optimization, the problems modeled with and without inspection activities are solved by the customized DVN algorithm and compared. Case 1: The action space with all maintenance and inspection actions; Case 2: The action space with only maintenance actions. The probabilities of the system successfully completing the next mission are 0.6961 and 0.6091 in Case 1 and Case 2, respectively. It can be seen that the probability of the system successfully completing the next mission can be enhanced by taking inspection activities into account. Apart from the action space of the agent, the time duration of the kth break also influences the probability of the system successfully completing the next mission. To further demonstrate the merits of incorporating inspection and imperfect maintenance under different resource constraints, another case, namely, Case 3: The action space with only perfect maintenance, is compared with Cases 1 and 2. The time duration of the kth break varies from 3 hours to 35 hours and the other parameters remain unchanged. The results of the three cases are presented in Fig. 7.5. As shown in Fig. 7.5, the probabilities of the system successfully completing the next mission in Case 1 are never worse than that in Case 2 and Case 3. However, when the break time is greater than 25 hours, the probabilities of the system successfully completing the next mission in the three cases are almost the same. This is because the time resource of the kth break is sufficient to maintain all components, and thus it is useless to carry out inspections. Therefore, in most cases, taking inspection into account can increase the probability of the system successfully completing the next mission, while it is unnecessary to incorporate inspections when the resources are sufficient for executing maintenance on all components. Furthermore, as more components tend to be maintained imperfectly in Case 2 as compared with Case 3, incorporating imperfect maintenance as in Case 2 can achieve a greater probability of the system successfully completing the next mission than that of Case 3.

140

7 Selective Maintenance and Inspection Optimization …

Fig. 7.5 Probability of the system completing the next mission versus the time duration of the break

7.4.2 A Multi-state Coal Transportation System The multi-state coal transportation system is considered as a practical application of the proposed selective maintenance and inspection optimization method. As shown in Fig. 7.6, the system consists of 5 subsystems. Feeder 1 loads coal from the bin to Conveyor 1. Conveyor 1 transfers the coal to the stacker reclaimer. Feeder 2 loads the coal from the stacker reclaimer to Conveyor 2. Based on the configuration, the performance capacity of the coal transportation system at any time instant t can be formulated as follows: G S (t) = min{G 1 (t) + G 2 (t) + G 3 (t), G 4 (t) + G 5 (t), G 6 (t) + G 7 (t) + G 8 (t), G 9 (t) + G 10 (t), G 11 (t) + G 12 (t) + G 13 (t) + G 14 (t)}.

(7.26)

The belief states of all components after the kth mission are tabulated in Table 7.3 and L k = 40 hours. The performance capacities and the state transition probabilities of each component after executing the next mission are tabulated in Tables 7.4 and 7.5, respectively. The state transition matrices of maintenance actions, observation probability matrices of inspection actions, as well as the time duration of maintenance and inspection actions are given in Appendix. Due to the “curse of dimensionality,” the DP algorithm is incapable to resolve the probability of the system successfully completing the next mission for the coal transportation system. By the customized DVN algorithm, the probability of the

7.4 Illustrative Examples

141 11

1

6 4

2

9

12

10

13

7 5

3

8 14

Feeder #1 (Subsystem 1)

Conveyor #1 Stacker-reclaimer Feeder #2 (Subsystem 2) (Subsystem 3) (Subsystem 4)

Conveyor #2 (Subsystem 5)

Fig. 7.6 Configuration of the coal transportation system Table 7.3 Belief states of each component at the end of the kth mission ID State 1 State 2 State 3 State 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14

0.1 0.1 0.1 0.2 0.4 0 0.2 0 0 0.1 0.1 0 0.1 0.2

0.3 0.1 0.1 0.25 0.35 0.1 0.2 0.2 0.2 0.2 0.1 0.1 0.2 0.2

0.5 0.2 0.4 0.55 0.25 0.2 0.1 0.4 0.4 0.3 0.2 0.2 0.3 0

0.1 0.6 0.4 – – 0.7 0.5 0.4 0.4 0.4 0.2 0.3 0.1 0.1

State 5 – – – – – – – – – – 0.4 0.4 0.3 0.5

system successfully completing the next mission is 0.6770, while it is 0.6783 by the Monte Carlo simulation with a sample size of 106 . The relative error of the customized DVN algorithm compared with the Monte Carlo simulation is 0.1916%. Moreover, the training time of the customized DVN algorithm is 9.894 × 103 seconds. Therefore, the customized DVN algorithm can resolve the probability of the system successfully completing the next mission efficiently. To demonstrate the effectiveness of incorporating inspection activities in the coal transportation system, the Monte Carlo simulation with a sample size of 106 for Case 1, that is, the action space with inspection and maintenance actions, and Case 2, that is, the action space with only maintenance actions, are conducted. The probability of the system successfully completing the next mission in the uth sample is denoted

142

7 Selective Maintenance and Inspection Optimization …

Table 7.4 Performance capacities of each component in the coal transportation system (unit: tons/h) ID State 1 State 2 State 3 State 4 State 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14

0 0 0 0 0 0 0 0 0 0 0 0 0 0

40 50 20 70 90 40 30 30 30 40 10 25 25 25

60 80 60 120 130 80 60 70 50 80 40 50 45 65

80 100 80 – – 100 80 90 80 120 60 70 75 80

– – – – – – – – – – 80 90 95 100

by Ru (k + 1). The expectation and standard deviation of the probability of the system successfully completing the next mission can be, respectively, calculated as μ(k + 1) =    σ (k + 1) = 

Nc 1 Ru (k + 1), Nc u=1

c 1 (Ru (k + 1) − μ(k + 1))2 . Nc − 1 u=1

(7.27)

N

(7.28)

The results of the two cases are tabulated in Table 7.6. As shown in Table 7.6, taking account of inspection activities results in a greater expectation and a smaller standard deviation of the probability of the system successfully completing the next mission.

7.5 Closure In this chapter, the selective maintenance and inspection problem for partially observable multi-state systems was introduced. To achieve the maximum probability of the system successfully completing the next mission, the components can be maintained

pl,(2,1) (k + 1)

0.5

0.3

0.2

0.5

0.2

0.4

0.3

0.2

0.4

0.3

0.5

0.2

0.3

0.2

ID

1

2

3

4

5

6

7

8

9

10

11

12

13

14

0.2

0.2

0.3

0.08

0.2

0.2

0.1

0.15

0.2

0.2

0.3

0.4

0.2

0.2

pl,(3,1) (k + 1)

0.3

0.15

0.2

0.2

0.15

0.1

0.18

0.12

0.25

0.2

0.2

0.3

0.3

0.3

pl,(3,2) (k + 1)

0.25

0.15

0.1

0.2

0.2

0.2

0.15

0.3

0.3

–

–

0.2

0.15

0.25

pl,(4,1) (k + 1)

0.2

0.3

0.2

0.3

0.2

0.2

0.3

0.2

0.4

–

–

0.4

0.3

0.2

pl,(4,2) (k + 1)

0.4

0.3

0.6

0.4

0.3

0.4

0.5

0.4

0.3

–

–

0.3

0.2

0.2

pl,(4,3) (k + 1)

Table 7.5 The state transition probabilities of each component executing the (k + 1)th mission pl,(5,1) (k + 1)

0.115

0.2

0.15

0.2

–

–

–

–

–

–

–

–

–

–

pl,(5,2) (k + 1)

0.15

0.12

0.2

0.12

–

–

–

–

–

–

–

–

–

–

pl,(5,3) (k + 1)

0.25

0.3

0.15

0.2

–

–

–

–

–

–

–

–

–

–

pl,(5,4) (k + 1)

0.2

0.3

0.25

0.2

–

–

–

–

–

–

–

–

–

–

7.5 Closure 143

144

7 Selective Maintenance and Inspection Optimization …

Table 7.6 Comparison between cases 1 and 2 Case ID μ(k + 1) 1 2

0.6769 0.6286

σ (k + 1) 0.0988 0.1106

and inspected during the break under limited time resources. To schedule maintenance and inspection activities dynamically, a finite-horizon MOMDP was formulated for the case in which the component states were partially observed and the remaining time resource was fully observed. We customized a DVN algorithm to improve computational efficiency. As shown in the illustrative examples, the proposed selective maintenance and inspection optimization can yield a better result than the case without consideration of inspection actions.

References 1. Corotis RB, Hugh Ellis J, Jiang M (2005) Modeling of risk-based inspection, maintenance and life-cycle cost with partially observable Markov decision processes. Struct Infrastruct Eng 1(1):75–84 2. Ghasemi A, Yacout S, Ouali MS (2009) Evaluating the reliability function and the mean residual life for equipment with unobservable states. IEEE Trans Reliab 59(1):45–54 3. Huang J, Chang Q, Arinez J (2020) Deep reinforcement learning based preventive maintenance policy for serial production lines. Expert Syst Appl 160:113701 4. Jiang T, Liu Y (2020) Robust selective maintenance strategy under imperfect observations: a multi-objective perspective. IISE Trans 52(7):751–768 5. Levitin G (2005) The universal generating function in reliability analysis and optimization. Springer, London 6. Liu X, Sun Q, Ye ZS, Yildirim M (2021) Optimal multi-type inspection policy for systems with imperfect online monitoring. Reliab Eng Syst Saf 207:107335 7. Liu Y, Chen Y, Jiang T (2020) Dynamic selective maintenance optimization for multi-state systems over a finite horizon: a deep reinforcement learning approach. Eur J Oper Res 283(1):166– 181 8. Papakonstantinou KG, Shinozuka M (2014) Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation. Reliab Eng Syst Saf 130:214–224 9. Papakonstantinou KG, Andriotis CP, Shinozuka M (2018) POMDP and MOMDP solutions for structural life-cycle cost minimization under partial and mixed observability. Struct Infrastruct Eng 14(7):869–882 10. Shahraki AF, Yadav OP, Vogiatzis C (2020) Selective maintenance optimization for multi-state systems considering stochastically dependent components and stochastic imperfect maintenance actions. Reliab Eng Syst Saf 196:106738 11. Shani G, Pineau J, Kaplow R (2013) A survey of point-based POMDP solvers. Auton Agent Multi-Agent Syst 27(1):1–51 12. Shi Y, Xiang Y, Li M (2019) Optimal maintenance policies for multi-level preventive maintenance with complex effects. IISE Trans 51(9):999–1011

References

145

13. Wei S, Bao Y, Li H (2020) Optimal policy for structure maintenance: a deep reinforcement learning framework. Struct Saf 83:101906 14. Wu S, Zuo MJ (2010) Linear and nonlinear preventive maintenance models. IEEE Trans Reliab 59(1):242–249

Chapter 8

Selective Maintenance for Systems Operating Multiple Consecutive Missions

8.1 Introduction A majority of the traditional selective maintenance optimization models concerned the allocation of limited maintenance resources in one single break between two adjacent missions. However, in many military and industrial scenarios, engineering systems are required to operate multiple consecutive missions and maintenance will be executed between every two adjacent missions. For example, a production line consisting of multiple machines is scheduled to operate a one-month production task and needs to be routinely maintained every weekend and the decision-makers need to consider how to allocate the limited maintenance resources to all the weekends. From a managerial perspective, the maximum probability of the production line completing all production missions is desired, and a holistic selective maintenance decision needs to be determined before the execution of the first mission. On the one hand, as each component in a repairable system deteriorates over time, the maintenance cost allotted to the component in each break can be different. On the other hand, the profiles of future missions may be uncertain due to various unexpected factors. Therefore, it may be inappropriate to evenly apportion the limited maintenance resources to each break. Decision-makers should not only determine the maintenance activities in each break but also consider the holistic allocation of limited maintenance resources to all breaks. Consequently, this chapter develops a selective maintenance optimization model for systems operating multiple consecutive missions. In the proposed model, the uncertainties associated with the time duration of each future mission and the working time of each component in each future mission are firstly characterized and quantified. Additionally, to determine an optimal selective maintenance strategy for multiple breaks, a new selective maintenance optimization model was formulated as a max-min optimization problem. The rest of this chapter was organized as follows. Section 8.2 introduced the specific selective maintenance problem. Section 8.3 briefly reviewed the Kijima type-I imperfect maintenance model. The conditional survival probability of a component © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_8

147

148

8 Selective Maintenance for Systems Operating …

Fig. 8.1 Chronological order of missions and breaks

was obtained in Sect. 8.4. In Sect. 8.5, the probability of a system successfully completing each future mission was derived. Section 8.6 presented the proposed selective maintenance optimization model. Two illustrative examples and some fundamental analysis results were presented in Sect. 8.7. Section 8.8 is the closure.

8.2 Problem Statements and Model Assumptions A repairable system consisting of multiple repairable components is required to operate multiple consecutive future missions with breaks between two adjacent missions. The chronological order of future missions and breaks is delineated in Fig. 8.1. When the system has completed the previous task, a holistic selective maintenance decision needs to be worked out at the beginning of the first break to determine an optimal allocation of the limited maintenance resources, say maintenance budget in this chapter, to finite quantities of breaks before the operation of multiple consecutive future missions. The assumptions pertaining to the repairable system in question and the related selective maintenance problem in this chapter are summarized below. • The system is composed of M repairable and s-independent binary-state (either working perfectly or failed completely) components that can be configurated arbitrarily. • The system is required to complete N consecutive future missions. For notational consistency and simplicity, the previous completed task will be denoted by the 0th mission. Therefore, we concern about the kth (k ∈ {0, . . . , N − 1}) mission, (k + 1)th break, and (k + 1)th mission hereinafter. • The duration of the 0th mission is a known constant Z 0 . Due to various unexpected factors, the duration of kth (k ∈ {1, . . . , N }) future mission is represented by a stochastic variable Z k . If Z k is a continuous quantity in the range of [z kL , z kU ], the probability density function (PDF) of Z k is denoted by f Z k (z). If Z k is a discrete quantity that manifests n D k possible values, the probability mass function (PMF) of Z k is denoted by pkD ( j) ( j ∈ {1, . . . , n D k }). • The states of component i at the beginning and at the end of the kth mission are represented by  1 if component i is functioning X i,k = 0 if component i is failed

8.3 Imperfect Maintenance Model

149

and  Yi,k =

1 0

if component i is functioning , if component i is failed

respectively. Obviously, the relation X i,k ≥ Yi,k always holds. • The failure time of component i from the beginning of the kth (k ∈ {1, 2, . . . , N }) future mission is represented as Ti,k (Ti,k ∈ [0, +∞)). The working time of the component in the kth mission conditional on Z k = z can be denoted by a random w w w (Ti,k ∈ [0, z]). If X i,k = 0, then Ti,k = 0; if the component fails within variable Ti,k w w = z as the component remains the kth mission, then Ti,k ∈ (0, z); otherwise, Ti,k w functioning throughout the kth mission. Ti,k is the accumulated additional age in the kth mission. Particularly, the working time of component i in the 0th mission w . is denoted by a known constant Ti,0

8.3 Imperfect Maintenance Model In this chapter, the Kijima type-I model [4, 5] is utilized to characterize the imperfect maintenance efficiency of a maintenance action. The Kijima type-I model assumes an age-reduction mechanism in which an imperfect maintenance action can reduce the physical age (therein call effective age) of a maintained component by an amount proportional to the additional age accumulated since the last maintenance. The effective ages of component i at the beginning and end of the kth mission are denoted by Ui,k and Vi,k , respectively. The additional age accumulated in the kth w ). Therefore, we can mission is actually the working time in the kth mission (i.e., Ti,k have w . Vi,k = Ui,k + Ti,k

(8.1)

During the (k + 1)th break, three maintenance options can be chosen for each component to restore component condition. Option #1 Do Nothing (DN). DN means no maintenance action is chosen to be performed. Therefore, the condition of component i remains unchanged after the break, and we have X i,k+1 = Yi,k and Ui,k+1 = Vi,k . Option #2 Component Replacement (CR). CR, as a sort of perfect maintenance, replaces a component by a new one. Obviously, we have X i,k+1 = 1 and Ui,k+1 = 0. Option #3 Imperfect Preventive/Corrective Maintenance (IPM/ICM). IPM/ICM refers to a preventive/corrective maintenance action performed on a functioning/failed component. Based on the Kijima type-I model, the effective age of a functioning/failed component i after an IPM/ICM action in the (k + 1)th break is formulated by

150

8 Selective Maintenance for Systems Operating … w Ui,k+1 = Ui,k + bi,k+1 Ti,k ,

(8.2)

where age reduction factor bi,k+1 ∈ [0, 1] is associated with the maintenance efficiency. A smaller value of bi,k+1 indicates a more efficient maintenance action and correspondingly consumes more maintenance resources. For a functioning component i, bi,k+1 = 1 and bi,k+1 ∈ [0, 1) correspond to the cases of DN and IPM, respectively. Particularly, bi,k+1 = 0 can generate a complete reduction on the additional age accumulated since the last maintenance (herein called imperfectly maintained to a maximum degree, IMMD). For a failed component i, bi,k+1 = 1 and bi,k+1 ∈ [0, 1) correspond to the cases of minimal repair (MR) and ICM, respectively. Particularly, an ICM action for a failed component can be separated into two progressive stages [8]: (1) the failed component is first restored to the functioning state by MR; (2) an additional maintenance action is subsequently performed to further reduce the effective age based on the Kijima type-I model. The value of bi,k+1 is closely related to the maintenance cost allotted to component i in the (k + 1)th break. The maintenance cost of component i in the (k + 1)th break is expressed by ci,k+1 = ci0 + ci,k+1 ,

(8.3)

where ci0 and ci,k+1 are the fixed maintenance cost and the variable preventive/ corrective maintenance (PM/CM) cost for component i. Additionally, the variable maintenance costs of component i for MR, IMMD, and CR are denoted by cimin , cimax , and cicr , respectively. Consequently, the relation between bi,k+1 and ci,k can be formulated by [3, 8]

bi,k+1

⎧ ζ 1 

⎪ i,k+1 ⎨1 − ci,k+1 Yi,k = 1, ci,k+1 ∈ 0, cimax cimax =  1 

,  min min ⎪ −cimin ζi,k+1 max ⎩1 − ci,k+1max Y = 0, c ∈ c , c + c i,k i,k+1 i i i c

(8.4)

i

where ζi,k+1 represents a characteristic parameter of which the value can be evaluated by the function of the effective age and mean residual life of the component [3, 8].

8.4 Survival Probability of a Component The survival probability of component i at the end of the kth mission can be derived in two cases based on the component state at the beginning of the kth mission. Case 1: X i,k = 0 In this case, component i has failed before the kth break (i.e., Yi,k−1 = 0) and is not maintained in the kth mission. Therefore, the survival probability of the component at the end of the kth mission is zero. If component i fails in the jth ( j < k) mission

8.4 Survival Probability of a Component

151

and is not maintained until the (k + 1) break, the additional age accumulated in the kth mission is zero. To avoid notational conflict, the effective age at the beginning of the νth (ν ∈ { j + 1, . . . , k}) mission and the working time in the νth mission are set w = Ti,wj , respectively. As a result, the state and effective age of as Ui,ν = Ui, j and Ti,ν the failed component at the beginning of the (k + 1)th mission can be obtained as mentioned in Sect. 8.3. Case 2: X i,k = 1 In this case, the survival probability of component i in the kth mission conditional on Ui,k = u can be derived as Ri,k (t | u) = Pr{Ti,k > t | Ui,k = u} =

Ri,0 (u + t) , Ri,0 (u)

(8.5)

where Ri,0 (·) is the unconditional survival function of component i. Furthermore, based on the value of Yi,k , the state and effective age of component i at the beginning of the (k + 1)th mission can be obtained in two separated sub-cases. Case 2.1: X i,k = 1 and Yi,k = 1 In this case, the component remains functioning throughout the kth mission. Therew fore, we can have Ti,k = Z k and Vi,k = Ui,k + Z k . Obviously, X i,k+1 = 1, and the effective age of the component at the beginning of the (k + 1)th mission can be obtained readily as mentioned in Sect. 8.3. Case 2.2: X i,k = 1 and Yi,k = 0 In this case, component i fails within the kth mission, and thus the failure time is trunw w cated and Ti,k ∈ (0, z) conditional on Z k = z. The truncated PDF of Ti,k conditional on Z k = z and Ui,k = u can be derived as f Ti,kw |Ui,k (t | u) =

f Ti,k |Ui,k (t | u) , 1 − Ri,k (z | u)

(8.6)

where f Ti,k |Ui,k (t | u) = −dRi,k (t | u)/dt. Based on Eq. (8.1), the PDF of Vi,k conditional on Z k = z and Ui,k = u is given by f Vi,k |Ui,k ,Z k (v | u, z) = f Ti,kw |Ui,k ,Z k (v − u | u, z).

(8.7)

Analogously, the state and effective age of the failed component at the beginning of the (k + 1)th mission can be obtained as mentioned in Sect. 8.3. Particularly, for the case of ICM, if bi,k+1 = 0, that is, IMMD, we have Ui,k+1 = Ui,k ; if bi,k+1 ∈ (0, 1], L U the PDF of Ui,k+1 (Ui,k+1 ∈ [u i,k+1 , .u i,k+1 ]) conditional on Z k = z and Ui,k = u can be derived as



f Ti,kw |Ui,k ,Z k (u  − u)/bi,k+1 | u, z  . (8.8) fUi,k+1 |Ui,k ,Z k (u | u, z) = bi,k+1

152

8 Selective Maintenance for Systems Operating …

u

u bi , k 1 z

u z

Fig. 8.2 Relation among PDFs

L U Based on Eqs. (8.1) and (8.2), we can have u i,k+1 = Ui,k and u i,k+1 = Ui,k + w bi,k+1 Z k . The relation among the conditional PDFs of Ti,k , Ti,k , Vi,k , and Ui,k+1 is illustrated in Fig. 8.2. To sum up, in Case 1 (i.e., X i,k = 0) the survival probability of component i at the end of the kth mission is zero. In Case 2 (i.e., X i,k = 1) taking account of the uncertainty associated with the effective age at the beginning of each future mission, the survival probability of component i during the kth mission conditional on Z = (Z 1 , . . . , Z k−1 ) = (z 1 , . . . , z k−1 ) can be formulated by a multiple integral

Ri,k (t | zk−1 ) = Pr Ti,k > t | Z 1 = z 1 , . . . , Z k−1 = z k−1 U

U

u i,2 u i,k = . . . Ri,k (t | u k ) fUi,k |Ui,k−1 ,Z k−1 (u k | u k−1 , z k−1 ) . . . L u i,2

(8.9)

L u i,k

fUi,2 |Ui,1 ,Z 1 (u 2 | u, z 1 ) du k . . . du 2 , where zk−1 = (z 1 , z 2 , . . . , z k−1 ). It is noteworthy that the integration order of the multiple integral in Eq. (8.9) is Ui,k → . . . → Ui,2 . Particularly, the condition of each component is assumed to be known constants for the 0th mission. In this regard, the survival probability of a component can be obtained readily by Eq. (8.5). To mitigate the computational burden in solving the multiple integral in Eq. (8.9), the Riemann sum is utilized to obtain the approximation result (see the book by Stewart [9]).

8.5 Probability of a System Successfully Completing Missions Mission 1

X i,1

…

Mission 2

Yi,1

X i,2

153 Mission k+1

Mission k

Yi,2

…

X i,k

Yi,k

X i,k

1

1

1

1

0

0

0

0

1

1

1

1

…

0

0

0

0

…

1

Yi,k

1

Fig. 8.3 Possible state paths

8.5 Probability of a System Successfully Completing Missions 8.5.1 Probability of a Component Successfully Completing Future Missions Based on the value of Yi,k , there are 2k possible state paths for component i from the first mission to the kth mission, as illustrated in Fig. 8.3. Obviously, there are 2k−1 possible state paths, denoted by {Yi,1 , . . . , Yi,k−1 , 1}, to reach the case of Yi,k = 1. As a result, the probability of component i successfully completing the kth future mission conditional on Z = z can be formulated as the sum of all 2k−1 survival probabilities Ri,k (zk ) = Pr{Yi,k = 1 | Z = z}  Ri,k (Z k | zk−1 ) = {Yi,1 ,...,Yi,k−1 ,1}

×

k−1 

  Yi, j Ri, j (z j | z j−1 ) + (1 − Yi, j ) 1 − Ri, j (z j | z j−1 ) .

(8.10)

j=1

8.5.2 Probability of a System Successfully Completing Future Missions Let Ys,k and φ(·) represent the system state at the end of the kth mission and the system structure function, respectively. Therefore, the system state can be derived by the composition of the states of all the M components, and we have Ys,k = φ(Y1,k , . . . , Y M,k ). As a result, the probability of the system successfully completing the kth future mission conditional on Z = z can be expressed as

154

8 Selective Maintenance for Systems Operating …

Rs,k (zk ) = Pr{Ys,k = 1 | Z = z}

(8.11)

As mentioned earlier, if the time durations of the future missions are characterized by continuous random variables, the probability of the system successfully completing the kth mission can be formulated as a k-dimensional integral E = Pr{Ys,k = 1} Rs,k

zk

z1

U

=

U

... z kL

Rs,k (zk ) f Z 1 (z 1 ) . . . f Z k (z k )dz 1 . . . dz k .

(8.12)

z 1L

On the contrary, if the time durations of the future missions are characterized by discrete random variables, the probability of the system successfully completing the kth mission can be formulated as a weighted sum D

D

E Rs,k

=

n1  e1 =1

p1D (e1 ) . . .

nk 

pkD (ek )Rs,k (z e1 , . . . , z ek ),

(8.13)

ek =1

where p Dj (e j ) = Pr{z j = z e j }. Analogously, to mitigate the computational burden in solving the multiple integral in Eq. (8.12), the Riemann sum [9] or Gaussian quadrature [2, 6, 7] can be utilized to obtain the approximation result.

8.6 Selective Maintenance Optimization 8.6.1 Selective Maintenance Optimization Model Decision-makers aim to determine a holistic selective maintenance decision to maximize all the probabilities of the system successfully completing all the future missions. Therefore, the selective maintenance decision will determine the allocation of the maintenance budget to all components in all future breaks. As a result, the decision variable vector composed of N M = N × M elements can be represented by (C1 , . . . , Ck+1 , . . . , C N ) = (C1,1 , . . . , C M,1 , . . . , C1,k+1 , . . . , C M,k+1 , . . . , C1,N , . . . , C M,N ),          1st break

(k+1)th break

N th break

where ci,k+1 represents the maintenance cost allocated to component i in the (k + 1)th break. Suppose that the maintenance budget is C0 , a max-min optimization model can then be formulated as

8.6 Selective Maintenance Optimization

155

E E E , (C∗1 , . . . , C∗k+1 , . . . , C∗N ) = arg max min Rs,1 , . . . , Rs,k+1 , . . . , Rs,N

(8.14)

subject to M N  

Ci,k ≤ C0 ,

(8.15)

ci,k ≤ cicr ,

(8.16)

k=1 i=1

ci,k ≥ 0, i ∈ {1, . . . , M}, k ∈ {1, . . . , N }.

(8.17)

Particularly, for a brand-new system put in use, the objective function changes E E E , . . . , Rs,k , . . . , Rs,N } as the probability of the system successfully to max min {Rs,2 completing the first mission is not affected by the allocation of the maintenance budget. Additionally, the first value of k in constraint (9.15) becomes 2. To reduce the computational burden, the decision variables, that is, maintenance costs ci,k+1 , are simplified to discretized maintenance levels li,k+1 that are decimal integers. Therefore, each solution can be represented by a string as following: s = (l1,1 , . . . , l M,1 , . . . , l1,k+1 , . . . , l M,k+1 , . . . , l1,N , . . . , l M,N ),          1st break

(k+1)th break

N th break

where li,k+1 ∈ {0, 1, . . . , NL } and NL is the maximum maintenance level. The relation among the maintenance action, cost, and level are tabulated in Table 8.1.

Table 8.1 Relation among the maintenance action, cost, and level li,k Ci,k Action Yi,k = 1 0 1 .. .

0 ci0 + cimin .. .

li,k .. . NL − 2 NL − 1 NL

ci0 + cimin + (li,k − 3) .. . ci0 + cimax ci0 + cimin + cimax ci0 + cicr

cicr −cimin NL −3

Yi,k = 0

DN IPM .. .

DN MR .. .

IPM .. . IMMD IMMD CR

ICM .. . ICM IMMD CR

156

8 Selective Maintenance for Systems Operating …

8.6.2 Customized Simulated Annealing-Based Genetic Algorithm Meta-heuristic algorithms can be utilized to efficiently resolve the resulting optimization problem. In this chapter, the simulated annealing-based genetic algorithm (SAGA) [1] is customized to seek for the global optimal solution. The pseudo-code of the customized SAGA is given in Algorithm 8.1. In the customized SAGA, the error function, as a convergence criterion, is calculated by Er = sbest − sworst / sbest , where · is the module of a vector. If the error function is less than a pre-specified tolerance, the optimization process will terminate, and the best individual is considered as the optimal solution. Otherwise, in each iteration, the roulette-wheel selection strategy is used to select individuals from the present population based on their fitness to generate a new population. The crossover, mutation, and swap operations are then executed to produce new individuals to explore the unsearched solution space. Specifically, the swap operation is employed by exchanging the two-side maintenance levels of a swap point in an individual based on the swap probability ps . For each individual sd (d ∈ {1, . . . , Np }), a new individual snew can be obtained with the crossover, mutation, and swap operations. If snew dominates sd , sd will be replaced by snew ; otherwise, the deficient individual can be accepted with a small probability of exp{− sd − snew /T0 }. Analogously, the error function is evaluated from the updated population, and the annealing temperature is set to T0 = η × T0 . Algorithm 8.1 Pseudo-code of the customized SAGA Inputs: Np : Population size; pc : Crossover probability; pm : Mutation probability; ps : Swap probability; T0 : Annealing temperature; η: Reducing temperature rate; τ : Tolerance of error function. 1: Generate initial population randomly, and improve the solutions by the CRA and CIA; 2: Select the best and worst individuals (sbest and sworst ), and calculate the error function Er ; 3: While Er > τ do 4: Do the Selection operation for the whole population; 5: For each individual, do the Crossover, Mutation, and Swap operations, improve the solutions by the CRA and CIA, and update the old individual; 6: Select the best and worst individuals (sbest and sworst ), calculate the error function Er , and set T0 = η × T0 ; 7: End While Output: The best solution.

In the selective maintenance optimization problem, an optimal solution can exactly be on the constraint bound or sufficiently close to the constraint bound because of the discretized maintenance costs. If the summation of all maintenance cost Ci,k

8.7 Illustrative Examples

157

(i ∈ {1, . . . , M} and k ∈ {1, . . . , N }) exceeds the budget, the corresponding solution is unfeasible apparently. On the other hand, if the residual budget is not zero or does not approach zero, the corresponding solution is undesirable because the residual budget can further be allotted to improve the objective function. To account for these two problems, two solution-improving algorithms—the cyclic reduction algorithm (CRA) and cyclic increment algorithm (CIA)—are developed to guarantee that the summation of all maintenance costs is equal to or approximates the budget. The pseudo-codes of the CRA and CIA are provided in Algorithms 8.2 and 8.3, respectively. In Algorithms 8.2 and 8.3, c0 represents the excess cost and residual budget in the CRA and CIA, respectively. Let C1 and C2 represent the maintenance cost matrix and maintenance cost interval matrix, respectively, where the dimensions of C1 and C2 are M × (NL ) and M × 3, respectively. The element C1 (i, j) in the ith row and the jth column of matrix C1 represents the maintenance cost of component i with the maintenance level j. The elements of matrix C2 can be derived based on the summary in Table 8.1. They are denoted by follows C2 (i, 1) = cimin , C2 (i, 2) = (cimax − cimin )/(NL − 3), C2 (i, 3) = cicr − (cimax + cimin ) As a consequence, matrix C is composed of N duplications of the maintenance cost matrix C1 , denoted by T  C = CT1 , CT1 , . . . , CT1 .

8.7 Illustrative Examples 8.7.1 A Five-Component System A five-component system, as shown in Fig. 8.4, is studied in this section as a numerical example to validate the proposed selective maintenance optimization model. The system is composed of two subsystems in series, and subsystems 1 and 2 have two and three components configurated parallelly, respectively. The system has completed a previous task and will continue to operate three consecutive future missions. To achieve maximum probabilities of the mission completion, the system will be maintained under a selective maintenance decision before the operation of each future mission. The time duration of the previous mission was Z 0 = 5 days. The failure time of component i is assumed to follow the Weibull distribution with scale parameter θi and shape parameter βi . The associated parameters of each component are listed in Table 8.2.

158

8 Selective Maintenance for Systems Operating …

Algorithm 8.2 Pseudo-code of the CRA Inputs: s: Maintenance level string; c0 : Excess cost; N M : Number of decision variables; C: Duplication of the maintenance cost matrix. 1: Set ch1 = 0 and ch2 = 0; 2: While c0 > 0 do 3: Generate a random permutation from 1 to N M , perm; 4: For k = 1 to N M do 5: pos = perm(k); 6: If s( pos) > 0 then 7: If s( pos) is equal to 1 then 8: Set cut = C( pos, s( pos)); 9: Else 10: Set cut = C( pos, s( pos)) − C( pos, s( pos) − 1); 11: End If 12: Set s( pos) = s( pos) − 1, c0 = c0 − cut, and ch1 = ch1 + 1; 13: If c0 ≤ 0 then 14: End While; 15: End If 16: End If 17: End For 18: If ch2 is equal to ch1 then 19: End While; 20: Else 21: ch2 = ch1; 22: End If 23: End While Output: The improved maintenance level string s. Fig. 8.4 Configuration of the five-component system

In this case study, we suppose that the time duration of each future mission can take several possible discrete values. The probability mass functions (PMFs) of the time durations of the three missions are tabulated in Table 8.3. Ten maintenance levels (i.e., NL = 10) can be chosen for each component. The parameters of the customized SAGA are set to Np = 50, pc = 0.5, pm = 0.2, ps = 0.5, T0 = 20, η = 0.6, and τ = 10−8 . Given the maintenance budget C0 = 210 units, the optimal selective maintenance strategy can be identified as presented in Table 8.4 (Scenario 1). On the other hand, the maintenance budget can be evenly assigned to the three breaks in advance, and

8.7 Illustrative Examples

159

Algorithm 8.3 Pseudo-code of the CIA Inputs: s: Maintenance level string; c0 : Residual budget; N M : Number of decision variables; NL : Maximum maintenance level; C: Duplication of the maintenance cost matrix; C2 : Maintenance cost interval matrix. 1: Find the {i 1 , . . . , i n }th maintenance levels where s(i j ) = 0 ( j = 1, . . . , n); 2: For j = 1 to n do 3: If c0 ≥ C(i j , 1) then 4: Set s(i j ) = 1 and c0 = c0 − C(i j , 1); 5: End If 6: End For 7: Set cmin = min(C2 ), ch1 = 0, and ch2 = 0; 8: While c0 ≥ cmin do 9: Generate a random permutation from 1 to N M , perm; 10: For k = 1 to N M do 11: pos = perm(k); 12: If s( pos) < NL then 13: If s( pos) is equal to 0 then 14: Set add = C( pos, s( pos) + 1); 15: Else 16: Set add = C( pos, s( pos) + 1) − C( pos, s( pos)); 17: End If 18: If c0 ≥ add then 19: Set s( pos) = s( pos) + 1, c0 = c0 − add, and ch1 = ch1 + 1; 20: End If 21: End If 22: End For 23: If ch2 is equal to ch1 then 24: End While; 25: Else 26: Set ch2 = ch1; 27: End If 28: End While Table 8.2 Parameters of components (unit of cost: $1,000, unit of time: day) ID θi βi ci0 cimin cimax cicr X i,0 Yi,0 1 2 3 4 5

26 30 14 18 24

1.8 1.7 2.4 2.2 1.9

2.5 2.5 1.8 1.8 1.8

2.5 3 2 2 2

32 40 24 25 26

40 48 30 32 35

1 1 1 1 1

1 0 0 1 1

Ui,0

w Ti,0

0.5 1.0 0.3 1.2 2.0

5.0 3.5 0.5 5.0 5.0

therefore a separate selective maintenance optimization model can be formulated M Ci,k ≤ C0 /3 (k ∈ {1, 2, 3}). The optimal by replacing Constraint (8.15) by i=1 selective maintenance strategy for the separate optimization model can be determined as shown in Table 8.4 (Scenario 2). It can be seen from the summary in Table 8.4, the

160

8 Selective Maintenance for Systems Operating …

Table 8.3 PMFs of the durations of the future missions (unit of time: day) Z1 9.4 10.6 PMF 0.4 0.6 Z2 9.5 10 10.5 PMF 0.3 0.4 0.3 Z3 9.4 9.8 10.2 0.1 0.2 0.3 PMF

Table 8.4 Optimization results (unit of cost: $1,000) ID Scenario 1 Maintenance levels k=1 k=2 k=3 1 2 3 4 5 E Rs,k act Cs,k Csact

2 2 2 0 0 0.8903 26.94 209.94

2 2 0 10 10 0.8779 90.60

10 1 2 9 3 0.8782 92.40

Scenario 2 Maintenance levels k=1 k=2 4 2 0 1 10 0.8840 69.03 207.74

2 1 0 10 6 0.8601 69.46

10.6 0.4

k=3 10 1 1 4 1 0.8594 69.26

max-min probability of the system successfully completing the three future missions act in Scenario 1 is greater than that in Scenario 2 (see the bold numbers). Let Cs,k act and Cs denote the actual total maintenance cost in the kth break and in all breaks, respectively. We can see that Csact in both scenarios approximates the maintenance act in the three breaks in Scenario 1 are different and not close budget; however, Cs,k to C0 /3. These suggest the advantage of the proposed optimization model that can determine a holistic selective maintenance decision for all future breaks. Intuitively, the probability of the system successfully completing the three future missions will increase with an increased maintenance budget. The max-min probabilities of the system successfully completing the three future missions (see the bold numbers) versus different maintenance budgets are depicted in Fig. 8.5. We can observe that under each setting of the maintenance budget, the proposed holistic selective maintenance strategy (Scenario 1) is superior to the strategy in which the maintenance budget is evenly assigned to each break (Scenario 2). The probability of the system successfully completing each future mission is directly affected by the time duration of the mission. A one-third fraction of the 33 experimental design is implemented to examine the impact of time durations on the optimization result. In the experiments, the time duration of each mission is characterized by a continuous random variable Z k (k ∈ {1, 2, 3}) that is uniformly distributed. The mean of Z k is denoted by z¯ k , and Z k ∼ U (0.9¯z k , 1.1¯z k ). The optimization results

8.7 Illustrative Examples

161

Fig. 8.5 Max-min probability versus maintenance budget Table 8.5 Optimization results (unit of cost: $1,000) act No. (¯z 1 , z¯ 2 , z¯ 3 ) Cs,k k=1 k=2 k=3 0 1 2 3 4 5 6 7 8 9

(10, 10, 10) (7.5, 7.5, 7.5) (7.5, 10, 12.5) (7.5, 12.5, 10) (10, 7.5, 10) (10, 10, 7.5) (10, 12.5, 12.5) (12.5, 7.5, 12.5) (12.5, 10, 10) (12.5, 12.5, 7.5)

19.59 24.96 10.79 5.50 37.04 41.26 14.59 43.76 54.40 53.39

100.10 92.83 58.04 122.40 51.90 118.60 113.10 47.19 88.60 120.69

89.26 90.69 140.69 81.97 120.40 49.11 80.97 118.60 66.90 35.61

E Rs,k k=1

k=2

k=3

0.8868 0.9587 0.8855 0.8796 0.9123 0.9152 0.8022 0.8255 0.8376 0.8352

0.8844 0.9587 0.8621 0.8642 0.9122 0.9162 0.8004 0.8265 0.8426 0.8349

0.8851 0.9588 0.8567 0.8550 0.9129 0.9162 0.8024 0.8253 0.8388 0.8426

with respect to different settings of the time durations of the three future missions are summarized in Table 8.5. We can observe that the actual total maintenance costs in three breaks are different remarkably in each experiment. The max-min probabilities of the system successfully completing the three future missions also vary significantly in each experiment.

162

8 Selective Maintenance for Systems Operating …

11 1

6

4

9

12

10

13

7

2 5 3

8

Feeder 1 (Subsystem 1)

Conveyor 1 Stacker-Reclaimer Feeder 2 (Subsystem 2) (Subsystem 3) (Subsystem 4)

14 Conveyor 2 (Subsystem 5)

Fig. 8.6 Configuration of studied coal transportation system

8.7.2 A Coal Transportation System A coal transportation system is used to further validate the effectiveness of the proposed model for a larger scale system. The coal transportation system in a power station is used to supply a boiler, and is composed of five subsystems, as shown in Fig. 8.6. Feeder 1 transfers the coal from the bin to conveyor 1. Conveyor 1 transports the coal to the stacker-reclaimer that lifts the coal up to the burner level. Feeder 2 loads conveyor 2, which supplies the burner feeding system of the boiler. The system consists of 14 s-independent components, and the failure time of each component follows the Weibull distribution with scale parameter θi and shape parameter βi . The associated parameters of each component are listed in Table 8.6. All the components in the system are completely new, and the system is set to complete the succeeding three consecutive missions, with maintenance activities implemented during each break between two adjacent missions. The maintenance budget is allocated to each component for two future breaks aimed at maximizing the probabilities of the system in successfully completing the second and third missions (the first mission is not affected by maintenance planning). In this example, each mission duration is quantified by the corresponding continuous random variable, Z k (k ∈ {1, 2, 3}). Let z¯ k denote the mean of the kth mission duration, and z¯ k = 10 days. The duration of the kth mission follows the uniform distribution, that is, Z k ∼ U (0.9¯z k , 1.1¯z k ), and NL = 10 is set as the maximum maintenance level. The parameters of SAGA are set to Np = 50, pc = 0.5, pm = 0.2, ps = 0.5, T0 = 20, η = 0.6, and τ = 10−8 . Given the budget C0 = 500 units, the optimal selective maintenance strategy with the proposed method is tabulated as Scenario 1 in Table 8.7. In the same manner, the budget is evenly assigned to the two breaks, and a separate selectivemaintenance optiM Ci,k ≤ 0.5C0 mization can be formulated by replacing Constraint (8.15), that is, i=1 (k ∈ {2, 3}). The resulting optimal solution is listed as Scenario 2 in Table 8.7. It can

8.7 Illustrative Examples

163

Table 8.6 Parameters of components (unit of cost: $1,000, unit of time: day) ID θi βi ci0 cimin cimax 1 2 3 4 5 6 7 8 9 10 11 12 13 14

22 25 25 27 26 30 18 26 26 27 15 17 17 19

1.4 2.1 1.5 2.5 1.7 2.3 2.4 1.9 1.7 2.0 2.7 1.4 2.3 2.1

1.8 2.5 2.5 3.0 2.2 1.8 2.4 2.6 2.0 2.2 2.6 2.6 2.2 1.8

2.0 3.0 3.5 3.6 3.0 2.0 3.2 3.5 3.0 2.8 3.0 3.2 2.5 2.2

28 34 42 44 36 26 38 40 32 35 36 36 32 28

cicr 36 42 48 51 44 32 46 49 40 45 45 48 40 35

be seen that the optimal allocations of the limited maintenance cost are considerably different for the two scenarios. The max-min probabilities of the system in successfully completing each mission in the two scenarios are highlighted in the row of E . The comparison of the probabilities of the system in successfully completing the Rs,k second and third missions for the two scenarios, with the percentages of Scenario 1 being greater than that of Scenario 2, indicates a chance of 0.17%. This indicates that the proposed method can yield higher probabilities that the system will successfully complete future missions than in the case where the maintenance budget is evenly allocated to all breaks. In Scenario 1, the actual total maintenance cost of the second break is remarkably lower than that of the third break ($57,270 or corresponding to 20.56%). Nevertheless, in Scenario 2, because the budget is evenly distributed, the actual total maintenance costs for the two breaks considerably approximate each other. In order to demonstrate the robustness of the customized SAGA, 30 independent runs of SAGA for the optimization problem are implemented. For Scenario 1, the mean value and standard deviation of the optimal results from these runs are 0.9562 and 0.0010, respectively; for Scenario 2, the mean value and standard deviation of the optimal results from these runs are 0.9541 and 0.0011, respectively. On the one hand, the standard deviation is considerably smaller compared with the mean value in both scenarios, which confirms that the customized SAGA is relatively robust for our specific problem. On the other hand, the mean value in Scenario 1 is greater than that in Scenario 2, which further validates the effectiveness of the proposed method. The optimal solutions in Table 8.7 are the best optimization results of Scenarios 1 and 2 from their respective 30 runs.

164

8 Selective Maintenance for Systems Operating …

Table 8.7 Optimization results (unit of cost: $1,000) ID Scenario 1 Maintenance levels k=1 k=2 k=3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 E Rs,k act Cs,k Csact

– – – – – – – – – – – – – – 0.9833 – 499.87

1 2 1 2 5 2 1 2 8 8 0 1 9 9 0.9583 221.30

1 10 1 10 1 10 1 1 7 5 10 1 2 1 0.9585 278.57

Scenario 2 Maintenance levels k=1 k=2 – – – – – – – – – – – – – – 0.9833 – 498.69

2 8 0 1 5 8 0 2 9 8 0 0 8 8 0.9567 248.99

k=3 1 6 1 10 1 6 1 1 4 9 10 1 1 2 0.9564 249.70

8.8 Closure In this chapter, a selective maintenance optimization model was developed for systems operating multiple consecutive future missions with uncertainty. Due to various unexpected factors, the profiles of future missions may be uncertain. Therefore, uncertainties were introduced into the duration of each future mission and the working time of each component in each future mission. The uncertainties were characterized and quantified by various distributions. As a result, the probability of a system successfully completing each future mission was formulated by an integral. A maxmin selective maintenance model was then constructed that can determine a holistic allocation of a limited maintenance budget. Two illustrative examples were analyzed to validate the proposed selective maintenance optimization model. The comparative results show that the proposed model considering multiple consecutive missions operation can perform better than the model in which the maintenance budget was evenly assigned in advance.

References

165

References 1. Chiang CH, Chen LH (2007) Availability allocation and multi-objective optimization for parallel-series systems. Eur J Oper Res 180(3):1231–1244 2. Evans DH (1972) An application of numerical integration techniques to statistical tolerancing. III—general distributions. Technometrics 14(1):23–35 3. Jiang T, Liu Y (2020) Selective maintenance strategy for systems executing multiple consecutive missions with uncertainty. Reliab Eng Syst Saf 193:106632 4. Kijima M (1989) Some results for repairable systems with general repair. J Appl Probab 26(1):89–102 5. Kijima M, Morimura H, Suzuki Y (1988) Periodical replacement problem without assuming minimal repair. Eur J Oper Res 37(2):194–203 6. Lee SH, Chen W (2009) A comparative study of uncertainty propagation methods for blackbox-type problems. Struct Multidiscipl Optim 37(3):239 7. Lee SH, Chen W, Kwak BM (2009) Robust design with arbitrary distributions using Gauss-type quadrature formula. Struct Multidiscipl Optim 39(3):227–243 8. Pandey M, Zuo MJ, Moghaddass R, Tiwari MK (2013) Selective maintenance for binary systems under imperfect repair. Reliab Eng Syst Saf 113:42–51 9. Stewart J (2014) Calculus: early transcendentals. Cengage Learning, Boston, USA

Chapter 9

Dynamic Selective Maintenance for Multi-state Systems Operating Multiple Consecutive Missions

9.1 Introduction Most traditional selective maintenance models concern on the success of the next mission only. However, in many engineering practices, systems are required to operate a sequence of missions and will be maintained in each break between two consecutive missions. For example, a truck is scheduled to finish a sequence of transportation tasks with breaks between two adjacent tasks. Another typical instance is that an aircraft is intended to complete multiple flights with inspection and maintenance between two consecutive flights. For multiple consecutive missions, the maintenance activities in a specific break affect not only the success of the next mission, but also the successes of subsequent missions. The decision-makers need to, therefore, trade off the success of the next mission against the future missions with limited maintenance resources. In Chap. 8, we have investigated the selective maintenance model for systems executing multiple consecutive future missions. A max-min selective maintenance model that can determine a holistic allocation of limited maintenance budget was formulated. For systems operating multiple consecutive future missions, the decisionmaking of selective maintenance relies on the condition of components, total or remaining maintenance resources, and the characteristics of future missions. The selective maintenance model in Chap. 8 determined maintenance actions for all the future breaks at the very beginning of the first mission. In fact, the condition of components in a repairable system can be inspected before the start of maintenance activities in each break. Therefore, the selective maintenance strategy needs to be dynamically adjusted or determined in each break. This chapter developed a basic dynamic selective maintenance model for multi-state systems (MSSs) operating multiple consecutive future missions. The dynamic selective maintenance problem was inherently a dynamic programming problem which would be formulated by a Markov decision process (MDP). A customized deep reinforcement learning (DRL) method in the actor-critic framework was then introduced to address the “curse of dimensionality.” © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3_9

167

168

9 Dynamic Selective Maintenance for Multi-state Systems …

The remainder of this chapter was rolled out as follows. Section 9.2 presented the problem statement and basic assumptions. The Kijima type-II imperfect maintenance model was described in Sect. 9.3. The expected number of the success of future missions was calculated in Sect. 9.4. In Sect. 9.5, an MDP was utilized to formulate the resulting sequential decision problem, and a customized DRL method was achieved to address the problem of the “curse of dimensionality.” Two illustrative examples were provided in Sect. 9.6. Section 9.7 was the closure.

9.2 Problem Statements and Model Assumptions A multi-component MSS is scheduled to execute multiple consecutive missions sequentially. To ensure the successes of subsequent missions, the maintenance action for each component is dynamically determined based on the condition of each component. For example, as shown in Fig. 9.1, a brand-new MSS consisting of three components is planned to perform multiple consecutive missions. In Scenario 1, at the end of the first mission, we observe that components 1 and 2 failed within the first mission. To ensure the successes of subsequent missions, maintenance actions will be executed in the first break. However, due to limited maintenance resources, only component 2 is selected to be repaired to the functioning state, whereas component 1 remains in the failure state. The system is then restored to the functioning state and continues to execute the second mission. At the end of the second mission, we observe that component 3 failed during the second mission. Based on the present states of all the components and the remaining maintenance resources, we determine to repair component 3 in the second break to restore the system state. In Scenario 2, components 2 and 3 failed within the first mission. However, only component 3 is selected to be repaired in the first break. As no component failed within the second mission, component 2 is then restored to the functioning state in the second break. The basic assumptions in the dynamic selective maintenance problem are as follows: Maintenance

Maintenance

Scenario 1

…

Component 1 Component 3 Component 2

…

Scenario 2 1st Mission

1st Break

Decision-Making

2nd Mission

2nd Break

3rd Mission

Decision-Making

Fig. 9.1 Selective maintenance for a system executing multiple consecutive missions

… time

9.3 Imperfect Maintenance Model

169

• An MSS consists of M s-independent binary-state components of which the failure times can follow arbitrary distributions. The performance capacity of component l at time t is denoted by G l (t) ∈ {gl,1 , gl,0 }, where gl,1 and gl,0 = 0 denote the performance capacities in the functioning state (state 1) and failure state (state 0), respectively. The states of component l at the beginning and the end of the kth mission are denoted by  1 if component l is functioning , (9.1) X l,k = 0 if component l is failed and  Yl,k =

1 if component l is functioning , 0 if component l is failed

(9.2)

respectively. Consequently, for the states of all the components, we can have Xk = (X 1,k , X 2,k , . . . , X M,k ) and Yk = (Y1,k , Y2,k , . . . , Y M,k ). • The performance capacity of the entire system is completely determined by the performance capacities of all the components and structure function φ(·) G(t) = φ(G 1 (t), G 2 (t), . . . , G M (t)),

(9.3)

where G(t) ∈ G = {g1 , g2 , . . . , g Ns }, and Ns denotes the number of possible performance capacities of the system. For example, for a two-component flow transmission system, we have G(t) = G 1 (t) + G 2 (t) if the two components are connected in parallel, or whereas G(t) = min{G 1 (t), G 2 (t)} is used for a series configuration. • The system is planned to perform K consecutive future missions. The time duration and mission demand of the kth mission are denoted by z k and Wk , respectively. A mission is successfully completed if the performance capacity of the system is greater than a pre-specified mission demand. The failure time of each component in each mission can be exactly known. The system configuration remains unchanged throughout all the missions. • The condition of each component can be restored by multiple optional maintenance actions in each break. The limited maintenance budget and maintenance time in each break are denoted by Clim and Tlim , respectively.

9.3 Imperfect Maintenance Model Suppose that NL + 1 maintenance actions are optional for each component, in which doing nothing is viewed as an optional maintenance action too.

170

9 Dynamic Selective Maintenance for Multi-state Systems …

The maintenance action selected in the kth break for component l is denoted by al,k (al,k ∈ {0, 1, . . . , NL}), where al,k = NL and al,k = 0 denote perfect maintenance and doing nothing, respectively. Specifically, if component l fails at the end of the kth mission, al,k = 1 represents a minimal repair that merely recovers component l back to a functioning state. For component l in the kth break, the maintenance cost associated with maintenance action al,k is given by  0 al,k = 0 Cl,k = 0 , (9.4) cl + cl (al,k , Yl,k ) al,k = 1, 2, . . . , NL where cl0 is the fixed maintenance cost of component l, such as the disassembly and assembly costs, the setup cost, and the cost of organizing maintenance personnel. cl (al,k , Yl,k ) denotes the variable maintenance cost that is determined by the selected maintenance action al,k and the state of component l at the end of the kth mission. Specifically, cl (al,k , 1) and cl (al,k , 0) represent the preventive maintenance and corrective maintenance for component l, respectively. The total maintenance cost to be consumed in the kth break, denoted by Ck , can be evaluated by Ck =

M 

Cl,k .

(9.5)

l=1

In the same manner, the maintenance time of component l can be defined as follows  0 al,k = 0 Tl,k = 0 , (9.6) tl + tl (al,k , Yl,k ) al,k = 1, 2, . . . , NL where tl0 is the fixed maintenance time of component l, and tl (al,k , Yl,k ) is the variable maintenance time associated with the selected maintenance action and the state of component l at the end of the kth mission. Suppose that the total maintenance time to be spent in the kth break, denoted by Tk , can be given as Tk =

M 

Tl,k .

(9.7)

l=1

Based on the age reduction mechanism, the Kijima type-II model is utilized to characterize the imperfection of maintenance actions in this chapter. In the Kijima type-II model, the degradation process of a component can be characterized by its effective age, which may not be necessarily the same as the calendar age. The effective age of component l at the beginning and end of the kth mission are denoted by Al,k and Bl,k , respectively. Based on the Kijima type-II model, one has the following

9.4 Dynamic Selective Maintenance Modelling

Al,k+1 = bl,k Bl,k ,

171

(9.8)

where bl,k (0 ≤ bl,k ≤ 1) is the age reduction factor of the selected maintenance action for component l in the kth mission. The value of bl,k is determined by the selected maintenance action al,k . For all the components, we can have vectors Ak = (A1,k , A2,k , . . . , A M,k ), Bk = (B1,k , B2,k , . . . , B M,k ), and ak = (a1,k , a2,k , . . . , a M,k ). Consequently, the states and effective ages at the beginning of the (k + 1)th mission, denoted by Xk+1 and Ak+1 , can be completely determined by Yk , Bk , and ak .

9.4 Dynamic Selective Maintenance Modelling In this section, the dynamic selective maintenance problem is formulated by an MDP. In accordance with the state and effective age of each component, the maintenance actions in each break should be dynamically determined to maximize the expected number of successes of future missions, that is, ak = π(Yk , Bk , k), where π(·) denotes a “maintenance policy” (also called a policy function).

9.4.1 States and Effective Ages of Components at the End of a Mission The state and effective age of component l at the end of the kth mission can be derived in two cases based on the component state at the beginning of the kth mission. Case 1: X l,k = 0 In this case, component l remains in the failure state throughout the kth mission. The effective age of the component also remains unchanged. Therefore, we have Yl,k = X l,k = 0 and Bl,k = Al,k . Case 2: X l,k = 1 In this case, the survival probability of component l at the end of the kth mission is derived as rl,k = 1 −

1 − Fl (Al,k + z k ) Fl (Al,k + z k ) − Fl (Al,k ) = , 1 − Fl (Al,k ) 1 − Fl (Al,k )

(9.9)

where Fl (t) and fl (t) denote the cumulative distribution function and probability density function of the failure time of the component, respectively. Furthermore, based on the value of Yl,k , the state and effective age of component l at the end of the kth mission can be obtained by one of the following two sub-cases.

172

9 Dynamic Selective Maintenance for Multi-state Systems …

Case 2.1: X l,k = Yl,k = 1 In this case, component l remains functioning throughout the kth mission. Therefore, the operating time during the kth mission is equal to the time duration of the kth mission, and we have Bl,k = Al,k + z k .

(9.10)

Case 2.2: X l,k = 1 and Yl,k = 0 In this case, component l fails within the kth mission. The operating time of component l, denoted by Tl,k (Tl,k ∈ (0, z k )), is a random variable with the probability density function formulated as follows f Tl,k (t) =

fl (Al,k + t) . Fl (Al,k + z k ) − Fl (Al,k )

(9.11)

Essentially, f Tl,k (t) is the conditional probability density function of fl (t) given that component l fails within the time interval of (Al,k , Al,k + z k ). At the end of the kth mission, therefore, the effective age of component l can be written as Bl,k = Al,k + Tl,k .

(9.12)

To sum up, the state transition probability of component l in the kth mission is defined as ⎧ 1 X l,k = 0, Yl,k = 0 ⎪ ⎪ ⎪ ⎨0 X l,k = 0, Yl,k = 1 . (9.13) pl,k (i, j) = Pr{Yl,k = j|X l,k = i} = ⎪ X l,k = 1, Yl,k = 0 1 − r l,k ⎪ ⎪ ⎩ X l,k = 1, Yl,k = 1 rl,k The effective age of component l at the end of the kth mission is one of the following three cases ⎧ ⎪ X l,k = 0, Yl,k = 0 ⎨ Al,k (9.14) Bl,k = Al,k + z k X l,k = 1, Yl,k = 1 . ⎪ ⎩ Al,k + Tl,k X l,k = 1, Yl,k = 0 To summarize, Yk and Bk can be completely determined by Xk and Ak . The transition of the state and effective age of each component is memoryless.

9.4 Dynamic Selective Maintenance Modelling

173

9.4.2 Probability of System Successfully Completing a Mission The probability of the repairable system successfully completing the kth mission, denoted by R(k), can be evaluated by the system state distribution. In this chapter, the UGF method [6] is utilized to obtain the system state distribution. The probability mass function of each component state can be represented by a polynomial form, that is, a UGF. The UGF of component l at the end of the kth mission is represented as u l,k =

1 

pl,i · z gl,i ,

(9.15)

i=0

where pl,i = Pr{Yl,k = i}. Given X l,k and Al,k , pl,Yl,k can be evaluated by Eq. (9.13). Let ps,i,k denote the probability of the system performance capacity being equal to gi at the end of the kth mission, and the UGF of the system at the end of the kth mission, denoted by Uk , can be evaluated by Uk =

Ns 

ps,i,k z = gi

1  Y1,k =0

i=1

···

1 

M

Y M,k =0

pl,Yl,k z

φ(g1,Y1,k ,...,g M,Y M,k )

.

(9.16)

l=1

The probability of a system successfully completing the kth mission can be, then, written as follows Ns  ps,i,k · 1(gi ≥ Wk ), (9.17) R(k) = i=1

where 1(·) is an indicator function that is equal to one if gi ≥ Wk is true or zero otherwise. If the demand of the kth mission is a random variable with H possible values, Eq. (9.17) can be further written as R(k) =

H  h=1

Pr{Wk = wh,k } ·

Ns 

ps,i,k · 1(gi ≥ wh,k ),

(9.18)

i=1

where wh,k is the hth possible value of the demand of the kth mission. The probability of a system successfully completing the kth mission is completely determined by the states and effective ages of all the components at the beginning of the kth mission. The probability R(k) can therefore be represented as a function of Xk and Ak , that is, R(k) = R(Xk , Ak ). Referring to Sect. 9.4.2 that Xk and Ak are completely determined by ak−1 , Bk−1 , and Yk−1 , the probability R(k) can be further represented by R(k) = R(Yk−1 , Bk−1 , ak−1 ).

174

9 Dynamic Selective Maintenance for Multi-state Systems …

9.4.3 Markov Decision Process Formulation As the transitions of the states and effective ages of all the components are independent of their histories, the sequential decision problem can be formulated as an MDP. The state space, the action space, and the reward function of the MDP are detailed as follows. State Space: The state space of the MDP consists of all the possible combinations of the states and effective ages of components in a system at the end of a mission. Action Space: The action space of the MDP is the set of feasible maintenance actions for all the components. In the kth break, the action space of the MDP, denoted by Sact k , can be formulated as   M M   act Sk = ak | Cl,k ≤ Clim , Tl,k ≤ Tlim . (9.19) l=1

l=1

Reward Function: Given the state and effective age, and the selected action of each component in the kth break, the reward function of the MDP is the probability of the system successfully completing the (k + 1) mission, that is, R(k + 1) = R(Yk , Bk , ak ). Given the states and the effective ages of all the components in a system at the end of the kth mission, that is, Yk and Bk , the maximum expected number of the successes of the future remaining K − k missions is the optimal value function of the MDP, denoted by V ∗ (Yk , Bk , k). If only one future mission is left (k = K − 1), the optimal value function (i.e., V ∗ (Y K −1 , B K −1 , K − 1)) is equivalent to the maximum probability of the system successfully completing the last mission and can be computed by the following V ∗ (Y K −1 , B K −1 , K − 1) =

max

aK −1 ∈Sact K −1

R(k) =

max

aK −1 ∈Sact K −1

R(Y K −1 , B K −1 , aK −1 ). (9.20)

Let = {l|X l,k = 1, Yl,k = 0} denote the set of components that fail within the kth mission, and one has SFk ⊆ {1, 2, . . . , M}. Let Mk denote the number of components in SFk . Hence, if more than one future mission is left, the Bellman equation of the MDP can be formulated as follows SFk

9.4 Dynamic Selective Maintenance Modelling

∗

V (Yk , Bk , k) = max E π

⎡

Y1,k+1 =0 Y2,k+1 =0



K 

Rn |Yk , Bk = maxact {R(Yk , Bk , ak ) ak ∈Sk

n=k+1

⎛ zk zk zk 1 1 M   ⎝ ··· pl,X l,k+1 ,Yl,k+1 ···

1 

+⎣

175

Y M,k+1 =0

l=1

0

0

0



Mk+1 ∗

f Tlm ,k+1 (tlm ,k+1 )V (Yk+1 , Bk+1 , k + 1)dtl1 ,k+1 dtl2 ,k+1 · · · dtl Mk+1 ,k+1

,

m=1

(9.21) is the mth component in is the operation time of comwhere lm ∈ ponent lm during the (k + 1)th mission (i.e., tlm ,k+1 = Blm ,k+1 − Alm ,k+1 ). For a brandnew system, the maximum expected number of the successes of future missions can be solved by the following F Sk+1

F Sk+1 ; tlm ,k+1

∗

N = V (X1 , A1 , 0) = max E ⎡ = R(X1 , A1 ) + ⎣

π

1  Y1,1 =0

z1 ··· 0

z1 M1 0

m=1

K 

R(k)|X1 , A1

k=1

···

1 

M

Y M,1 =0

pl,X l,1 ,Yl,1 ·

(9.22)

l=1

⎞⎤⎫ ⎬ f Tlm ,1 (tlm ,1 ) · V ∗ (Y1 , B1 , 1)dtl1 ,1 · · · dtl M1 ,1 ⎠⎦ . ⎭

Given the states, the effective ages, and the selected maintenance actions of all the components in a system at the end of the kth mission, that is, Yk , Bk , and ak , the maximum expected number of the successes of the future remaining K − k missions is an optimal “Q-function” (also called “state-action” value function or “actionvalue” function) of the MDP, denoted by Q ∗ (Yk , Bk , k, ak ), which can be written as K

 ∗ Q (Yk , Bk , k, ak ) = R(k + 1) + max E Rn |Yk , Bk , ak π

n=k+2



= R(Yk , Bk , ak ) + max E π

K 

(9.23)

Rn |Yk , Bk , ak .

n=k+2

The relation between the optimal value function and the optimal Q-function can be written as follows V ∗ (Yk , Bk , k) = maxact Q ∗ (Yk , Bk , k, ak ). ak ∈Sk

(9.24)

176

9 Dynamic Selective Maintenance for Multi-state Systems …

The optimal value function can be solved by the optimal Q-function. On the other hand, the optimal Q-function can be computed by the optimal value function via the Bellman equation. Based on the optimal Q-function and Bellman equation, the optimal selective maintenance strategy can be found by the following π ∗ (Yk , Bk , k) = argmax Q ∗ (Yk , Bk , k, ak ).

(9.25)

ak ∈Sact k

9.5 Customized Deep Reinforcement Learning Method Due to the effective age of a component is a continuous variable, the state space of the resulted MDP is uncountable. Moreover, the state and action spaces of the resulted MDP increase exponentially with increase in the number of components in a system. For example, without the constraints of maintenance resources, the number of possible maintenance actions is (NL + 1) M for each combination of the states and the effective ages of components. Hence, traditional methods, such as the dynamic programming, ant colony algorithm, and genetic algorithm, cannot be readily utilized to resolve the dynamic selective maintenance problem. Alternatively, as a promising tool to resolve complicated large-scale MDP problems, the deep reinforcement learning (DRL) method, which has been studied in the literature [1–5]. In this section, a DRL algorithm is customized in the framework of the actor-critic algorithms to resolve the MDP in this chapter and overcome the “curse of dimensionality.”

9.5.1 Actor-Critic Framework In DRL algorithms, an agent interacts with a dynamic environment, discovering an optimal policy to maximize expected cumulative rewards. In an actor-critic framework, an agent is composed of a critic and an actor. By evaluating the value of the optimal Q-function, the critic judges the effectiveness of the selected maintenance actions. Based on the states and effective ages of all the components, an actor will determine the maintenance action of each component for the maximum Q-value. By performing the maintenance actions selected by the actor, the agent can estimate the states and effective ages of all the components at the end of the mission, together with the reward, that is, the probability of the system successfully completing the next mission. As the iteration evolves, the actor adjusts its parameters for a greater Q-value from the critic, whereas the critic changes its parameters based on the rewards from the selected maintenance actions to accurately judge the effectiveness of the selected maintenance actions. An illustration of the customized DRL algorithm is given in Fig. 9.2.

9.5 Customized Deep Reinforcement Learning Method

kth Mission

kth Break

177

(k+1)th Mission

(k+1)th Break … time

Yk , B k , k

Reward (Rk 1 ) ak

Q-Value Critic

Actor Actions Agent

Fig. 9.2 An illustration of the actor-critic framework

The critic is represented by a multi-layer artificial neural network, denoted by Q-network, to approximate the optimal Q-function, that is, Q(Yk , Bk , k, ak ) ≈ Q ∗ (Yk , Bk , k, ak ), where θ Q denotes the parameters of the network. The artificial neural network is the so-called Q-network. In the same manner, the actor is also μ μ μ μ represented by an actor network, that is, ak = μ(Yk , Bk , k) = (a1,k , a2,k , . . . , a M,k ), μ where al,k is the maintenance action for component l in the kth break. However, output μ al,k is a continuous value and has to be converted into a discrete value to represent all the optional maintenance actions for component l. To deal with this problem and search the optimal maintenance actions in the large discrete action space, a postprocess inspired by the Wolpertinger architecture [1] is used in this chapter. An illustration of the postprocess in a case of a two-component system is delineated in Fig. 9.3. The dark dots are the optional maintenance actions for the two components and the purple dot represents an output from the actor network (i.e., μ ak ). The detailed procedures of the postprocess are as follows. Step 1: Produce a proto-action. Based on the inputs of Yk , Bk , and k, the actor network μ obtains a continuous output vector, that is, ak = μ(Yk , Bk , k). Then, by rounding μ μ down elements in ak , ak is normalized to an integer vector, which is denoted by a p proto-action and represented as ak . p

Step 2: Find all the neighboring solutions to the proto-action ak . All the optional p maintenance actions with the 2-norm distance to the proto-action ak being less than p a prespecified value of L will be identified as the neighbors of the proto-action ak . A greater value of L implies a larger local search space and a longer computation time. Step 3: Select the feasible neighboring solutions complying with all the constraints of maintenance resources. If all of the neighboring solutions violate the constraints, the neighboring solution with the minimal maintenance cost and time will be chosen as a new proto-action, and the iteration goes to Step 2 for finding all the neighbors to the new proto-action. Steps 2 and 3 will be repeatedly executed until at least one of the neighbors of the proto-action satisfies all the constraints. The illustration of selecting

178

9 Dynamic Selective Maintenance for Multi-state Systems …

Fig. 9.3 Illustration of the postprocess for the actor

the feasible neighbors around the proto-action is delineated in Fig. 9.4, where the dark dots represent the optional maintenance actions for all the components. Step 4: Identify the optimal maintenance actions. Based on Yk , Bk , and k, the solution with the maximum value of the Q-network will be selected by the actor as the optimal maintenance actions for all the components. In accordance with actor network μ and Q-network Q, the process of selecting the maintenance actions for all the components can be represented by a policy function as π(Yk , Bk , k|μ, Q) = ak .

9.5 Customized Deep Reinforcement Learning Method

179

Fig. 9.4 Illustration of searching for feasible neighboring solutions

9.5.2 Agent Training: Experience Replay and Target Network The customized DRL algorithm iteratively simulates the processes of executing missions and maintenance decision-making until reaching a prespecified maximum number of iterations Imax . In each iteration, the agent sequentially selects maintenance actions for all the breaks. The detailed simulation procedure of the agent in the kth break is as follows: First, given Yk and Bk , the agent executes the maintenance actions selected by the actor (i.e., ak ) for all the components. Second, the probability of the repaired system successfully completing the (k + 1)th mission (i.e., R(k + 1)) can be evaluated by Eq. (9.18). Third, the states and effective ages of all the components, that is, Yk+1 and Bk+1 , at the end of the (k + 1)th mission can be randomly simulated. After executing a simulation iteration, a transition realization denoted by (Yk , Bk , k, ak , R(k + 1), Yk+1 , Bk+1 ) can be recorded. The agent can be iteratively trained by a number of recorded transitions, which are stored in a memory, denoted by D. The memory will be overwritten by the N D most recent transition records. To break the correlations among records, which have an adverse impact on the agent training, the experience replay technique [3] is utilized. Based on the experience replay technique, a minibatch with the size of Nm (Nm < N D ) is randomly sampled from memory D and used to update the Q-network and actor network. Moreover, the target network technique is also used to enhance the robustness of the DRL algorithm. After every C iteration, the Q-network and the actor network are cloned to obtain a target Q-network and a target actor network, respectively, and

180

9 Dynamic Selective Maintenance for Multi-state Systems …

the two target networks are used to generate the target values for the update of the Q-network and the actor network in each iteration. The target Q-network and target ˆ and μ(·), actor network are respectively denoted by Q(·) ˆ whereas the parameters of the target Q-network and target actor network are denoted by θˆQ and θˆμ , respectively. Based on the experience replay and target network techniques, the agent can iteratively update its Q-network and actor network by the transition records of the minibatch. The jth record in the minibatch is denoted by (Yk j , j , Bk j , j , k j , ak j , j , Rk j +1, j , Yk j +1, j , Bk j +1, j ), and the corresponding target value of the Q-network, denoted by y j , can be computed by  Rk j +1, j kj = K − 1 yj = , (9.26) Rk j +1, j + max{0, min{K − k j − 1, q j }} k j < K − 1 ˆ k j +1, j , Bk j +1, j , k j + 1, π(Yk j +1, j , Bk j +1, j , k j + 1|μ, ˆ ˆ Q)), and the where q j = Q(Y maximum expected number of the successes of future remaining K − k j − 1 missions falls in the range of [0, K − k j − 1]. The parameters of the Q-network can be updated by the following θ Q = arg min θQ

Nm 

(y j − Q(Yk j , j , Bk j , j , k j , ak j , j ))2 .

(9.27)

j=1

The parameters of the actor network can be updated by the following Nm    aˆ k , j − μ(Yk , j , Bk , j , k j ) , θμ = arg min j j j 2 θμ

(9.28)

j=1

ˆ Q), and ·2 is the 2-norm of a vector. where aˆ k j , j = π(Yk j , j , Bk j , j , k j |μ, After every C iterations, the target Q-network and target actor network are updated by  θˆQ = θ Q . (9.29) θˆμ = θμ The pseudo-code of the customized DRL algorithm is provided in Algorithm 9.1. As shown in the pseudo-code, a ε-greedy policy is applied to address the “explorationexploitation dilemma.” In addition to the ε-greedy policy, for exploration, a noise, denoted by a vector of Nπ = (Nπ,1 , Nπ,2 , . . . , Nπ,M ), is included by the actor to select maintenance actions, where Nπ,l is noise to be added to the selected maintenance action for component l. Hence, the maintenance actions for all the components in the kth break becomes π(Yk , Bk , k|μ, Q) + Nπ . In this chapter, noise is generated from the normal distribution as follows:

9.6 Illustrative Examples

181

Nπ,l ∼

Imax − I · N (0, σ 2 ), Imax

(9.30)

where I is the index of iterations; N (0, σ 2 ) is a normally distributed random number with a mean of zero and a variance of σ 2 . As the iteration evolves, the magnitude of the noise will gradually decrease. Algorithm 9.1 The pseudo-code of the customized DRL algorithm Inputs: I max : The maximum number of iterations; I : The iteration index; N D : The size of memory; Nm : The size of minibatch; K : The number of missions; C: The frequency of target network updating. 1: Randomly initialize the Q-network and actor network with parameters θ Q and θμ ; 2: Initialize the two target networks θˆQ = θ Q and θˆμ = θμ ; 3: Initialize the memory and minibatch; 4: for I = 1 to Imax do 5: Initialize X1 and A1 ; 6: Execute the first mission and obtain Y1 and B1 ; 7: for k = 1 to K − 1 do 8: if I ≤ 0.1Imax then 9: Randomly select maintenance actions; 10: else 11: Select maintenance actions π(Yk , Bk , k|μ, Q) + Nπ with the probability of 1 − ε, or 12: randomly select maintenance actions with the probability of ε; 13: Normalize the selected maintenance actions; 14: end if 15: R(k + 1) = R(Yk , Bk , ak ); 16: Simulate Yk+1 and Bk+1 ; 17: Store the transition record (Yk , Bk , k, ak , R(k + 1), Yk+1 , Bk+1 ) in memory D; 18: end for 19: Sample a random minibatch of Nm transition records from memory D; 20: Update Q-network and the actor network by Eqs. (9.26)-(9.28); 21: Update the two target networks for every C iterations: θˆQ = θ Q and θˆμ = θμ ; 22: end for Output: The target Q-network and target actor network.

9.6 Illustrative Examples 9.6.1 A Four-Component System The configuration of a four-component flow transmission system is delineated in Fig. 9.5. The structure function of the system is defined as: G(t) = min{G 1 (t), G 2 (t) + G 3 (t), G 4 (t)}.

182

9 Dynamic Selective Maintenance for Multi-state Systems …

Fig. 9.5 Configuration of a four-component system

2 1

4 3

If al,k > 0, the variable maintenance cost and time associated with a particular maintenance action al,k are respectively formulated as follows  −1 clrf · aNl,kL −1 Yl,k = 0 cl (al,k , Yl,k ) = rp al,k , (9.31) Yl,k = 1 cl · NL  t rf · al,k −1 tl (al,k , Yl,k ) = lrp Nal,kL −1 tl · NL

Yl,k = 0 . Yl,k = 1

(9.32)

The efficiency of a particular maintenance action al,k in the kth break is quantified by defining the age reduction factor bl,k of the maintenance action as a function of the maintenance cost of component l, and it is formulated as follows ⎧  1 ⎪ ⎨1 − cl (al,krf ,0) mlf Yl,k = 0 c , (9.33) bl,k =  l  1p ⎪ c (a ⎩1 − l l,krp ,1) ml Yl,k = 1 c l

rp

where clrf and cl are the corrective and preventive replacement costs for a failed component and a functioning component, respectively, namely, clrf = cl (NL , 0) and rp p p cl = cl (NL , 1). m lf (m lf > 0) and m l (m l > 0) are two characteristic parameters that can precisely determine the relation between the variable maintenance cost and the age reduction factor of a failed component l and a functioning component l, respectively. The studied system is planned to execute three missions with a maintenance budget Clim = $50,000 and a limited duration of Tlim = 3 days for each break. The pre-specified demand of each mission is 50 tons/h, and the time duration of each mission is L k = 10 days. Three optional maintenance actions are available to each component (i.e., NL = 2). The failure time of component l follows the Weibull distribution with the shape parameter βl and the scale parameter ηl . The parameter settings for each component, including the performance capacity, the maintenance cost, the parameters of the failure time distribution, and the initial effective ages and states of all the components are tabulated in Table 9.1. The dynamic selective maintenance optimization problem is resolved by the customized DRL algorithm. Four artificial neural networks, that is, the Q-network, target Q-network, actor network, and target actor network, are constructed, each of which has three hidden layers and each hidden layer consists of ten neurons. The hyperparameters of the customized DRL algorithm are listed in Table 9.2.

9.6 Illustrative Examples

183

Table 9.1 Parameter settings of each component (unit of time: day, unit of cost: $1,000, unit of performance capacities: tons/h) Component ID (l) 1 2 3 4 gl βl ηl p ml rp cl rp tl m lf clrf tlrf cl0 tl0 Al,1 X l,1

90 2.6 40 2.2 26 0.8 3.2 40 1 5 0.3 0 1

55 1.5 25 2.5 10 0.5 2.5 15 0.5 2 0.1 0 1

80 2.4 38 2.2 10 0.5 2.0 15 0.5 2 0.1 0 1

100 2.6 40 2.0 26 0.7 3.0 30 0.8 6 0.2 0 1

Table 9.2 Hyperparameters of the customized DRL algorithm Hyperparameters Value Maximum number of iterations (Imax ) Distance in the postprocess (L) Size of the memory (N D ) Size of the minibatch (Nm ) Parameter of the ε-greedy policy (ε) Noise (Nπ,l ) Update period of target networks (C)

1000 1 200 32 0.01 (Imax − I )/Imax · N (0, 0.52 ) 10

The runtime of training the neural networks is 752.50 seconds on a Laptop with an Intel Core i7-9750H 2.60 GHz CPU and 16 G RAM. As evaluated by the Qnetwork, the expected number of successes of the three missions is 2.6965, whereas the average number of successful missions is 2.7054 from a Monte Carlo simulation with 106 samples. Therefore, the Q-network can obtain an accurate estimation of the expected number of successes of the three missions. To further examine the effectiveness of the customized DRL algorithm, the dynamic programming algorithm is implemented to solve the problem and serves as the benchmark for comparison. As the dynamic programming algorithm can only tackle with a discrete state space, the effective age of each component is discretized with an interval of 2 days. Obviously, a more accurate result can be obtained for a shorter service age division, and more computational resources will be consumed.

184

9 Dynamic Selective Maintenance for Multi-state Systems …

Table 9.3 The comparison between the dynamic programming and the customized DRL algorithm, where the runtime is in seconds Methods N Relative error Runtime Dynamic programming The customized DRL algorithm

2.7130

–

2.6965

0.61%

19,149 753

The expected number of successes of the three missions of the dynamic programming algorithm and the customized DRL algorithm are presented in Table 9.3. As shown in Table 9.3, the maximum expected number of successes of the three missions resolved by the dynamic programming algorithm is 2.7130, which is close to the result from the customized DRL algorithm. However, the runtime of the dynamic programming algorithm (19,149 seconds) is much greater than the runtime of the customized DRL algorithm. Moreover, due to the “curse of dimensionality,” the memory space and the computational time consumption of the dynamic programming algorithm will significantly increase with respect to the number of discretized effective ages. The maximum expected number of successes of the three missions varies for both the maintenance budget Clim and the duration Tlim of each break. By changing the maintenance budget from 0 to $80,000 and the break duration from 0 to 3 days, the maximum expected number of the successes of the three missions was found by the customized DRL algorithm as shown in Fig. 9.6. The figure shows that with the increase of both the maintenance budget and the break duration, the maximum expected number of successes of the three missions gradually increases.

9.6.2 A Multi-state Coal Transportation System In this section, a coal transportation system with ten components is exemplified. The configuration of the system is shown in Fig. 9.7, where coal is conveyed through Feeder #1, Conveyor, Stacker-reclaimer, and Feeder #2 in succession. Based on the performance capacities of all the components, the performance capacity of the system is expressed as G(t) = min{G 1 (t) + G 2 (t) + G 3 (t), G 4 (t) + G 5 (t), G 6 (t) + G 7 (t) + G 8 (t), G 9 (t) + G 10 (t)}.

(9.34)

The failure time of each component follows a Weibull distribution. The parameters for all the components are tabulated in Table 9.4. The system is planned to perform four missions. The maintenance budget and the time duration of each break are set to be $150,000 and 3 days, respectively. The time duration of each mission is 10 days. During each break, eight optional maintenance actions can be selected for

185

Expected Number of The Successes of The Three Missions

9.6 Illustrative Examples

Fig. 9.6 The expected number of successes of the three missions versus the maintenance budget and the break durations Fig. 9.7 The configuration of a multi-state coal transportation system

1

6 9

4 2

7 5

10

3

8

Feeder #1 (Subsystem 1)

Conveyor Stacker-reclaimer Feeder #2 (Subsystem 2) (Subsystem 3) (Subsystem 4)

components (i.e., NL = 7). The maintenance cost and time associated with each available maintenance action are given by Eqs. (9.31)–(9.33). The demand of each mission is a random variable with the probability distribution presented in Table 9.5. In this example, the dynamic programming is inapplicable due to the large-scale action and state spaces, state spaces. Alternatively, the customized DRL algorithm is used to solve this problem. Each of the four networks, that is, the Q-network, target Q-network, actor network, and target actor network, has three hidden layers, and each hidden layer consists of ten neurons. The hyperparameters of the customized DRL

186

9 Dynamic Selective Maintenance for Multi-state Systems …

Table 9.4 Parameter settings of each component (unit of time: day, unit of cost: $1,000, unit of performance capacities: tons/h) Component ID (l) 1 2 3 4 5 6 7 8 9 10 gl βl ηl p ml rp cl rp tl m lf clrf tlrf cl0 tl0 Al,1 X l,1

55 1.5 25 2.5 15 0.13 2.50 25 0.25 3 0.03 0 1

80 2.4 38 2.2 20 0.20 2 32 0.31 4 0.03 0 1

120 1.6 28 2.6 25 0.20 3 35 0.33 3 0.030 0 1

Table 9.5 Mission demand Demand 120 (tons/h) Probability 0.1

90 2.6 40 2.2 20 0.12 3.20 35 0.32 5 0.04 0 1

145 1.8 28 1.8 25 0.21 4 34 0.34 2 0.02 0 1

70 2.4 34 2.4 15 0.14 3.20 20 0.19 3 0.03 0 1

95 2.5 26 2.8 24 0.20 3 30 0.27 6 0.05 0 1

80 2 28 2.3 20 0.17 2.80 35 0.31 5 0.05 0 1

95 1.2 26 2.0 18 0.18 2.50 28 0.26 3 0.04 0 1

90

60

30

10

0.25

0.35

0.2

0.1

130 1.4 35 2.5 20 0.20 2.80 35 0.32 6 0.05 0 1

Table 9.6 Hyperparameters of the customized DRL algorithm Hyperparameters Value Maximum number of iterations (Imax ) Distance in the postprocess (L) Size of the memory (N D ) Size of the minibatch (Nm ) Parameter of the ε-greedy policy (ε) Noise (Nπ,l ) Update period of target networks (C)

5000 1 200 32 0.01 (Imax − I )/Imax · N (0, 0.52 ) 50

algorithm are listed in Table 9.6. Estimated by the Q-network, the maximum expected number of successes of the four missions is 3.4801. By using the Monte Carlo simulation with 106 samples, the agent can successfully complete 3.4831 missions on average, which is close to the estimation of the Q-network. The runtime of training the agent is 4874 seconds on a Laptop with an Intel Core i7-9750H 2.60 GHz CPU and 16 G RAM.

References

187

The effectiveness of taking into account the imperfect maintenance is illustrated by comparing the results from the case where only three optimal maintenance actions, that is, doing nothing, minimal repair, and replacement, are optional. The maximum expected number of the successes of the four missions for this case was resolved by the customized DRL algorithm, and it is only 3.4057 which is inferior to the case of imperfect maintenance.

9.7 Closure For MSSs which are planned to perform multiple consecutive missions, a dynamic selective maintenance optimization was developed in this chapter. To maximize the success of future missions, components in a system can be restored to a better condition in the break between two adjacent missions, and multiple optional maintenance actions are available for each component. The resulting stochastic dynamic programming problem was formulated as a discrete-time finite-horizon MDP. To overcome the “curse of dimensionality” and cope with the uncountable state space, a DRL algorithm was customized in the framework of actor-critic algorithms. Subject to the constraints of maintenance resources, a postprocess was proposed for the actor to select the optimal maintenance actions in a large discrete action space. Demonstrated by two illustrative examples, the customized DRL algorithm can dynamically determine the optimal maintenance actions in a computationally efficient manner.

References 1. Dulac-Arnold G, Evans R, van Hasselt H, Sunehag P, Lillicrap T, Hunt J, Mann T, Weber T, Degris T, Coppin B (2015) Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679 2. Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 3. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533 4. Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M (2014) Deterministic policy gradient algorithms. In: International conference on machine learning, pp 387–395 5. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, Cambridge 6. Ushakov I (1986) A universal generating function. Sov J Comput Syst Sci 24(5):118–129

Appendix

Parameters for the Multi-state Coal Transportation System in Chapter 7

The state transition matrices of maintenance actions, the observation probability matrices of inspection actions, the time durations of the inspection and maintenance actions for each component in the coal transportation system are provided as following: Time durations: rep

rep

rep

tlins (1) = 1 h, tlins (2) = 2 h, tlins (3) = 3 h, tl (1) = 6 h, tl (2) = 11 h, and tl (3) = 15 h (l ∈ {1, 2, . . . , 14}). Observation probability matrices for components 1, 2 and 3: ⎡ ⎤ 0.7 0.2 0.1 0 ⎢ 0 0.7 0.2 0.1⎥ ⎥ P1ins (1) = P2ins (1) = P3ins (1) = ⎢ ⎣0.1 0.1 0.7 0.1⎦, 0 0.1 0.1 0.8 ⎡ ⎤ 0.85 0.15 0 0 ⎢ 0.1 0.8 0.1 0 ⎥ ⎥ P1ins (2) = P2ins (2) = P3ins (2) = ⎢ ⎣0.05 0.1 0.8 0.05⎦, 0 0.05 0.1 0.85 ⎡ ⎤ 1000 ⎢0 1 0 0⎥ ⎥ P1ins (3) = P2ins (3) = P3ins (3) = ⎢ ⎣0 0 1 0⎦. 0001 Observation probability matrices for components 4 and 5: ⎡ ⎤ 0.7 0.2 0.1 P4ins (1) = P5ins (1) = ⎣0.15 0.7 0.15⎦, 0 0.2 0.8

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Y. Liu et al., Selective Maintenance Modelling and Optimization, Springer Series in Reliability Engineering, https://doi.org/10.1007/978-3-031-17323-3

189

190

Appendix: Parameters for the Multi-state Coal Transportation …

⎡

⎤ 0.9 0.1 0 P4ins (2) = P5ins (2) = ⎣0.05 0.8 0.15⎦, 0 0.15 0.85 ⎡ ⎤ 100 P4ins (3) = P5ins (3) = ⎣0 1 0⎦. 001 Observation probability matrices for components 6, 7 and 8: ⎡ ⎤ 0.7 0.2 0.1 0 ⎢ 0.1 0.75 0.15 0 ⎥ ⎥ P6ins (1) = P7ins (1) = P8ins (1) = ⎢ ⎣0.15 0.15 0.7 0 ⎦, 0 0 0.25 0.75 ⎡ ⎤ 0.8 0.1 0.1 0 ⎢0.1 0.85 0.05 0 ⎥ ⎥ P6ins (2) = P7ins (2) = P8ins (2) = ⎢ ⎣ 0 0.1 0.75 0.15⎦, 0 0 0.15 0.85 ⎡ ⎤ 1000 ⎢0 1 0 0⎥ ⎥ P6ins (3) = P7ins (3) = P8ins (3) = ⎢ ⎣0 0 1 0⎦. 0001 Observation probability matrices for components 9 and 10: ⎡ ⎤ 0.75 0.15 0.1 0 ⎢0.15 0.7 0.05 0.1⎥ ins ⎥ P9ins (1) = P10 (1) = ⎢ ⎣ 0 0.1 0.7 0.2⎦, 0 0.05 0.15 0.8 ⎡ ⎤ 0.8 0.1 0.1 0 ⎢0.05 0.75 0.15 0.05⎥ ins ⎥ P9ins (2) = P10 (2) = ⎢ ⎣ 0 0.1 0.8 0.1 ⎦, 0 0 0.05 0.95 ⎡ ⎤ 1000 ⎢0 1 0 0⎥ ins ⎥ P9ins (3) = P10 (3) = ⎢ ⎣0 0 1 0⎦. 0001 Observation probability matrices for components 11, 12, 13 and 14: ⎡ ⎤ 0.7 0.2 0.05 0.05 0 ⎢ 0.1 0.7 0.15 0.05 0 ⎥ ⎢ ⎥ ins ins ins ins ⎥ P11 (1) = P12 (1) = P13 (1) = P14 (1) = ⎢ ⎢ 0.05 0.1 0.75 0.1 0 ⎥, ⎣ 0.05 0.05 0.1 0.7 0.1 ⎦ 0 0.05 0.05 0.15 0.75

Appendix: Parameters for the Multi-state Coal Transportation …

⎡

0.75 0.15 0.1 ⎢ 0.05 0.8 0.1 ⎢ ins ins ins ins 0.1 0.7 P11 (2) = P12 (2) = P13 (2) = P14 (2) = ⎢ ⎢0 ⎣0 0.05 0.1 0 0 0 ⎡ ⎤ 10000 ⎢0 1 0 0 0⎥ ⎢ ⎥ ins ins ins ins ⎥ P11 (3) = P12 (3) = P13 (3) = P14 (3) = ⎢ ⎢ 0 0 1 0 0 ⎥. ⎣0 0 0 1 0⎦ 00001

191

0 0 0.1 0.75 0.2

⎤ 0 0.05 ⎥ ⎥ 0.1 ⎥ ⎥, 0.1 ⎦ 0.8

State transition matrices of maintenance for components 1, 2 and 3: ⎡ ⎤ 0.5 0.3 0.2 0 ⎢ 0 0.4 0.5 0.1⎥ rep rep rep ⎥ P1 (1) = P2 (1) = P3 (1) = ⎢ ⎣ 0 0 0.4 0.6⎦, 0 0 0 1 ⎡ ⎤ 0.2 0.4 0.3 0.1 ⎢ 0 0.2 0.6 0.2⎥ rep rep rep ⎥ P1 (2) = P2 (2) = P3 (2) = ⎢ ⎣ 0 0 0.3 0.7⎦, 0 0 0 1 ⎡ ⎤ 0001 ⎢0 0 0 1⎥ rep rep rep ⎥ P1 (3) = P2 (3) = P3 (3) = ⎢ ⎣0 0 0 1⎦. 0001 State transition matrices of maintenance for components 4 and 5: ⎡ ⎤ 0.6 0.3 0.1 rep rep P4 (1) = P5 (1) = ⎣ 0 0.5 0.5⎦, 0 0 1 ⎡ ⎤ 0.3 0.5 0.2 rep rep P4 (2) = P5 (2) = ⎣ 0 0.35 0.65⎦, 0 0 1 ⎡ ⎤ 001 rep rep P4 (3) = P5 (3) = ⎣0 0 1⎦. 001 State transition matrices of maintenance for components 6, 7 and 8: ⎡ ⎤ 0.5 0.3 0.2 0 ⎢ 0 0.5 0.35 0.15⎥ rep rep rep ⎥ P6 (1) = P7 (1) = P8 (1) = ⎢ ⎣ 0 0 0.4 0.6 ⎦, 0 0 0 1

192

Appendix: Parameters for the Multi-state Coal Transportation …

⎡

0.3 0.45 0.2 ⎢ 0 0.2 0.8 rep rep rep P6 (2) = P7 (2) = P8 (2) = ⎢ ⎣ 0 0 0.3 0 0 0 ⎡ ⎤ 0001 ⎢0 0 0 1⎥ rep rep rep ⎥ P6 (3) = P7 (3) = P8 (3) = ⎢ ⎣0 0 0 1⎦. 0001

⎤ 0.05 0 ⎥ ⎥, 0.7 ⎦ 1

State transition matrices of maintenance for components 9 and 10: ⎡ ⎤ 0.4 0.4 0.2 0 ⎢ 0 0.45 0.4 0.15⎥ rep rep ⎥ P9 (1) = P10 (1) = ⎢ ⎣ 0 0 0.35 0.65⎦, 0 0 0 1 ⎡ ⎤ 0.3 0.4 0.2 0.1 ⎢ 0 0.3 0.5 0.2⎥ rep rep ⎥ P9 (2) = P10 (2) = ⎢ ⎣ 0 0 0.2 0.8⎦, 0 0 0 1 ⎡ ⎤ 0001 ⎢0 0 0 1⎥ rep rep ⎥ P9 (3) = P10 (3) = ⎢ ⎣0 0 0 1⎦. 0001 State transition matrices of maintenance for components 11, 12, 13 and 14: ⎡ ⎤ 0.4 0.3 0.2 0.1 0 ⎢ 0 0.4 0.4 0.2 0 ⎥ ⎢ ⎥ rep rep rep rep ⎥ P11 (1) = P12 (1) = P13 (1) = P14 (1) = ⎢ ⎢ 0 0 0.3 0.5 0.2 ⎥, ⎣ 0 0 0 0.4 0.6 ⎦ 0 0 0 0 1 ⎡ ⎤ 0.2 0.4 0.2 0.1 0.1 ⎢ 0 0.2 0.3 0.4 0.1 ⎥ ⎢ ⎥ rep rep rep rep ⎥ P11 (2) = P12 (2) = P13 (2) = P14 (2) = ⎢ ⎢ 0 0 0.1 0.6 0.3 ⎥, ⎣ 0 0 0 0.2 0.8 ⎦ 0 0 0 0 1 ⎡ ⎤ 00001 ⎢0 0 0 0 1⎥ ⎢ ⎥ rep rep rep rep ⎥ P11 (3) = P12 (3) = P13 (3) = P14 (3) = ⎢ ⎢ 0 0 0 0 1 ⎥. ⎣0 0 0 0 1⎦ 00001