Oxford Handbook of Medical Statistics [2nd Edition] 9780198743583, 9780191061257, 0191061255, 9780198743583, 0198743580

Table of contents :
Cover......Page 1
Series......Page 3
Oxford Handbook of
Medical Statistics......Page 4
Copyright......Page 5
Contents......Page 6
Foreword to the first edition......Page 8
Preface to the first edition......Page 10
Preface to the second edition......Page 12
Acknowledgements for the first edition......Page 14
Acknowledgements for the second edition......Page 16
Symbols and abbreviations......Page 18
1 How to use the Oxford Handbook of
Medical Statistics......Page 20
2 Research design......Page 28
3 Collecting and handling data......Page 126
4 Presenting research findings......Page 180
5 Choosing and using statistical software for
analysing data......Page 210
6 Summarizing data......Page 230
7 Probability and distributions......Page 262
8 Statistical tests......Page 300
9 Diagnostic studies......Page 408
10 Other statistical methods......Page 424
11 Analysing multiple observations per subject......Page 458
12 Analysing multiple variables per subject......Page 484
13 Meta-​analysis......Page 548
14 Bayesian statistics......Page 588
15 Glossary of terms......Page 616
Index......Page 626

Citation preview

 

Oxford Handbook of

Medical Statistics

ii

Published and forthcoming Oxford Handbooks Oxford Handbook for the Foundation Programme 4e Oxford Handbook of Acute Medicine 3e Oxford Handbook of Anaesthesia 4e Oxford Handbook of Cardiology 2e Oxford Handbook of Clinical and Healthcare Research Oxford Handbook of Clinical and Laboratory Investigation 4e Oxford Handbook of Clinical Dentistry 6e Oxford Handbook of Clinical Diagnosis 3e Oxford Handbook of Clinical Examination and Practical Skills 2e Oxford Handbook of Clinical Haematology 4e Oxford Handbook of Clinical Immunology and Allergy 3e Oxford Handbook of Clinical Medicine –​Mini Edition 9e Oxford Handbook of Clinical Medicine 10e Oxford Handbook of Clinical Pathology Oxford Handbook of Clinical Pharmacy 3e Oxford Handbook of Clinical Specialties 10e Oxford Handbook of Clinical Surgery 4e Oxford Handbook of Complementary Medicine Oxford Handbook of Critical Care 3e Oxford Handbook of Dental Patient Care Oxford Handbook of Dialysis 4e Oxford Handbook of Emergency Medicine 4e Oxford Handbook of Endocrinology and Diabetes 3e Oxford Handbook of ENT and Head and Neck Surgery 2e Oxford Handbook of Epidemiology for Clinicians Oxford Handbook of Expedition and Wilderness Medicine 2e Oxford Handbook of Forensic Medicine Oxford Handbook of Gastroenterology & Hepatology 2e Oxford Handbook of General Practice 4e Oxford Handbook of Genetics Oxford Handbook of Genitourinary Medicine, HIV, and Sexual Health 2e Oxford Handbook of Geriatric Medicine 3e

Oxford Handbook of Infectious Diseases and Microbiology 2e Oxford Handbook of Integrated Dental Biosciences 2e Oxford Handbook of Humanitarian Medicine Oxford Handbook of Key Clinical Evidence 2e Oxford Handbook of Medical Dermatology 2e Oxford Handbook of Medical Imaging Oxford Handbook of Medical Sciences 2e Oxford Handbook for Medical School Oxford Handbook of Medical Statistics Oxford Handbook of Neonatology 2e Oxford Handbook of Nephrology and Hypertension 2e Oxford Handbook of Neurology 2e Oxford Handbook of Nutrition and Dietetics 2e Oxford Handbook of Obstetrics and Gynaecology 3e Oxford Handbook of Occupational Health 2e Oxford Handbook of Oncology 3e Oxford Handbook of Operative Surgery 3e Oxford Handbook of Ophthalmology 4e Oxford Handbook of Oral and Maxillofacial Surgery 2e Oxford Handbook of Orthopaedics and Trauma Oxford Handbook of Paediatrics 2e Oxford Handbook of Pain Management Oxford Handbook of Palliative Care 3e Oxford Handbook of Practical Drug Therapy 2e Oxford Handbook of Pre-​Hospital Care Oxford Handbook of Psychiatry 3e Oxford Handbook of Public Health Practice 3e Oxford Handbook of Rehabilitation Medicine 3e Oxford Handbook of Reproductive Medicine & Family Planning 2e Oxford Handbook of Respiratory Medicine 3e Oxford Handbook of Rheumatology 4e Oxford Handbook of Sport and Exercise Medicine 2e Handbook of Surgical Consent Oxford Handbook of Tropical Medicine 4e Oxford Handbook of Urology 4e

 

Oxford Handbook of

Medical Statistics SECOND EDITION

Janet L. Peacock Emeritus Professor of Medical Statistics, King’s College London, UK Professor of Epidemiology, Dartmouth College, USA

Phil J. Peacock Specialty Registrar in Paediatric Emergency Medicine, Oxford University Hospitals, UK

1

iv

1 Great Clarendon Street, Oxford, OX2 6DP, United Kingdom Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries © Oxford University Press 2020 The moral rights of the authors have been asserted First Edition published in 2011 Second Edition published in 2020 Impression: 1 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by licence or under terms agreed with the appropriate reprographics rights organization. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above You must not circulate this work in any other form and you must impose this same condition on any acquirer Published in the United States of America by Oxford University Press 198 Madison Avenue, New York, NY 10016, United States of America British Library Cataloguing in Publication Data Data available Library of Congress Control Number: 2019953209 ISBN 978–​0–​19–​874358–​3 Printed and bound in China by C&C Offset Printing Co., Ltd. Oxford University Press makes no representation, express or implied, that the drug dosages in this book are correct. Readers must therefore always check the product information and clinical procedures with the most up-​to-​date published product information and data sheets provided by the manufacturers and the most recent codes of conduct and safety regulations. The authors and the publishers do not accept responsibility or legal liability for any errors in the text or for the misuse or misapplication of material in this work. Except where otherwise stated, drug dosages and recommendations are for the non-​pregnant adult who is not breast-​feeding Links to third party websites are provided by Oxford in good faith and for information only. Oxford disclaims any responsibility for the materials contained in any third party website referenced in this work.

 

v

Contents Foreword to the first edition  vii Preface to the first edition  ix Preface to the second edition  xi Acknowledgements for the first edition   xiii Acknowledgements for the second edition   xv Symbols and abbreviations  xvii 1 How to use the Oxford Handbook of Medical Statistics 2 Research design 3 Collecting and handling data 4 Presenting research findings 5 Choosing and using statistical software for analysing data 6 Summarizing data 7 Probability and distributions 8 Statistical tests 9 Diagnostic studies 10 Other statistical methods 11 Analysing multiple observations per subject 12 Analysing multiple variables per subject 13 Meta-​analysis 14 Bayesian statistics 15 Glossary of terms

Index  607

1 9 107 161

191 211 243 281 389 405 439 465 529 569 597

vi

 

vii

Foreword to the first edition All healthcare professionals want to provide safe and effective care to their patients. This means that everyone has to keep up with the speed of innovation and be in a position to apply the findings of new research. Historically individuals have tended to delegate the assessment of the quality of research to journal editors, the peer review system and guideline developers. However for many reasons this may not be sufficient. All professionals have to make a judgement call on whether the research findings or guideline recommendations that they are assessing are relevant to the patient in front of them. They will have to decide whether the drug trial designed to determine the short term safety and efficacy against placebo in a selected population in the USA is really relevant to the elderly, ethnically diverse population with multiple co-​morbidities facing them on a Friday afternoon. To make things even more complicated, many of the questions raised in day to day practice will never be answered by randomized controlled trials. So other methods need to be applied, all with their own challenges and potential biases. This means, like it or not, that a sound understanding of medical statistics is essential for all health professionals. Many doctors and medical students find statistics difficult to understand, and voice the need for a concise but thorough account of the subject. They plead for the statistical analysis to draw on real life situations and to use examples that they can understand. This book responds completely to that plea by providing an accessible format that allows individual topics to be easily found and understood. It takes the reader, not only through the theory of the underlying statistics, but also the practical steps to set up and interpret all the key research designs. The authors are an experienced academic medical statistician who has conducted many collaborative research studies and taught statistics to students and doctors (to a very high standard—​I should know—​she taught me), and a junior academic doctor who has published his own work. They have written a book that meets all the needs of doctors and students carrying out their own research, and for those appraising others’ research. Professor Peter Littlejohns Clinical and Public Health Director National Institute for Health and Clinical Excellence May 2010

vii

 

ix

Preface to the first edition To practice evidence-​based medicine, doctors need to be able to critically appraise research evidence. The majority of medical research involves quantitative methods and so it is essential to be able to understand and interpret statistics. In addition, many doctors conduct research which requires the use of statistics throughout the research process from design, to data collection and analysis, to interpretation and dissemination. Doctors study statistics at undergraduate and postgraduate level and there is an increasing move towards teaching programmes that are based on real clinical problems and real data. However, in our experience both as teacher and former medical student, courses do not always fully equip doctors to critically appraise research evidence or to conduct research and communicate the findings. We have written this book to help bridge this gap by covering the span of topics from research design, through collecting and handling data to simple and complex statistical analyses. We have aimed to be as comprehensive as possible in this handbook and so we have included all commonly-​used statistical methods as well as more advanced methods such as multifactorial regression, mixed models, GEEs, Bayesian models that are seen in medical papers. However, medical statistics is a broad and ever-​growing discipline and so it is inevitable that some newer or less commonly-​used topics have not found their way into this edition. For all methods we have provided clear guidance on when methods may be used and how the results of analyses are interpreted using examples from the medical literature and our own research. We have chosen to give formulae and worked examples for the ‘simpler’ methods as we know that the more mathematically minded readers may want to understand where the numbers come from. For those who do not wish to know, or who simply don’t have time, these can be ignored without loss of continuity. This book is written in the popular Oxford Handbook style with one topic per double-​page spread, providing easy access to discrete topics for busy doctors and students. Writing in this format has provided a challenge to us since many topics in medical statistics build on other topics and therefore assume prior knowledge. For this reason we have included many cross-​ references to other sections of the book so that other relevant information is clearly signposted. We have also included references for further reading where we believe that readers may wish to explore the topic in more detail. Writing any material in a punchy, brief style carries the danger of omitting material or ‘dumbing it down’. We have fought hard to avoid doing this, not excluding material but making the format both accessible and thorough. We hope that you agree that we have managed to make this work.

x

 

xi

Preface to the second edition To practice evidence-​based medicine, doctors need to critically appraise research evidence. The majority of medical research involves quantitative methods and so it is essential to be able to understand and interpret statistics. In addition, many doctors conduct research which requires the use of statistics throughout the research process—​from design, to data collection and analysis, and to the interpretation and dissemination. We wrote this book to help equip doctors and other healthcare professionals to critically appraise research evidence, to conduct research, and communicate the findings, or prepare for the statistics component of undergraduate and postgraduate examinations. The positive feedback we have received from friends, colleagues, and readers worldwide suggests we managed to do this and has inspired us to build upon the success of the first edition and try to make the second edition even better. In this edition, we have tried to bridge the gap between health professionals and statisticians by highlighting themes and principles common to each. Where appropriate we have introduced analytical methods from a clinician’s perspective, demonstrating how many statistical techniques mirror the approach taken by healthcare professionals in everyday practice. We have continued to provide real-​life clinical examples throughout the book to help demonstrate the practical use of statistics in healthcare research. While existing designs and statistical methods tend not to change, new designs, new methods, and new approaches are constantly coming into common use and so we have updated as needed. We have added material on translational medicine, early phase trials, observational ‘real’ data, and registers/​databases, and have updated and expanded sections on reporting guidelines. We have also expanded many statistical sections such as missing data and multiple imputation, propensity scores, and analysing costs data. We have also made some amendments to aid clarity, on the basis of feedback from readers. We hope you enjoy reading this new edition.

xii

 

xiii

Acknowledgements for the first edition So many people have helped us in so many ways with the design, writing, and publication of this book. Unfortunately it is inevitable that in naming people we may have missed some out, but we are incredibly grateful to everyone who has helped in any way. Our first thanks go to the OUP clinical reviewers, Tom Turmeziei, Kam Cheong Wong and Ryckie Wade, who provided invaluable feedback on the manuscript, especially in the early days, which helped us to shape the book. Our statistical colleagues, Jenny Freeman and Andrew Smith, gave us very thorough reviews of the draft script and their comments have made the book so much better. We wish to thank Diane Morrison who proof-​read the first draft for us to a very short deadline. Of course any errors which remain are our own. We are very grateful to the OUP editors, Catherine Barnes, Sara Chare, Liz Reeve, and Selby Marshall for agreeing with us that this book needed to be written and for helping us to make it happen, and to Kate Wanwimolruk for guiding us through the production process. We especially want to express our appreciation to Anna Winstanley for the tremendous encouragement and enthusiastic support she has given us throughout the project, as well as her patience when we didn’t always make our writing deadlines. We want to thank colleagues at Dartmouth College New Hampshire, USA, where we both have links, especially Margaret Karagas, who hosted Janet so generously to enable her to make a start writing the book. We thank our senior academic colleagues, Paul Roderick for his encouragement and support, and Martin Bland who has always been such a help and inspiration to us both. Finally we wish to say a huge thank you to our spouses, Eric and Becky, for all their helpful comments and suggestions during the writing and proof-​ reading process, but most of all for their continued confidence in us and graciousness when at times we neglected them so this book could be completed.

xvi

 

xv

Acknowledgements for the second edition This second edition builds upon the first, and we want to reiterate our thanks and appreciation to all who helped in creating the previous edition, including the supportive editorial team at Oxford University Press (OUP), and friends and colleagues who provided formal and informal reviews of the text. With this new edition, there are of course many new people to thank: Michael Hawkes, our commissioning editor at OUP, who has supported us through the writing process and has been patient when we missed deadlines. Our work colleagues in the University, NHS, and overseas have red our work, fed back, and encouraged us. We especially thank our families, Eric, Becky, Matthew, and Anna, for their love of books, their confidence in us, and understanding when spare time was spent working on this text. Janet and Phil December 2019

xvi

 

xvii

Symbols and abbreviations 0 caution M website E cross reference 2 important e advanced topic ∞ infinity α alpha β beta μ mu ρ rho τ tau χ chi ± plus or minus × multiply ° degree > greater than ≥ greater than or equal to < less than ≤ less than or equal to BPD bronchopulmonary dysplasia CI confidence interval DF degrees of freedom FRC functional residual capacity GEE generalized estimating equation

GP

general practitioner hazard ratio ICC intraclass correlation coefficient ITT intention to treat LR likelihood ratio MAR missing at random MCAR missing completely at random MCID minimum clinically important difference MNAR missing not at random NNT number needed to treat NPV negative predictive value OR odds ratio PPV positive predictive value QI quality improvement RCT randomized controlled trial ROC receiver operating characteristic RR relative risk or risk ratio SD standard deviation SE standard error HR

xvii

 

Chapter 1

How to use the Oxford Handbook of Medical Statistics How doctors think  2 Why does statistics matter to medicine?  4 Working together  6 Using this book  8

1

2

2

Chapter 1  

How to use OH Medical Statistics

How doctors think Diagnosing is at the heart of much of what doctors do in clinical practice. Whether diagnosing a new disease process, spotting a potential complication of an illness, or recognizing a possible side effect of a treatment, doctors are constantly obtaining information, analysing it, and using this to come up with a diagnosis. This is the same whether identifying a cancer which needs urgent treatment or reassuring a patient that they have a simple viral illness requiring no active treatment. In many clinical encounters, doctors are looking for evidence for or against a particular diagnosis. Consider a patient who attends their general practitioner (GP) complaining of headaches—​the doctor will ask about worrying symptoms, examine the patient for concerning physical signs, and perhaps carry out some simple tests (e.g. blood pressure). If all of these are normal—​that is, there is no evidence of any serious underlying disease process—​then the doctor may diagnose a simple headache and reassure the patient. Many doctors initially find statistics complicated or counterintuitive. It can cause confusion when statisticians talk about accepting or rejecting the ‘null hypothesis’. However, much of what medical statistics seeks to do mirrors the diagnostic process doctors undertake every day, both consciously and subconsciously. Imagine a clinical trial comparing two different treatments for bowel cancer. Statistical tests might be used to compare 5-​year survival between an existing and a new treatment. If we find ‘evidence’ that 5-​year survival is higher with the new treatment then we may conclude the new treatment is better and recommend its use. Conversely, if outcomes are similar between the two groups we may not have enough evidence to say one treatment is better than the other. This does not mean there isn’t a difference—​just that we have no evidence that there is a difference. This is identical to the previous clinical example—​if a patient reports a headache that wakes them from sleep, has been vomiting, and has papilloedema then this is evidence suggestive of raised intracranial pressure and the patient will be investigated or treated appropriately. In the absence of worrying symptoms, the patient may be reassured, but the doctor cannot say with absolute certainty that the headaches are not due to a sinister cause, just that they have not found any evidence suggestive of serious underlying pathology, and so they ‘reject’ that diagnosis.

 

How doctors think

3

4

4

Chapter 1  

How to use OH Medical Statistics

Why does statistics matter to medicine? There is an increasing drive towards the practice of evidence-​ based medicine—​ that is, making sure wherever possible doctors and other healthcare professionals are managing patients in a way which has been proven to be effective. Many doctors and other healthcare professionals will get involved in the collection and analysis of data in some form—​whether in carrying out an audit of practice within a department or taking part in a local research study. An understanding of medical statistics is necessary to do this. In order to practise evidence-​based medicine, clinicians need to not only read published research papers, but also be able to assess the evidence themselves and decide whether or not to change their practice on the basis of the evidence presented. Whether reading a journal article, looking at a research poster, or listening to a presentation at a conference, an understanding of medical statistics is essential to critically appraise the research that has been carried out, assess whether the results justify the conclusions, and decide what (if any) change to current practice is needed. The need for an understanding of statistics has been recognized by the UK medical Royal Colleges, and a basic knowledge of medical statistics is expected in Membership examinations. An extract from the Royal College of Paediatrics and Child Health examination syllabi is presented in Box 1.1 as an example.

 

Why does statistics matter to medicine?

Box 1.1  Rationale for understanding statistics in medicine Theory and science • Understand research methodologies • Understand the principles of statistical testing, evidence-​based medicine, its limitations, and applications in practice • To be able to recognize appropriate statistical testing and to choose the correct test in context • Know the principles of clinical and research governance • Know the principles of screening in research and clinical practice Applied knowledge in practice • To be able to apply evidence-​based medicine to clinical practice • To be able to interpret a research paper or systematic review appropriately Source: data from Royal College of Paediatrics and Child Health training curriculum: Royal College of Paediatric and Child Health 2018: MRCPCH Theory Examination Syllabi. https://​www. rcpch.ac.uk/​sites/​default/​files/​2018-​08/​mrcpch_​theory_​examination_​syllabi_​v1.pdf

5

6

6

Chapter 1  

How to use OH Medical Statistics

Working together Introduction This book aims to give clinicians and researchers in healthcare sufficient understanding to critically review publications in the literature and to appreciate the details of how a research project is designed, conducted, analysed, and interpreted. It will also provide readers with the information required to carry out and analyse simple studies. However, the book is not intended or able to turn researchers into expert statisticians!

How can a statistician help? Medical statisticians (also known as biostatisticians) have expertise in design, data collection, analysis, and more. They tend to use ‘statistical thinking’ to probe and find the best solutions within research settings. The following list covers some of the things a medical statistician can give useful input on: • Clearly defining the aims and research question(s) of a study • Planning the study, including deciding on its design—​so the design fits the question and results are not biased • Calculating the number of subjects needed to get statistically robust and clinically meaningful results • Designing or advising on the data capture tool—​perhaps paper data forms, an electronic database, or both • Writing a plan for analysis of the study data • Analysing the data and writing up the results • Interpreting the findings

Get statistical advice early Most medical statisticians agree that it is best if researchers seek statistical advice early on when a new study or a new analysis is being planned. It can be difficult to analyse data or draw meaningful conclusions from a study when there has been no statistical input early on. Sometimes simple advice is all that is needed—​consultancy. Sometimes fuller involvement—​collaboration—​is worth considering. Finding a medical statistician is definitely beyond the scope of this book as they are in short supply! University medical schools will usually have a statistics team who may be available to advise. In our experience, the most effective partnerships happen when people collaborate together over several projects and spend time making it work well.

Consultancy versus collaboration Consultancy This is usually just for advice, typically a fixed-​time appointment, from 20 minutes to an hour. Here the statistician is acting like a GP, diagnosing the problem and using their skills to give advice but not giving any complicated treatment. That happens elsewhere and involves a firmer collaborative agreement that is usually costed and/​or funded. The following are the sorts of things that typically get covered in a statistical consultancy appointment: • First thoughts on a possible new study

 

Working together

• Simple sample size calculations • What statistical test/​analysis to do • Replying to reviewers’ comments on a submitted paper. Analysis is not usually possible in a consultancy setting unless it is very simple Collaboration This is a longer-​term relationship where the statistician is typically part of the study team throughout the study and takes responsibility for all statistical aspects including the planning, design, and data collection as well as final analysis. This is usually funded through a grant or costed agreement.

Challenges to effective working • Language/​vocabulary: both statistics and medicine are technical subjects with their own vocabulary, acronyms, and jargon. Statisticians and clinicians need to ensure they use clear and understandable terminology • Communication: as in all areas of medicine, good communication between team members is essential but can at times be challenging, particularly when time is pressured. Good communication takes time and effort but is fruitful in the end • Expectations: these can differ between individuals, for example, regarding timelines, degree of input from each, responsibilities, availability for meetings, costs/​funds, authorship, and academic credit

7

8

8

Chapter 1  

How to use OH Medical Statistics

Using this book Different users This book is written for clinicians, medical students, and other healthcare professionals who critique and undertake research in their field. The scope of the book is intentionally broad to include areas of statistics that doctors and other practitioners may come across in their own research or in reading papers written by others. Thus the scope far exceeds that required for trainee doctors but it is hoped that medical students will still enjoy using the book to access material that they need and keep the book for use in their later medical careers. The book is also suited to postgraduate students in medicine and public health where the statistics curriculum is usually extensive. Finally, the book is ideal for PhD students in medicine and health whose needs are broad and varying.

Symbols Important points We have used the symbol 2 to indicate points that we think are particularly important to note. Notes of caution The symbol 0 is used to indicate where we suggest the reader notes that caution is needed. Identifying advanced material In writing this book, we were reluctant to label material as ‘basic’ or ‘advanced’—​different people will want and need to know different things, and we hope that the book is written in a way which is accessible to all readers. We are aware, however, that some topics are more complex than others, and beyond the scope of an undergraduate medical degree course or general postgraduate medical training. While we have endeavoured to make these sections accessible to all, some readers may choose to skip these advanced topics. These can be identified by the symbol  e.

 

Chapter 2

9

Research design Introduction  10 Introduction to research  12 Research questions  14 Translational medicine  16 Interventional studies  18 Phases of clinical trials  20 Adaptive designs  22 Biomarker designs  23 Pilot and feasibility studies  24 Randomized controlled trials  26 Randomization in RCTs  28 Patient consent in research studies  30 Blinding in RCTs  32 RCTs: parallel groups and crossover designs  34 Zelen randomized consent design  36 Superiority and equivalence trials  40 Cluster trials  42 Intention-​to-​treat analysis  44 Case–​control studies  46 Cohort studies  50 Prognostic studies  54 Cross-​sectional studies  56 Case study and series  58 Deducing causal effects  60 Quality improvement  62 Designing a clinical audit  64

Data collection in audit  66 Research versus audit  68 Data collection: sources of data  70 Registers  72 Data collection: outcomes  74 Dichotomization of outcomes:  P values  76 Dichotomization of outcomes: sample size  78 Regression to the mean  80 Collecting additional data  82 Sampling strategies  84 Choosing a sample size  86 Sample size for estimation studies: means  88 Sample size for estimation studies: proportions  90 Sample size for comparative studies  92 Sample size for comparative studies: means  94 Sample size for comparative studies: proportions  96 Sample size calculations: further issues  98 e Sample size in cluster trials  100 Using a statistical program to do the calculations  102 Research study documents  104 Statistical analysis plan  106

10

10

Chapter 2  

Research design

Introduction It is important to understand the main issues involved in study design in order to be able to critically appraise existing work and to design new studies. In this chapter we describe the main features of the design of interventional and observational studies and the differences and similarities between research and audit. We discuss when a sample size calculation is needed, describe the main principles of the calculations, and outline the steps involved in preparing a study protocol. Most sections are illustrated with examples and we give particular attention to the statistical issues that arise in designing and appraising research.

 

Introduction

11

12

12

Chapter 2  

Research design

Introduction to research Engaging with research At any one time, a clinician or medical student who is engaging with quantitative research may be doing so for one or more of the following reasons: • To critically appraise research reported by others • To conduct primary research that aims to answer a specific question or questions, and thus generate new knowledge or extend existing knowledge • To gain research skills and experience, often as part of an educational programme • To test the feasibility of a particular research design or technique The following issues are important for all of  these: • What is the study question or aim? • What design is appropriate to answer the question(s)? • What statistics are appropriate for the study?

Conducting and appraising primary research Primary research requires rigorous methods so that the design, data, and analysis provide sound results that stand up to scrutiny and add to current knowledge. Similarly, when critically appraising research, it is important to have a solid understanding of good research methodology.

Conducting research as part of an educational programme When research is conducted purely for educational purposes, such as with a medical student project, the main purpose is not to generate new knowledge but instead to provide practical training in research that will equip the individual to conduct sound primary research at a later stage. It is important that, as far as possible, research projects conducted within an educational programme are carried out rigorously. However, since these research projects usually face constraints, such as a narrow time frame and a limited budget, it may not be possible to fully meet the high standards set for primary research. For example, it may not be possible to recruit sufficient subjects to satisfy standard sample size calculations in the time given for a student project. If the purpose of the research is truly educational and not primarily to further knowledge, and this is made clear in any reporting, then this is not a problem.

Publishing research conducted as part of an educational programme Although student projects are often limited in scope, they may be sufficiently novel and of a high enough standard to be published. This is to be encouraged to provide further experience of the publication process and to encourage high standards. For examples of student projects that have been published, see Peacock and Peacock (2006), Peacock et al. (2009), and Thomas et al. (2017).

 

Introduction to research

References Peacock PJ, Peacock JL. Emergency call work-​load, deprivation and population density: an investigation into ambulance services across England. J Public Health (Oxf ) 2006; 28:111–​15. Peacock PJ, Peters TJ, Peacock JL. How well do structured abstracts reflect the articles they summarize? Eur Sci Editing 2009; 35:3–​5. Thomas E, Peacock PJ, Bates SE. Variation in the management of SSRI-​exposed babies across England. BMJ Paediatr Open 2017; 1:e000060.

13

14

14

Chapter 2  

Research design

Research questions Introduction Research aims to establish new knowledge around a particular topic. The topic might arise out of the researcher’s own experience or interest, or from that of a mentor or senior, or it may be a topic commissioned by a funding body. Sometimes a research study follows on directly from a previous study, either conducted by the researcher themselves or another researcher, and on other occasions it may be a completely new topic. As the research idea grows, the researcher generates a specific question or set of questions that he/​she wants to pursue. It can be quite difficult to focus down on specific questions if the topic is broad and there are many things that are interesting to explore. The scope of the study will determine how many questions can be investigated—​an individual with no research funds may only be able to centre on a single question, whereas one with a funded programme of research can investigate a number of related questions. Even when a particular study investigates many questions, it is important that each question is tightly framed so that the right data can be collected and the appropriate analyses conducted. If questions are too vague or too general then the study will be difficult to design and may not ultimately be able to answer the real questions of interest.

Research questions These should be: • Specific with respect to time/​place/​subjects/​condition as appropriate • Answerable such that the relevant data are available or able to be collected • Novel in some sense so that the study either makes a contribution to knowledge or extends existing knowledge • Relevant to current medicine

Types of question Most questions fall into one or more of the following categories: • Descriptive, for example, incidence/​prevalence; trends/​patterns; opinion/​knowledge; life history of disease • Evaluative, for example, efficacy/​safety of treatments or preventive programmes; may be comparative • Explanatory, for example, causes of disease; mechanisms for observed processes or actions or events

 

Research questions

Examples • What is the prevalence of diabetes mellitus in the population? This is a simple descriptive study • How effective is influenza vaccination in the community-​based elderly? This is a comparative study, comparing individuals who had vaccines with those who did not • Does lowering blood pressure reduce the risk of coronary heart disease? This is an evaluative study, investigating the efficacy of lowering blood pressure • Is prognosis following stroke dependent on age at the time of the event? This is an observational study • Why does smoking increase the risk of heart disease? This is an explanatory study investigating the mechanism behind an observed relationship • What evidence is there for the effectiveness of antidepressants in treating depression? This study is a meta-​analysis of existing interventional studies

15

16

16

Chapter 2  

Research design

Translational medicine What is it? Translational medicine or translation research is often described as research that goes ‘from bench to bedside’ so that discoveries can be turned into treatments, devices, or programmes of care that improve patient health. It is sometimes described simply as ‘translating research into practice’ so that new treatments and new information lead to benefit for patients (Woolf  2008). Why is it important? Recent years have seen unprecedented advances in basic biomedical sciences including human genomics, other omics, stem cell biology, biomedical engineering, molecular biology, and immunology (Sung et al. 2003). These advances need translating into tangible benefits such as: • Patient benefit: to ensure health research leads to improved patient outcomes • Promote innovation: to drive innovation in laboratory and clinical sciences • Return on investment: to ensure public funding is value for money

The translational pipeline Figure 2.1 shows the five basic components of translational research displayed as a pipeline:  basic science discoveries (phase 0), leading to early testing in healthy individuals (phase 1), then in patients (phase 2), leading to full testing in patients (phase 3), and finally leading to the adoption of effective interventions into patient care (phase 4).

Challenges in translational medicine Blockages in the pipeline To maximize effectiveness and efficiency it is important to identify and remove blockages between phases. Particular problems arise where early discoveries do not get carried through to testing. A common problem arises in the difficulties in moving effective interventions through to healthcare practice in a timely manner (Sung et al. 2003). Implementing existing known best practice and information This is a problem for existing interventions that are known to be effective but are either not implemented at all or their implementation is incomplete—​some patients are given the best treatment and some are not. For example, it is well known that putting babies to sleep on their backs reduces the risk of cot death (Fleming et al. 1990), but many parents (and indeed some healthcare professionals) do not follow this practice. Interdisciplinary working One of the great benefits of translational medicine is the recognition of the importance of interdisciplinary working among professionals: life sciences, clinical sciences, social sciences, biostatistics, health psychology, health economics, and so on.

 

Translational medicine

Phase 0 Early discovery

Phase 1 Early trials in healthy individuals

Phase 2 Early trials in patients

Phase 3 Full trials in patients

Phase 4 Adoption into healthcare practice

Figure 2.1  The translational pipeline.

Quality of research Robust research is transparent and reproducible and yet there has been a growing recognition that poor quality and/​or inadequately reported research methods hampers this both in clinical sciences (Smith 2014) and life sciences (Editorial 2013; Masca et al. 2015).

References Editorial. Reducing our irreproducibility. Nature 2013; 496:398. Fleming PJ, Gilbert R, Azaz Y, Berry PJ, Rudd PT, Stewart A, Hall E. Interaction between bedding and sleeping position in the sudden infant death syndrome:  a population based case-​control study. BMJ 1990; 301:85–​9. Masca NG, Hensor EM, Cornelius VR, Buffa FM, Marriott HM, Eales JM, et al. RIPOSTE: a framework for improving the design and analysis of laboratory-​based research. Elife 2015; 4:e05519. Smith R. Medical research still a scandal. 2014. M http://​blogs.bmj.com/​bmj/​2014/​01/​31/​richard-​ smith-​medical-​research-​still-​a-​scandal/​. Sung NS, Crowley, Jr WF, Genel M, Salber P, Sandy L, Sherwood LM, et al. Central challenges facing the national clinical research enterprise. JAMA 2003; 289:1278–​87. Woolf SH. The meaning of translational research and why it matters. JAMA 2008; 299:211–​13.

17

18

18

Chapter 2  

Research design

Interventional studies Study designs Intervention studies test the effect of a treatment or programme of care. The purpose is usually to test for efficacy but in early drug trials, safety and dosage are established first (E see Phases of clinical trials, p. 20).

No control group • Preliminary drug trials investigating safety and tolerance are often uncontrolled

Control group • It is highly desirable to have a control or comparison group in efficacy studies to be able to demonstrate superiority or inferiority • For example, it may be useful to know that a new drug lowers blood pressure, but it is more important to know how it compares to medications already in common use, especially as existing drugs are likely to be cheaper

Historical controls • Patients given a new treatment are compared with patients who have already been treated with an existing treatment regimen and who at the time of testing the new treatment have already been treated, assessed, and discharged • The comparison of the treatment group and the control group is not concurrent and may be problematic as other factors change over time, such as hospital staff and patient mix • Interpretation is difficult—​it is impossible to be sure that any differences observed between the new treatment group and the control group are solely due to the treatments received

Randomization between intervention and control group • This is the best way to ensure comparisons are concurrent and unbiased (E see Randomization in RCTs, p. 28)

When randomization is not possible • It is hard to test the efficacy of a treatment that is widely used and accepted against no treatment or a placebo • For example, the use of adrenaline for cardiac arrest is generally accepted as effective. It would be difficult, if not impossible, to formally test this against a control treatment

Natural experiments • Individuals receive different interventions concurrently but in a non-​randomized  manner Example 1 The effect of the fluoridation of drinking water may involve a comparison of subjects in areas where the water is subject to natural, artificial, or no fluoridation. Subjects are not allocated to the different types of fluoridation—​ this is determined by where they live.

 

Interventional studies

Example 2 The effect of treatment may be compared in patients who choose conservative surgery for breast cancer rather than radical surgery. Patients are not randomized.

When intervention studies are unethical • It is not ethical to experiment on humans when the intervention is likely to cause harm • It is not ethical to test whether environmental agents cause harm, and so observational studies are used to determine effects • Natural experiments may allow a better comparison to be made of individuals who are exposed and unexposed than a cross-​sectional analysis. For example, before and after studies have been used to compare health status before and after the introduction of the smoking ban in public places in the USA and the UK (Eisner et al. 1998; Allwright et al. 2005). In this way, a reasonable assessment of the effect of passive smoke exposure was made

Design and analysis for non-​randomized studies and natural experiments • Collect as much data as possible on the subjects’ key characteristics • Use statistical analysis to adjust for these differences • Note that, even with statistical adjustment, there may still be differences between the groups that are unknown and so comparisons may still be biased. We probably won’t know • Interpretation of non-​randomized trials is difficult and firm conclusions are hard to draw (E see Deducing causal effects, p. 60)

References Allwright S, Paul G, Greiner B, Mullally BJ, Pursell L, Kelly A, et al. Legislation for smoke-​free workplaces and health of bar workers in Ireland: before and after study. BMJ 2005; 331:1117. Eisner MD, Smith AK, Blanc PD. Bartenders’ respiratory health after establishment of smoke-​free bars and taverns. JAMA 1998; 280:1909–​14.

19

20

20

Chapter 2  

Research design

Phases of clinical trials Introduction Clinical trials are usually conducted in phases, as depicted in the translational research pipeline (E see Figure 2.1, p. 17). Each phase has specific aims as described in the following sections. The ‘early’ trials do not aim to give a firm conclusion about the effects of the intervention but rather seek to see whether the intervention shows promise with respect to improving patient outcome, and whether there are any concerns about safety.

Phase 1 trials These are early trials where a new drug or treatment is tested on a small group of people. They usually set out to establish safety and/​or tolerance and may seek to determine the appropriate dose in drug trials where this is not known. There are usually no control subjects. Although these trials are small, the design may use complex statistical methods, particularly for dose-​finding studies that may use statistical modelling throughout the study to identify when the dose should change as patients go through, and to identify when the optimum dose can be established. The design of these studies is an emerging area of research. At the time of writing, the UK National Institute for Health Research (NIHR) Statistics Group (M http://​www.statistics-​group.nihr.ac.uk) includes a research section in early phase trials and has published recommendations for dose-​funding studies (Love et al. 2017). An example of a safety trial is given in Box 2.1 (Petrof et al. 2015). e A more complex phase 1 dose-​finding study is described in Chapter 14 (E see Bayesian methods in early phase trials: example, p. 585).

Phase 2 trials These are also early trials but typically use a larger sample size than the corresponding phase 1 trial. They are usually controlled with allocation to active or control intervention being randomized (E see Randomization in RCTs, p. 28). Their main aim is usually to assess effectiveness and safety. Like phase 1 trials, phase 2 trials are not designed to be definitive but rather to guide decision-​making as to whether a new intervention is sufficiently promising to warrant testing in a full trial. Sample size can be based on

Box 2.1  Example of a safety trial

A phase 1 trial tested a cell-​based therapy in ten patients with epidermolysis bullosa, a rare serious skin disorder where patients have very fragile skin. Most adverse events reported were considered to be due to the underlying condition and not the therapy. No adverse events required discontinuation of therapy. The conclusion was drawn that there was no evidence to suggest the therapy was harmful, although it was noted that the sample was very small. See Petrof G et al. Potential of systemic allogeneic mesenchymal stromal cell therapy for children with recessive dystrophic epidermolysis bullosa. J Invest Dermatol 2015; 135:2319–​21.

 

Phases of clinical trials

Box 2.2  Example of a phase 2 trial A randomized phase 2 trial compared open, robotic, and laparoscopic radical cystectomy in 60 patients. The main effectiveness outcomes were incidence of complications at 30 and 90 days. There was evidence for a difference in 30-​day complications between the three modes of surgery which was statistically significant. However, the trial provided little evidence that robotic surgery was harmful and there were suggestions that it was superior to open surgery. A full trial is needed to give a definitive answer. See Khan MS et al. A single-​centre early phase randomised controlled three-​arm trial of open, robotic, and laparoscopic radical cystectomy (CORAL). Eur Urol 2016; 69:613–​21.

probability methods (Piantadosi 2005)  or a power-​based method can be used with a power lower than the usual 80% or 90% (E see Sample size for comparative studies, p. 92). As with phase 1 trials, the designs of these trials can be complex. They tend to estimate the intervention effect with 95% confidence intervals but they do not do a significance test. An example is shown in Box 2.2 (Khan et al. 2016).

Phase 3 trials These trials are designed to be definitive, i.e. to be large enough to detect the smallest difference in outcome that is clinically meaningful. They should also be large enough to estimate the incidence of adverse effects with reasonable precision.

Phase 4 These are studies that are carried out after the previous three phases to collect information on side effects and adverse effects associated with long-​term  use.

References Khan MS, Gan C, Ahmed K, Ismail AF, Watkins J, Summers JA, et  al. A single-​centre early phase randomised controlled three-​arm trial of open, robotic, and laparoscopic radical cystectomy (CORAL), Eur Urol 2016; 69:613–​21. Love SB, Brown S, Weir CJ, Harbron C, Yap C, Gaschler-​Markefski B, et al. Embracing model-​based designs for dose-​finding trials. Br J Cancer 2017; 117:332–​9. Petrof G, Lwin SM, Martinez-​Queipo M, Abdul-​Wahab A, Tso S, Mellerio JE, et al. Potential of systemic allogeneic mesenchymal stromal cell therapy for children with recessive dystrophic epidermolysis bullosa. J Invest Dermatol 2015; 135:2319–​21. Piantadosi S. Clinical trials: a methodologic perspective. Chichester: Wiley, 2005.

21

2

22

Chapter 2  

Research design

Adaptive designs Introduction Adaptive design trials are clinical trials that change either the design, analysis, or both on the basis of the emerging data or outcomes. They aim to do one or more of the following: • Increase efficiency by reducing the number of patients included • Reduce the time required for the trial to reach a conclusion • Increase the likelihood of demonstrating an effect if one exists • Provide more useful information on dose-​response relationship

Early phase adaptive designs These aim to determine whether an intervention is safe and, if the intervention is a drug, what the best dose is. They seek to allocate a higher proportion of participants to treatments or doses that are effective and fewer to those that are not. Where the best dose is unknown they explore a range of doses in order to determine the maximum tolerated dose (MTD). The designs often involve complex statistics including Bayesian methods (E see Chapter 14), and there may be several possible acceptable designs (E see Phase 1 trials p. 20).

Phase 3 adaptive designs These seek to make pre-​planned changes to the future conduct of the trial on the basis of emerging data, while maintaining the statistical integrity of the conclusions. Examples of adapted designs include: • Designs that allow ‘seamless’ transition between phases 2 and 3 • Designs that permit sample size recalculation with either blinded or unblinded data • Group sequential designs that allow early stopping for efficacy, futility, or harm • Population enrichment designs that remove treatment groups or other subgroups in which the intervention is less effective As for early phase adaptive designs, these may be statistically complex including using Bayesian methods (E see Chapter 14).

Further reading Bhatt DL, Mehta C. Adaptive designs for clinical trials. N Engl J Med 2016; 375:65–​74. Piantadosi S. Clinical trials: a methodologic perspective. Chichester: Wiley, 2005.

 

Biomarker designs

Biomarker designs Introduction These are designs that seek to discover whether a specific patient characteristic, or biomarker, can be used to identify a subgroup of participants in which an intervention is more effective. Typically patients are randomized to either a biomarker-​led strategy of care or standard care. For the patients in the biomarker arm, their care is directed according to their biomarker status. An example is given of a biomarker trial in patients following kidney transplant in Box 2.3 (Dorling et al. 2014).

Box 2.3  Example of biomarker design The long-​term use of anti-​rejection drugs in patients who have received a transplant can lead to failure of the graft. In this multicentre UK study, patients who were more than 1  year post-​renal transplantation were randomized to ‘blinded’ or ‘unblinded’ arms before being screened for human leucocyte antigen (HLA) antibodies. In the ‘unblinded’ arm, the test results were revealed. Patients with antibodies (i.e. biomarker positive) had biomarker-​led care with their anti-​rejection drugs changed according to an optimal treatment protocol. Patients in the ‘blinded’ arm received standard care. The trial is ongoing at time of writing and will determine whether the biomarker-​led regimen is effective in preventing graft failure 3 years from screening. Full details of the design are given in the protocol paper, including the planned final analysis testing superiority in the biomarker-​positive group and non-​inferiority overall (E see Non-​inferiority designs, p.  40). This analysis aims to ensure that if the results do show a significantly reduced graft failure rate in biomarker-​ positive patients receiving (unblinded) biomarker-​led treatment, that this is not at the expense of a poorer outcome overall in the unblinded group. See Dorling, A et al. Can a combined screening/​treatment programme prevent premature failure of renal transplants due to chronic rejection in patients with HLA antibodies: study protocol for the multicentre randomised controlled OuTSMART trial. Trials 2014; 15:30.

Further reading Wason J, Marshall A, Dunn J, Stein RC, Stallard N. Adaptive designs for clinical trials assessing biomarker-​guided treatment strategies. Br J Cancer 2014; 110:1950–​7.

Reference Dorling A, Rebollo-​Mesa I, Hilton R, Peacock JL, Vaughan R, Gardner L, et  al. Can a combined screening/​treatment programme prevent premature failure of renal transplants due to chronic rejection in patients with HLA antibodies: study protocol for the multicentre randomised controlled OuTSMART trial. Trials 2014; 15:30.

23

24

24

Chapter 2  

Research design

Pilot and feasibility studies Introduction Pilot and feasibility studies are preliminary studies conducted in preparation for a full study. They aim to test the process and protocols to make sure that the study will run as planned and achieve its aims.

Overall study aims Definitions of pilot and feasibility studies vary but there is general agreement that: • Pilot studies are a small-​scale version of the intended full study • Feasibility studies test the practicalities of conducting the study

Key objectives of pilot and feasibility trials (See Lancaster et al. 2004; Lancaster 2015.) • Test the integrity of the study protocol for the future trial • Gain initial estimates for sample size calculations • Test data collection forms or questionnaires • Test the randomization procedure • Estimate rates of recruitment and consent • Determine the acceptability of the intervention • Identify the most appropriate primary outcome

What pilot and feasibility trials are not Pilot and feasibility studies are not designed (or powered) to test the effectiveness of an intervention or treatment. Effectiveness is tested in a full trial. Hence hypothesis tests are not the main focus and may not be needed at all. If any treatment comparisons are carried out then these are reported with confidence intervals but not P values. 0 Sometimes a trial is described as a pilot because it is small even though the aim is to test effectiveness. However, this is not a true pilot study.

Pilot or feasibility study? Exact definitions of pilot and feasibility studies vary and so to try to resolve this, Eldridge and colleagues (2016) have developed a framework for their definition. They used multiple methods among a wide range of trialists to reach the following consensus: • Pilot studies are a subset of feasibility studies; they are not mutually exclusive • A feasibility study asks whether something can be done, should we proceed with it, and if so, how? • A pilot study asks the same questions but also has a special design feature: in a pilot study, a future study, or part of a future study, is conducted on a smaller scale Eldridge and colleagues (2016) recommended that: • These studies should be identified using the term ‘pilot’ or ‘feasibility’ in the title or abstract of publications • Researchers should report the study objectives and methods related to feasibility • Researchers should clearly state that the study is in preparation for a future full trial designed to assess the effect of an intervention

 

Pilot and feasibility studies

Sample size for pilot and feasibility studies The sample size for these studies needs to be sufficient to answer the aims. Various authors have made recommendations, such as Julious (2005), but the important thing is that the sample size is justified appropriately.

Applying for grants for pilot and feasibility studies The Medical Research Council (2006) and NIHR (2016) have each stated their working definitions of pilot and feasibility trials. As these are slightly different, researchers should check their design fits with the funding stream they are applying to. The NIH sometimes requires a published pilot study prior to an application for funding a definitive randomized controlled trial (RCT).

References Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, Bond CM. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One 2016; 11:e0150205. Julious S. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat 2009; 4:287–​91. Lancaster GA. Pilot and feasibility studies come of age! Pilot Feasibility Stud 2015; 1:1. Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract 2004; 10:307–​12. Medical Research Council. Developing and evaluating complex interventions: new guidance. 2006. M http://​www.mrc.ac.uk/​complexinterventionsguidance. National Institute for Health Research. Pilot studies. 2016. M http://​www.nets.nihr.ac.uk/​glossary.

25

26

26

Chapter 2  

Research design

Randomized controlled trials Introduction An RCT is an intervention study in which subjects are randomly allocated to treatment options. RCTs are the accepted ‘gold standard’ of individual research studies. They provide sound evidence about treatment efficacy which is only bettered when several RCTs are pooled in a meta-​analysis.

Choice of comparison group • The choice of the comparison group affects how we interpret evidence from a trial • A comparison of an active agent with an inert substance or placebo is likely to give a more favourable result than comparison with another active agent • Comparison of an active agent against a placebo when an existing active agent is available is generally regarded as unethical (see Box 2.4 from the Declaration of Helsinki, item 32 (M http://​www.wma.net)) • For example, it would not be ethical to test a new anticholesterol drug against a placebo; any comparison of new therapy would have to be against the currently proven therapy, statins

Box 2.4   Placebos The benefits, risks, burdens and effectiveness of a new intervention must be tested against those of the best current proven intervention, except in the following circumstances: • The use of placebo, or no treatment, is acceptable in studies where no current proven intervention exists; or • Where for compelling and scientifically sound methodological reasons the use of placebo is necessary to determine the efficacy or safety of an intervention and the patients who receive placebo or no treatment will not be subject to any risk of serious or irreversible harm. Extreme care must be taken to avoid abuse of this option. (Declaration of Helsinki, item 32; M http://​www.wma.net)

 

Randomized controlled trials

Comparison with ‘usual care’ When an intervention is a programme of care, for example, an integrated care pathway for the management of stroke, it is common practice for the comparison group to receive the usual or standard care.

Declaration of Helsinki The Declaration of Helsinki was first developed in 1964 by the World Medical Association to provide guidance about ethical principles for research involving human subjects. It has had multiple revisions since, with the latest full version published in 2008 with additional protections for research in children added in 2012. Although not legally binding of itself, many of its principles are contained in laws governing research in individual countries, and the declaration is widely accepted as an authoritative document on human research ethics. The declaration addresses issues such as: • Duties of those conducting research involving humans • Importance of a research protocol • Research involving disadvantaged or vulnerable persons • Considering risks and benefits • Importance of informed consent • Maintaining confidentiality • Informing participants of the research findings The full 35-​point declaration is available online at M http://​www.wma.net.

27

28

28

Chapter 2  

Research design

Randomization in RCTs Why randomize? • Randomization ensures that the subjects’ characteristics do not affect which treatment they receive. The allocation to treatment is unbiased • In this way, the treatment groups are balanced by subject characteristics in the long run and differences between the groups in the trial outcome can be attributed as being caused by the treatments alone • This provides a fair test of efficacy for the treatments, which is not confounded by patient characteristics • Randomization makes blindness possible (E see Blinding in RCTs, p. 32)

Randomizing between treatment groups The usual way to do random allocation is by using a computer program based on random numbers. The random allocation process may work in two different ways: • The program is interactive and provides the allocation code for each patient as he/​she is entered into the trial. This may be a code which refers to a treatment to maintain blindness or if the treatment cannot be blinded (e.g. with a technology), it will be the name of the actual intervention • A computer-​generated list of sequential random allocations is produced and administered by someone who is independent of the team that is recruiting patients to the trial. In this way, there is no bias in recruitment or allocation. In drug trials, the pharmacy may conduct the randomization and provide numbered containers to which it holds the code, so that the researcher and the patient can be kept blind to the actual allocation

Audit trail It is important to have an audit trail of the recruitment and randomization process including keeping a log of the recruited patients. This information is needed for later reporting of the trial and assists with checking that the trial is being conducted according to the protocol.

Non-​random allocation 0 Alternate allocation, or a method based on patient identifiers such as hospital number or date of birth, are not random methods and are not recommended because they are open, and in the case of alternate allocation, predictable. These methods make blinding difficult and leave room for the researcher to change the allocation or recruit according to the treatment that is to be received (e.g. to give a sicker patient the new treatment).

Stratification for prognostic factors If there are important prognostic factors that need to be accounted for in a particular trial, the random allocation can be stratified so that the treatment groups are balanced for the prognostic factors. For example, in trials of treatment for heart disease, the random allocation may be stratified by gender so that there are similar numbers of men and women receiving each treatment.

 

RANDOMIZATION IN RCTs

Minimization Minimization is another method of allocating subjects to treatment groups while allowing for important prognostic factors (Pocock 1983; Altman and Bland 2005). The allocation takes place in a way that best maintains balance in these factors. At all stages of recruitment, the next patient is allocated to that treatment which minimizes the overall imbalance in prognostic factors. For a worked example, see Altman and Bland (2005) or Pocock (1983). ‘Minim’ software to do minimization is available free from Martin Bland’s website (M http://​www-​users.york.ac.uk/​~mb55/​guide/​minim.htm).

Blocking Blocking is used to ensure that the number of subjects in each group is very similar at any time during the trial. The random allocation is determined in discrete groups or blocks so that within each block there are equal numbers of subjects allocated to each treatment. Example using blocks of size 4 and two treatments A, B There are six possible blocks or arrangements of A and B, which give equal numbers of As and Bs: AABB; ABAB; BBAA; BABA; ABBA; BAAB. We randomly choose blocks, so say the first two chosen blocks are: BBAA; AABB. Then the first eight subjects will be allocated B, B, A, A, A, A, B, B. The total subjects on A and B as subjects 1 to 8 are recruited will be: (0,1), (0,2), (1,2) (2,2), (3,2), (4,2), (4,3), (4,4). Hence, at all times, the total on A and the total on B will only differ by a maximum of 2 and so the treatment numbers will always be very similar and the numbers will be exactly balanced after every fourth subject is randomized. Further extensions of ‘blocking’ are available with a mixture of different block sizes, whereby random combinations of blocks are selected.

Further reading Altman DG, Bland JM. Statistics notes: treatment allocation in controlled trials: why randomise? BMJ 1999; 318:1209. Altman DG, Bland JM. Statistics notes: how to randomise. BMJ 1999; 319:703–​4.

References Altman DG, Bland JM. Treatment allocation by minimisation. BMJ 2005; 330:843. Pocock SJ. Clinical trials: a practical approach. Chichester: Wiley, 1983.

29

30

30

Chapter 2  

Research design

Patient consent in research studies Introduction It is generally accepted that all subjects participating in research give their prior informed consent. The Declaration of Helsinki (item 24; M http://​ www.wma.net) states the following: In medical research involving competent human subjects, each potential subject must be adequately informed of the aims, methods, sources of funding, any possible conflicts of interest, institutional affiliations of the researcher, the anticipated benefits and potential risks of the study and the discomfort it may entail, and any other relevant aspects of the study. The potential subject must be informed of the right to refuse to participate in the study or to withdraw consent to participate at any time without reprisal. Special attention should be given to the specific information needs of individual potential subjects as well as to the methods used to deliver the information. After ensuring that the potential subject has understood the information, the physician or another appropriately qualified individual must then seek the potential subject’s freely-​given informed consent, preferably in writing. If the consent cannot be expressed in writing, the non-​written consent must be formally documented and witnessed. (Declaration of Helsinki, item 24; M http://​www.wma.net)

Informed consent • This requires giving patients a detailed description of the study aims, what participation is required, and any risks they may be exposed to • Consent must be voluntary • Consent is confirmed in writing and a cooling-​off period is provided to allow subjects to change their minds • Consent must be obtained for all patients recruited to an RCT • Giving or withholding consent must not affect patient treatment or access to services • For questionnaire surveys, consent is often implicit if the subject returns the questionnaire where it is clear in the accompanying information that participation is voluntary • Consent may not be required if the study involves anonymized analyses of patient data only

When consent may be withheld In some situations, obtaining patient consent to a study may be problematic.

 

Patient consent in research studies

Example 1 For example, where the intervention is so desirable that patients would not want to risk being randomized to the control group. This is particularly so when it is not possible to mask the intervention such as where the intervention is a programme of care and the control treatment is ‘usual care’. Subjects may not be willing to enter the trial and risk not getting the new intervention, or they may enter the trial but drop out if they are allocated to the control group. One solution in situations like these is for the researcher to decide in advance to offer the intervention to all control group subjects after the trial has finished, assuming that the intervention proves to be effective. For example, in exercise therapy trials, control group subjects may be offered the exercise regimen at the end of the trial if it has been shown to work. Such an approach is stated in the Declaration of Helsinki (item 33; M http://​ www.wma.net) (Box 2.5) and would need to be costed into the trial.

Box 2.5  Post-​trial intervention At the conclusion of the study, patients entered into the study are entitled to be informed about the outcome of the study and to share any benefits that result from it, for example, access to interventions identified as beneficial in the study or to other appropriate care or benefits. (Declaration of Helsinki, item 33; M http://​www.wma.net)

Example 2 Patients may be reluctant to agree to enter a trial of a new therapy when there is an existing treatment which is known to work. In such situations, assuming that there is equipoise, it is the responsibility of the clinician to explain the study clearly enough to allow the patient to make an informed choice of whether or not to take part. Further discussion of patient consent is beyond the scope of this book but the General Medical Council UK website has detailed guidance (M http://​www.gmc-​uk.org/​ethical-​guidance/​ethical-​guidance-​for-​doctors/​ consent).

31

32

32

Chapter 2  

Research design

Blinding in RCTs Concealing the allocation • Blinding is when the treatment allocation is concealed from either the subject or assessor or both • It is done to avoid conscious or unconscious bias in reported outcomes • A trial is double blind if neither the subject nor the assessor knows which treatment is being given • A trial is single blind if the treatment allocation is concealed from either the subject or the assessor but not both • 2 Note that randomization makes blinding possible and is its most important role Examples A subject who knows that he is receiving a new treatment for pain which he expects to be beneficial may perceive or actually feel less pain than he would do if he thought he was receiving the old treatment. An assessor who knows that a subject is receiving the new steroid treatment for chronic obstructive pulmonary disease, which he expects to work better than the old one, may tend to round up measurements of lung function. If the treatment allocation is concealed, then both the patient and assessor will make unbiased assessments of the effects of the treatments being tested.

Placebo • A placebo is an inert treatment that is indistinguishable from the active treatment • In drug trials it is often possible to use a placebo drug for the control which looks and tastes exactly like the active drug • The use of a placebo makes it possible for both the subject and assessor to be blinded

When blinding is not possible In some situations blinding is not possible, such as in trials of technologies where concealment is impossible. For example, in trials comparing different types of ventilator, it is impossible to blind the clinician, and similarly in trials of surgery versus chemotherapy. Possible solutions are the use of sham treatments, such as sham surgery, but this may not be ethically acceptable. Trials of the effectiveness of acupuncture have used sham acupuncture for the control group to maintain blindness (Scharf et al. 2006) and trials involving injections sometimes use saline injections in the control group, although this may raise ethical objections. Sometimes ingenuity can be employed to address blindness, such as in a trial of electrical stimulation in non-​healing fractures, where patients in the control group also received an electric current of non-​therapeutic power but sufficient to interfere with radio in the same way as the active coil did (Simonis et al. 2003).

 

BLINDING IN RCTs

Double placebo (double dummy) If a trial involves two active treatments that have different modes of treatment, for example, a tablet versus a cream, a double placebo (‘double dummy’) can be used whereby each patient receives two treatments. In the example given, patients would receive either the active tablet plus a placebo cream, or a placebo tablet plus an active cream. A double dummy can also be used if the timing of treatment is different for the two drugs being tested, for example, if one drug is given once a day in the morning (drug A) and the other is given twice a day, morning and evening (drug B). In this case, one group of patients would receive the active drug A in the morning and placebo drug B both morning and evening and the other would receive the placebo drug A in the morning and active drug B both morning and evening.

Active placebo Trials may use an active placebo, which mimics the treatment in some way to maintain blindness. For example, some treatments give patients a dry mouth and so the presence or absence of this side effect may indicate to the patient which treatment they are on. Example In a trial of dextromethorphan and memantine to treat neuropathic pain, patients in the placebo group were given low-​dose lorazepam to mimic the side effects of dextromethorphan and memantine and thus help conceal the treatment allocation (Sang et al. 2002).

References Sang CN, Booher S, Gilron I, Parada S, Max MB. Dextromethorphan and memantine in painful diabetic neuropathy and postherpetic neuralgia:  efficacy and dose-​response trials. Anesthesiology 2002; 96:1053–​61. Scharf HP, Mansmann U, Streitberger K, Witte S, Kramer J, Maier C, et al. Acupuncture and knee osteoarthritis: a three-​armed randomized trial. Ann Intern Med 2006; 145:12–​20. Simonis RB, Parnell EJ, Ray PS, Peacock JL. Electrical treatment of tibial non-​union: a prospective, randomised, double-​blind trial. Injury 2003; 34:357–​62.

33

34

34

Chapter 2  

Research design

RCTs: parallel groups and crossover designs Two or more parallel groups • This is a trial with a head-​to-​head comparison of two or more treatments • Subjects are allocated at random to a single treatment or a single treatment programme for the duration of the trial • Usually, the aim is to allocate equal numbers to each trial, although unequal allocation is possible • The groups are independent of each other

Crossover trials • This involves a single group study where each patient receives two or more treatments in turn • Each patient therefore acts as their own control and comparisons of treatments are made within patients • The two or more treatments are given to each patient in random order • Crossover trials are useful for chronic conditions such as pain relief in long-​term illness or the control of high blood pressure where the outcome can be assessed relatively quickly • They may not be feasible for treatments for short-​term illnesses or acute conditions that once treated are cured, for example, antibiotics for infections • It is important to avoid the carry-​over effect of one treatment into the period in which the next treatment is allocated. This is usually achieved by having a gap or washout period between treatments to prevent there being any carry-​over effects of the first treatment when the next treatment starts • The simplest design is a two-​treatment comparison in which each patient receives each of the two treatments in random order with a washout period of non-​treatment in between • There are some particular statistical issues that may arise in crossover trials which are related to the washout period and carry-​over effects, and how and whether to include patients who do not complete both periods. Senn (2002) gives a full discussion of the issues and possible solutions.

Example: crossover trial A randomized, double-​blind, placebo-​controlled crossover study tested the effectiveness of valproic acid to relieve pain in patients with painful polyneuropathy. Thirty-​one patients were randomized to receive either valproic acid (1500 mg daily) and then placebo, or placebo followed by valproic acid. Each treatment lasted for 4 weeks. No significant difference in total pain or individual pain rating was found between treatment periods on valproic acid and placebo (total pain (median) = 5 in the valproic acid period versus 6 in the placebo period; P = 0.24) (Otto et al. 2004).

 

RCTs: PARALLEL GROUPS AND CROSSOVER DESIGNS

Choice of design: parallel group or crossover? Advantages of parallel group designs • The comparison of the treatments takes place concurrently • Can be used for any condition, especially an acute condition which is cured or self-​limiting such as an infection • No problem of carry-​over effects Disadvantages of parallel group designs • The comparison is between patients and so usually needs a bigger sample size than the equivalent crossover trial Advantages of crossover designs • Treatments are compared within patients and so differences between patients are accounted for explicitly • Usually need fewer subjects than the equivalent parallel group trials • Can be used to test treatments for chronic conditions Disadvantages of crossover designs • Cannot be used for many acute illnesses • Carry-​over effects need to be controlled • Likely to take longer than the equivalent parallel designs • Statistical analysis is more complicated if subjects do not complete all periods

References Otto M, Bach FW, Jensen TS, Sindrup SH. Valproic acid has no effect on pain in polyneuropathy: a randomized, controlled trial. Neurology 2004; 62:285–​8. Senn S. Cross-​over trials in clinical research. Chichester: Wiley, 2002.

35

36

36

Chapter 2  

Research design

Zelen randomized consent design Introduction This design can be used when comparing a new treatment programme with usual care and attempts to address problems with patient consent (E see Patient consent in research studies, p. 30).

Allocation to treatments • Subjects are randomly allocated to treatment or usual care • Only those subjects who are allocated to treatment are invited to participate and to give their consent • Subjects allocated to usual care (control) are not asked to give their consent • Among the treatment group, some subjects will refuse and so this design results in three treatment groups (Zelen 1979, 1990): 1.  Usual care (allocated) 2. Intervention 3.  Usual care (but allocated to intervention) • The analysis is performed with patients analysed in the original randomized groups, that is, 1 versus 2 + 3 (E see Intention-​to-​treat analysis, p. 44)

Double randomized consent • Patients are randomized to intervention or control and then their consent is sought, whichever group they are allocated to • Patients are allowed to choose either the treatment they are allocated to or the other treatment • The analysis is performed with patients analysed in the original randomized groups, whichever treatment they chose or received • (E see Intention-​to-​treat analysis, p. 44)

Justification The single randomized Zelen design has been criticized as being unethical since some subjects are not informed that they are in a trial. However, it is generally agreed that some trials could not take place without the use of this design because in some situations patients would not wish to take part if they were allocated to the control group. It could be argued that this therefore justifies its use (Torgerson and Roland 1998).

 

Zelen randomized consent design

Advantages of Zelen’s single randomized design • It avoids patient refusal at the outset due to the possibility of their being allocated to the control group • It avoids later withdrawal in subjects who initially consent but then withdraw when they are allocated to the control group • It allows a new and potentially desirable programme to be evaluated rigorously in a randomized trial Disadvantages of Zelen’s single randomized design • Patients in the control group do not know they are in a trial, which has ethical implications • The design leads to three groups and will lead to bias if subjects are not analysed in the group to which they were allocated irrespective of the treatment they chose or received • Will only work if the data required are routinely collected, otherwise no data will be available for the control group • It is less efficient statistically than a straightforward two-​group design since, when subjects choose not to accept the allocated treatment, the true treatment effect is diluted Advantages of Zelen’s double randomized design • It randomizes patients but allows them to choose which treatment they prefer • It avoids the ethical problems of not seeking consent for patients allocated to the control group • It thus allows a new and potentially desirable programme to be evaluated rigorously in a randomized trial Disadvantages of Zelen’s double randomized design • It almost inevitably leads to severe contamination of the groups since some patients will choose the opposite treatment to which they have been allocated • It is less efficient statistically than a straightforward two-​group design since, when subjects choose not to accept the allocated treatment, the true treatment effect is diluted

References Torgerson DJ, Roland M. Understanding controlled trials: what is Zelen’s design? BMJ 1998; 316:606. Zelen M. A new design for randomized clinical trials. N Engl J Med 1979; 300:1242–​5. Zelen M. Randomized consent designs for clinical trials: an update. Stat Med 1990; 9:645–​56.

37

38

38

Chapter 2  

Research design

Zelen randomized consent design (continued) Example In this trial the investigators sought to determine whether an intervention using postcards could reduce the number of episodes of repeated deliberate self-​poisoning (Carter et al. 2005). Potentially eligible patients were identified from a database of patients who had presented at the emergency department with poisoning. They were randomized to either receive the postcard intervention or to the control group, but the allocation was hidden until recruitment. After randomization, patients were screened for eligibility and consent was sought from those randomized to the postcard intervention. Figure 2.2 shows the flow chart. The primary analysis was by intention to treat comparing the proportions with repeated attendance at the emergency department with self-​poisoning in the postcard versus the control group, pooling those who did and did not consent to the intervention. The primary analysis was not statistically significant but a secondary outcome, the number of repetitions, was significantly reduced in the postcard group. The design necessarily meant that some intervention patients did not receive the intervention through withheld consent. This is likely to have to have reduced the difference between the groups but the authors argued that this design was suited to this study and clinical population.

 

Zelen randomized consent design (continued)

Randomisation (hidden until recruitment) Eligibility assessed (n = 922)

Eligible (n = 772)

Treatment as usual group (n = 394)

Postcard plus treatment as usual group (n = 378)

Ineligible (n = 150): Unable to consent (n = 60) Risk to interviewer (n = 55) No fixed abode (n = 15) Criteria incorrectly applied (n = 15) Language problem (n = 2) Other (n = 3)

Consented to intervention (n = 302) Did not consent to intervention (n = 76)

Follow-up at 12 months (n = 394)

Follow-up at 12 months (n = 378)

Figure 2.2  Flow chart of participants through a Zelen design trial. Reproduced from BMJ, Carter GL (2005) “Postcards from the EDge project: randomised controlled trial of an intervention using postcards to reduce repetition of hospital treated deliberate self-​ poisoning”, 331(7520): 374–​375 with permission from BMJ Publishing Group Ltd.

Reference Carter GL. Postcards from the Edge project:  randomised controlled trial of an intervention using postcards to reduce repetition of hospital treated deliberate self poisoning. BMJ 2005; 331:375–​5.

39

40

40

Chapter 2  

Research design

Superiority and equivalence trials Superiority trials • These seek to establish that one treatment is better than another • When the trial is designed, the sample size is set so that there is high statistical power to detect a clinically meaningful difference between the two treatments • For such a trial a statistically significant result is interpreted as showing that one treatment is more effective than the other

Equivalence trials • These seek to test if a new treatment is similar in effectiveness to an existing treatment • They are appropriate if the new treatment has certain benefits such as fewer side effects, being easier to use, or being cheaper • The trial is designed to be able to demonstrate that, within given acceptable limits, the two treatments are equally effective • Equivalence is a pre-​set maximum difference between treatments such that, if the observed difference is less than this, the two treatments are regarded as equivalent • The limits of equivalence need to be set to be appropriate clinically • The tighter the limits of equivalence are set, the larger the sample size that will be required • If the condition under investigation is serious, then tighter limits for equivalence are likely to be needed than if the condition is less serious • The calculated sample size tends to be bigger for equivalence trials than superiority trials

Non-​inferiority trials • This is a special case of the equivalence trial where the researchers only want to establish if a new treatment is no worse than an existing treatment • In this situation the analysis is by nature one-​sided (E see Tests of statistical significance, p. 290)

Practicalities • In general, the design and implementation of equivalence trials is less straightforward than superiority trials • If patients are lost to follow-​up or fail to comply with the trial protocol, then any differences between the treatments is likely to be reduced and so equivalence may be incorrectly inferred • It is especially important that equivalence trials need very strict management and good patient follow-​up to minimize these problems • It is often helpful to include a secondary analysis where subjects are analysed according to the treatment they actually received, ‘per protocol’ analysis

 

Superiority and equivalence trials

Examples • Is atorvastatin more effective at reducing blood cholesterol levels than simvastatin? This is an example of a superiority trial • Are angiotensin receptor blockers (e.g. valsartan) as effective at reducing blood pressure in hypertensive patients as angiotensin-​ converting enzyme inhibitors (e.g. ramipril)? This is an example of an equivalence trial • Does biomarker-​led care reduce the risk of graft failure in renal transplant patients? This trial uses both superiority of biomarker-​led care in biomarker-​positive patients and non-​inferiority of screening for biomarker status overall. For further details, E see Biomarker designs, p. 23, Dorling et al. (2014), and the following text

Superiority and equivalence • It is important to distinguish between superiority and equivalence when designing a trial • The choice depends on the purpose of the trial • A trial designed for one purpose may not be able to adequately fulfil the other • In general, equivalence trials tend to need larger samples • A trial designed to test superiority is unlikely to be able to draw the firm conclusion that two treatments which are not significantly different can be regarded as equivalent For further details of equivalence trials, see the books on clinical trials by Matthews (2006) and Girling and colleagues (2003).

References Dorling A, Rebollo-​Mesa I, Hilton R, Peacock JL, Vaughn R, Gardner L, et  al. Can a combined screening/​treatment programme prevent premature failure of renal transplants due to chronic rejection in patients with HLA antibodies: study protocol for the multicenter randomised controlled OuTSMART trial. Trials 2014; 15:30. Girling DJ, Parmar MKB, Stenning SP, Stephens RJ, Stewart LA. Clinical trials in cancer principles and practice. Oxford: Oxford University Press, 2003. Matthews JNS. Introduction to randomized controlled clinical trials, 2nd ed. Boca Raton, FL: Chapman & Hall/​CRC, 2006.

41

42

42

Chapter 2  

Research design

Cluster trials Introduction In most randomized trials, individual participants are allocated to an intervention. In a cluster randomized trial, a group of individuals, or ‘cluster’, are allocated to all receive the same intervention. So, if there are two interventions A and B, some clusters will receive A and others will receive B. Cluster trials are sometimes used in primary care studies where it would be difficult to allocate individual patients in a general practice to different treatments. They are also sometimes used in hospital studies where, for example, a whole ward or clinic is the ‘cluster’.

Why randomize clusters? To avoid contamination When individuals are in a natural grouping such as a general practice, they may have contact with other patients in the trial who receive the same or a different intervention. This might affect their compliance and response to the intervention. Feasibility Some treatments are naturally administered to groups of individuals, for example, if the intervention is an exercise class. Others would be difficult to administer to individuals simply because of the complexity of an intervention, for example, if the intervention was a programme of care.

Consequences of allocating clusters • Two individuals in the same cluster are more alike than two individuals in different clusters. This clustering needs to be accounted for in the analysis (E see Cluster samples: analysis, p. 456) • A cluster trial needs a larger sample than the equivalent trial randomized at the individual level and so the clustering needs to be considered in the sample size calculations. These calculations use a measure called the ‘intraclass correlation coefficient’ or ‘ICC’ which quantifies the extent to which individuals within the same cluster are more alike than those in different clusters (E see (ADV) Sample size in cluster trials, p. 100)

 

Cluster trials

Challenges with cluster trials • Number of clusters: the number of clusters required depends partly on the number of individuals available within each cluster (E see Sample size in cluster trials, p. 100). If the number of clusters is small, there is a greater chance of imbalance in baseline characteristics between treatment groups. In addition, there needs to be a reasonable number of clusters for the analyses to be valid. Eldridge and Kerry (2012) give a full discussion of the choice of number of clusters • If a whole cluster drops out for some reason, the impact on power and balance between the arms is greater than if an individual drops out in an individually randomized trial

Examples Two examples are given in a later chapter (E see Cluster samples: analysis, p. 456).

Further reading Kerry SM, Bland JM. Analysis of a trial randomised in clusters. BMJ 1998; 316:54. Kerry SM, Bland JM. Sample size in cluster randomisation. BMJ 1998; 316:549.

Reference Eldridge S, Kerry SM. A practical guide to cluster randomised trials in health services research. Chichester: Wiley, 2012.

43

4

44

Chapter 2  

Research design

Intention-​to-​treat analysis Introduction The statistical analysis of RCTs is relatively straightforward where there are complete data. The primary analysis is a direct comparison of the treatment groups, and this is performed with subjects being included in the group to which they were originally allocated. This is known as analysing according to the intention to treat (ITT) and is the only way in which there can be certainty about the balance of the treatment groups with respect to baseline characteristics of the subjects. ITT analysis therefore provides an unbiased comparison of the treatments.

Change of treatment If patients change treatment they should still be analysed together with patients in their original, randomly allocated group, since a change of treatment may be related to the treatment itself. If a patient’s data are analysed as if they were in their new treatment group, the balance in patient characteristics which was present at random allocation will be lost. A per protocol analysis, where patients are analysed according to the treatment they have actually received, may be useful in addition to the ITT analysis if some patients have stopped or changed treatment. e Complier average causal effect (CACE) methods can be used to disentangle the effects of treatment in compliers—​for more details see Ye et al. (2014).

Missing data Missing data are unfortunately common in all research studies, particularly where data are collected at several time points. Where there are missing data, it may not be possible to include a particular individual in the analysis, and clearly if there are a lot of missing data, the validity of the results is called into question. Where possible, all subjects should be included in the analysis. In a trial with follow-​up it may be possible to include subjects with no final data if they have some interim data available, either by using the interim data directly or by statistical modelling. These issues should be addressed through careful design of outcome data and strategies to minimize loss to follow-​up. All subjects recruited should be accounted for at all stages so that a detailed account can be given of how the trial was conducted and what happened to all subjects. This is particularly important for the interpretation of the findings and so is included when the study is written up. A fuller discussion of missing data is given elsewhere (E see Missing data, p. 432).

 

Intention-to-treat analysis

ITT and missing data • Analyse subjects in the groups they were originally allocated to even if they change treatment or don’t comply • This provides an unbiased comparison of the treatments • Per protocol analysis may be useful but only in addition to ITT and not as the primary analysis (see following example) • Keep a record of all subjects to be able to account for their treatment and for any subjects who withdraw

Example RCT of introduction of allergenic foods in breastfed infants This trial evaluated the early introduction of allergenic foods in the diet of breastfed infants to test the hypothesis that early introduction provided protection against the development of immunoglobulin E-​mediated food allergy. The results showed that the early introduction group had a non-​significant reduction in allergy at age 3 years in the intention-​to-​treat analysis (relative risk (RR) 0.80; 95% confidence interval (CI) 0.51, 1.25). The researchers had observed some non-​compliance with early introduction of foods and so a per protocol analysis was conducted. This showed that in those who complied with early introduction, there was a significantly lower risk of allergy at age 3 (RR 0.33; 95% CI 0.13, 0.83). The researchers were unable to draw firm conclusions about the benefits of early introduction but noted no evidence of harm and a suggestion of efficacy in those that complied. See Perkin MR, et al. (2016). Randomized trial of introduction of allergenic foods in breast-​fed infants. N Engl J Med 2016; 374:1733–​43.

Further reading Matthews JNS. Introduction to randomized controlled clinical trials. Boca Raton, FL: Chapman & Hall/​ CRC, 2006. Piantadosi S. Clinical trials: a methodologic perspective. Chichester: Wiley, 2005. Pocock SJ. Clinical trials: a practical approach. Chichester: Wiley, 1983.

References Perkin MR, Logan K, Tseng A, Raji B, Ayis S, Peacock J, et al. Randomized trial of introduction of allergenic foods in breast-​fed infants. N Engl J Med 2016; 374:1733–​43. Ye C, Beyene J, Browne G, Thabane L. Estimating treatment effects in randomised controlled trials with non-​compliance: a simulation study. BMJ Open 2014; 4:e005362.

45

46

46

Chapter 2  

Research design

Case–​control studies Observational studies In observational studies, the subjects receive no additional intervention beyond what would normally constitute usual care. Subjects are therefore observed in their natural state.

Case–​control study • This study investigates causes of disease, or factors associated with a condition • It starts with the disease (or condition) of interest and selects patients with that disease for inclusion, the ‘cases’ • A comparison group without the disease is then selected, ‘controls’, and cases and controls are compared to identify possible causal factors • Case–​control studies are usually retrospective in that the data relating to risk factors are collected after the disease has been identified. This has consequences, which are discussed later in this section

When to use a case–​control design • To investigate risk factors for a rare disease where a prospective study would take too long to identify sufficient cases—​for example, for Creutzfeldt–​Jakob disease • To investigate an acute outbreak in order to identify causal factors quickly—​for example, where an answer is needed about the causes of an outbreak of food poisoning, or an outbreak of Legionnaire’s disease

Choice of controls As with intervention studies, the choice of controls affects the comparison that is made. Common choices include: • Patients in the same hospital but with unrelated diseases or conditions • Patients one-​to-​one matched to controls for key prognostic factors such as age and sex • A random sample of the population from which the cases come Clearly the best control group is the third option, but this is rarely possible. For this reason, some case–​control studies include more than one control group for robustness.

Matched controls Matching is popular but needs to be carefully specified, for example, ‘age matched within 2 years’ gives the range within which matching can be made. It is not usually possible to match for many factors, as a suitable match may not exist. In a matched design, the statistical analysis should take account of the matching and factors used for matching cannot be investigated due to the design. Where one subject in a matched pair has missing data, then both subjects are omitted from the statistical analysis.

 

Case–control studies

Sample size for controls It is common to choose the sample size so that there is the same number of cases as controls. For a given total sample size this gives the greatest statistical power, that is, the greatest possibility of detecting a true effect. If the number of available cases is limited, then it is possible to increase the power by choosing more controls than cases. However, the gain in power diminishes quickly so that it is rarely worth choosing more than three controls per case (Taylor 1986).

Collecting data on risk factors Since case–​control studies start with cases that already have the disease, data about their exposure to possible risk factors prior to diagnosis is collected retrospectively. This is both an advantage and a disadvantage. The advantage is that the exposure has already happened and so the data simply need to be collected; no follow-​up period is needed. The disadvantage relates to the quality of the data. Data taken from clinical notes may contain errors that cannot be rectified or gaps that cannot be filled. Data obtained directly from subjects about their past is susceptible to recall bias because cases may have different recall of past events, usually better, than the controls. For example, a case with a gastrointestinal condition may be more conscious of what they have eaten in the past than a healthy control who may have simply forgotten.

Reference Taylor JM. Choosing the number of controls in a matched case–​control study, some sample size, power and efficiency considerations. Stat Med 1986; 5:29–​36.

47

48

48

Chapter 2  

Research design

Case–​control studies (continued) Limitations of design • The choice of control group affects the comparisons between cases and controls • Exposure to risk factor data is usually collected retrospectively and may be incomplete, inaccurate, or biased • If the process that leads to the identification of cases is related to a possible risk factor, interpretation of results will be difficult (‘ascertainment bias’). For example, suppose the cases are young women with high blood pressure recruited from a contraception clinic. In this situation, a possible risk factor, the oral contraceptive (OC) pill, is linked to the recruitment of cases and so OC use may be more common among cases than population controls for this reason alone. • Time-​course relationships need careful interpretation since changes in biological quantities may precede the disease or be a result of the disease itself. For example, a raised serum troponin level is associated with myocardial infarction, but is only raised after the event. Therefore, a case–​control study may find that high troponin levels are associated with myocardial infarction but this cannot in fact be a risk factor • Risk estimates for exposures cannot be estimated directly because the case and control groups are not representative samples of their respective target populations and so estimates of risks are biased. This has implications for the statistical analysis and the interpretation of results. Risks are usually estimated using odds and ratios of odds, and these only approximate to risks and ratios of risks when the disease under investigation is rare • This limitation can be overcome with certain designs, for example, where a case–​control study is nested in a cohort study where all cases and controls are identified prospectively and a truly random sample of controls is available (E see Cohort studies, p. 50). In this situation, the relative risk can be calculated directly

 

Case–control studies (continued)

Example of a case–​control study A study investigated the association between genitourinary infections in the month before conception to the end of the first trimester, and gastroschisis (Feldkamp et al. 2008). The subjects were 505 babies with gastroschisis (the ‘cases’), and 4924 healthy liveborn infants as controls. The study reported data (Table 2.1) showing a positive relationship between exposure to genitourinary infections and gastroschisis (odds ratio = 2.02; 95% CI 1.54, 2.63). Table 2.1  Genitourinary infections in the month before conception to the end of the first trimester, and gastroschisis Exposed to infection?

Cases

Controls

Yes

81/​505 (16%)

425/​4924 (9%)

No

424/​505 (84%)

4499/​4924 (91%)

Source: data from Feldkamp ML, Reefhuis J, Kucik J, Krikov S, Wilson A, Moore CA, et al. Case–​ control study of self-​reported genitourinary infections and risk of gastroschisis: findings from the national birth defects prevention study, 1997–​2003. BMJ 2008; 336(7658):1420–​3.

Reference Feldkamp ML, Reefhuis J, Kucik J, Krikov S, Wilson A, Moore CA, et al. Case–​control study of self reported genitourinary infections and risk of gastroschisis: findings from the national birth defects prevention study, 1997–​2003. BMJ 2008; 336:1420–​3.

49

50

50

Chapter 2  

Research design

Cohort studies Introduction A cohort study is an observational study that aims to investigate causes of disease or factors related to a condition but, unlike a case–​control study, it is longitudinal and starts with an unselected group of individuals who are followed up for a set period of time. Cohort studies are sometimes used to confirm the findings of case–​control studies, such as happened when Doll and Hill (1950) observed a relationship between smoking and lung cancer in a case–​control study and subsequently established the longitudinal study of doctors in the UK (Doll et al. 2004).

Design of a cohort study • This starts with an unselected group of ‘healthy’ individuals • The subjects are followed up to monitor the disease or condition of interest and potential risk factors • The length of follow-​up is chosen to allow sufficient subjects to get the disease and risk factors to be explored • In the simplest case, where there is a single risk factor that is either present or absent, the incidence of disease can be related directly to the presence of the risk factor • It is usually prospective, with the risk factor data being recorded before the disease is confirmed • It can be retrospective but requires that full risk factor data are obtained on all individuals with and without the disease of interest using data that were recorded prospectively

When to use a cohort study design • When precise estimates of risk associated with particular factors are required, for example, when a case–​control study has established that an association exists but is unable to provide estimates of the risk • When information on past risk factors in individuals with disease is unavailable or too unreliable to use • When the time-​course of a risk factor is of interest, for example, with smoking, where cohort studies have been able to demonstrate the cumulative adverse effects of long-​term smoking and the potential benefits of quitting after smoking for different lengths of time (Doll et al. 2004) • When resources and time are sufficient to support a lengthy study

Difficulties with cohort studies • A large number of subjects are needed to obtain enough individuals who get the disease or condition, particularly if it is uncommon • The length of follow-​up may be substantial to get enough diseased individuals and so the cohort study is not feasible for rare diseases • There is difficulty in maintaining contact with subjects, particularly if the follow-​up is lengthy • The resources required may be very high

 

Cohort studies

Table 2.2  Relative risk of death in men aged 50–​71 at enrolment by BMI BMI at age 50

Relative risk

9 months (n = 56) Measure

Mean z-​score

Mean difference (95% CI)

≤9 m

>9 m

Unadjusted

Adjusteda

Test for reception of grammar

–​1.46

–​2.25

0.78 (0.08 to 1.48)

0.90 (0.32 to 1.47)

British picture vocabulary scale

–​1.86

–​2.36

0.50 0.64 (–​0.11 to 1.11) (0.13 to 1.16)

Aggregate score

–​1.76

–​2.38

0.61 0.76 (–​0.02 to 1.24) (0.26 to 1.27)

Aggregate score minus non-​verbal

–​0.82

–​1.68

0.86 (0.32 to 1.40)

a

0.82 (0.31 to 1.33)

Adjusted for severity of hearing impairment, maternal education.

Interpretation • Each row of the table is a separate multiple regression analysis where the outcome is the measure named in the first column • In each multiple regression the outcome has been adjusted for the same variables, severity of hearing impairment, and maternal education • The results are presented as mean differences between the two groups, unadjusted and adjusted • The adjustment has increased the magnitude of the difference for all measures except ‘aggregate score minus non-​verbal’ where the difference is slightly smaller • All adjusted differences are statistically significant as shown by the 95% confidence intervals, which exclude the null value of zero Conclusions of study • The study concluded that early detection of hearing loss was associated with higher scores for language (see original article for more details)

 

Multiple regression: examples

Notes • All mean z-​scores were negative, showing that these children had poorer language attainment than children of the same age without hearing loss • This is important information to aid interpretation showing the importance of presenting the means in the two groups as well as the differences

Reference Kennedy CR, McCann DC, Campbell MJ, Law CM, Mullee M, Petrou S, et al. Language ability after early detection of permanent childhood hearing impairment. N Engl J Med 2006; 354:2131–​41.

477

478

478

Chapter 12  

Analysing multiple variables per subject

Multiple regression: examples (continued) Using multiple regression to produce an equation A study of factors affecting fetal growth in 1513 singleton babies used multiple regression to adjust the babies’ birthweights for their mother’s height, the sex of the infant, and mother’s parity (Brooke et al. 1989). The birthweight was analysed as a ratio of the observed birthweight to the expected birthweight-​for-​gestational age derived from UK birthweight standards. Table 12.3 shows the adjusted coefficients: Table 12.3  Results of multiple regression to predict birthweight Variable

Regression coefficient

95% CI

Height (cm)

0.0036

0.0026, 0.0046

Sex (female = 1, male = 2)

0.0440

0.0311, 0.0569

Parity (1st baby = 0, 2nd or later = 1)

0.0353

0.0224, 0.0482

Intercept

0.3335

0.1651, 0.5019

Interpretation • The coefficient for height, 0.0036, estimates the difference in birthweight ratio between two women whose height differed by 1 cm • The coefficient for sex, 0.0440, estimates the mean difference in birthweight ratio between boys and girls • The coefficient for parity, 0.0353, estimates the mean difference in birthweight ratio between second or later babies and first babies • The intercept is a constant which estimates the value of the birthweight ratio when maternal height is zero • All coefficients are statistically significant as shown by the 95% CIs which exclude the null value, zero • The multiple regression results correspond to the following equation:

birthweight ratio = 0.3335 + (0.0036 × height ) + (0.044 × sex ) + ( 0.0353 × parity ) • The equation was used to adjust each baby’s birthweight for the maternal and infant variables which were regarded as ‘nuisance’ variables in this context • The resulting adjusted birthweight ratio was used as the outcome variable in further multifactorial analyses of smoking, alcohol, and other lifestyle factors on fetal growth

 

Multiple regression: examples (continued)

Notes • In order to use the equation it is necessary to know the coding that was used for sex of infant and parity • Differences in birthweight ratios have an easy interpretation as a percentage difference: for example, the birthweight ratios 1.05 and 1.01 have a difference of 0.04 and so the difference in the two birthweights is 4% • In this example, the equation was used to compute an adjusted outcome (see later for further details); in other situations, an equation may be required to compute a set of predicted values such as when computing predicted values for lung function measurements given a subject’s age, height, and sex

Additional details: how to calculate adjusted birthweight The researchers adjusted the birthweight ratios to height  =  160  cm, male sex, and parity 1 on the advice of the study obstetrician. This was achieved using the following equation derived from the original multiple regression (Brooke et al. 1989): Adjusted birthweight =

birthweight ratio − 0.0036 × (height −160 ) − 0.044 × (sex − 2 ) − 0.0353 × (parity −1) Where sex was coded 1 (female), or 2 (male); and parity was coded 0 (first baby), 1 (second or later baby). So for a mother of height 155 cm, who had a girl who was her first baby, with a birthweight ratio of 1.00, the adjusted birthweight was given by:

1.00 − 0.0036 × (155 −160 ) − 0.044 × (1 − 2 ) − 0.0353 × ( 0 −1) = 1.0973

Reference Brooke OG, Anderson HR, Bland JM, Peacock JL, Stewart CM. Effects on birth weight of smoking, alcohol, caffeine, socioeconomic factors, and psychosocial stress. BMJ 1989; 298:795–​801.

479

480

480

Chapter 12  

Analysing multiple variables per subject

Multiple regression and analysis of variance Analysis of variance table for multiple regression The results of a multiple regression can be shown as an analysis of variance (ANOVA) table (E see Analysis of variance table, p. 328). To illustrate, Table 12.4 shows the analysis of variance table for the multiple regression of birthweight ratio and mother’s height, sex of infant, and the parity. Table 12.4  Analysis of variance table for multiple regression to predict birthweight Factor

DF

Sum of squares Variance estimate

F ratio

P value

Model Height Sex of infant Parity

3

1.9082

0.6360

39.15

0.0001

Residual

1509

24.5152

0.0162

Total

1512

26.4234

(E See Table 12.3, p. 478.)

Explanation of table • Row 2 gives the statistics for the model that was fitted, that is, the set of variables height, sex of infant, and parity. Row 4 gives the overall totals. Row 3 gives the residual or unexplained part of the variation • DF is degrees of freedom; it is the number of variables in the model (3) for row 2, total number observations − 1 (1512), for row 4, and the difference between these, 1512 − 3 = 1509, for row 3 • Total sum of squares is the sum of squares of the overall mean minus each observation squared. The other sums of squares cannot be easily calculated by hand • The model and residual sums of squares add up to the total, that is, 1.9082 + 24.5152 = 26.4234 • Variance estimate is the sum of squares/​DF: F ratio is ratio of two variances: 0.6360/​0.0162 = 39.15 • P value is probability associated with an F value of 39.15 if the null hypothesis that the model variables collectively are unrelated to the outcome, birthweight ratio, is true. As it is very small, we conclude that the model variables height, sex, and parity are related to birthweight

Two-​way analysis of variance The method of one-​way analysis of variance (E see One-​way analysis of variance p. 324) can be extended to allow two factors to be analysed together. For example, Table 12.5 shows data from a clinical trial investigating the effect of different topical analgesics and different gauge needles on reported pain on injection (Nott and Peacock 1990).

 

Multiple regression and analysis of variance

Table 12.5  Median visual analogue scale (0–​10) pain score after injection in 120 subjects Treatment

Needle size (gauge)

All

22

20

18

EMLA™, 60

0.3

0.2

1.2

0.4

EMLA™, 5

0.4

0.5

1.1

1.0

Placebo

0.8

1.4

2.3

1.9

Nil

1.6

1.2

2.8

2.3

All

0.5

0.9

1.9

These data were analysed using two-​way analysis of variance but could equally have been analysed using multiple regression to give the same answer since analysis of variance is a special case of multiple regression.

Balanced designs In a two-​way analysis of variance, the data are said to be balanced if there are equal numbers of subjects for each combination of factors. Otherwise the design is unbalanced. In balanced designs, the sums of squares add up and analysis of variance can be done by hand using formulae. This is of no great importance nowadays since computers are so readily available to most people, but the issue was critical when most calculations were done manually. This is why many older text books describe analysis of variance and multiple regression separately.

Unbalanced designs Unbalanced data are common in medical research. This affects the way that analyses are done in modelling situations, such as when adding another variable to a particular group of variables to see if the model is improved. In such a situation it is necessary to do the following: • Fit the model (i) without and then (ii) with the new variable • Test the addition of the new variable using the extra sum of squares that the new variable adds to the model • For an example of this, E see Linear and non-​linear terms p. 484

0 Choice of method: analysis of variance or multiple regression Some statistical programs will do both methods but the different commands may deal with predictor variables differently. For example, in Stata the ‘anova’ command assumes all variables are categorical unless the user specifies otherwise, whereas its ‘regression’ command assumes that all variables are continuous unless otherwise specified.

Reference Nott MR, Peacock JL. Relief of injection pain in adults. EMLA cream for 5 minutes before venepuncture. Anaesthesia 1990; 45:772–​4.

481

482

482

Chapter 12  

Analysing multiple variables per subject

Main effects and interactions Introduction The regression analyses considered so far have only included main effects. In other words, it was assumed that the effect of one predictor on an outcome was constant for all values of other predictor variables in the model. For example, in the birthweight study data presented earlier in the chapter, it was assumed that the effect of maternal height on birthweight was the same for both males and females and also that the difference in mean birthweight between males and females did not vary with mother’s height. These assumptions were reasonable but in other situations they may not be so.

Example An ecological study investigated the inter-​relationships between tooth decay in children, water fluoridation, and deprivation ( Jones et al. 1997). The study found a protective effect of both natural and artificial fluoridation on tooth decay and an adverse effect of deprivation. But the study also reported an interaction such that the observed benefit of fluoridation was greater in more deprived areas than in less deprived areas. Figure 12.1 depicts this.

Figure 12.1  Mean decayed, missing, or filled tooth score and Jarman

underprivileged area score, by non-​fluoridated, artificially fluoridated, and naturally fluoridated electoral wards. Reproduced from Jones CM et al. (1997) “Water fluoridation, tooth decay in 5 year olds, and social deprivation measured by the Jarman score: analysis of data from British dental surveys” BMJ 315:514–​517 with permission from BMJ Publishing Group.

 

Main effects and interactions

Interpretation of graph • Both artificial and natural fluoridation were associated with a lower ward tooth decay score than no fluoridation • The differences in mean ward tooth decay score were small for areas with low Jarman deprivation score (left-​hand end of the graph) and were much greater for areas with high deprivation scores (right-​hand end of the graph) • There is an interaction between fluoridation and deprivation such that the effects of fluoridation are greater in areas with high deprivation Notes • When there is an interaction between two factors as there is here, the regression lines are not parallel—​there is a different line for each group • If an interaction term had not been included in the model, three parallel lines would have been computed but these would not have represented the data very well • The statistical model used here was:

tooth decay = fluoridation + deprivation + (fluoridation × deprivation) where fluoridation is in three groups and deprivation score is continuous, and the multiplicative term ‘fluoridation × deprivation’ is the interaction term

Reference Jones CM, Taylor GO, Whittle JG, Evans D, Trotter DP. Water fluoridation, tooth decay in 5 year olds, and social deprivation measured by the Jarman score: analysis of data from British dental surveys. BMJ 1997; 315:514–​17.

483

48

484

Chapter 12  

Analysing multiple variables per subject

Linear and non-​linear terms Introduction The basic assumption of regression is that the relationship between a predictor and outcome variable is linear. If this is not true, it may be possible to find a transformation of the variable that will give a linear relationship so that regression methods can be used. For example, if the relationship is U-​shaped (quadratic), then a relationship of the following form can be used:

y = a + bx + cx 2

Example The EMLA™ trial investigated whether using EMLA™ anaesthetic cream for just 5 minutes before injection was better than using nothing (Nott and Peacock 1990) (E see Table 12.5, p. 481 for the summary data). The trial included four treatment groups: EMLA™ applied 60 minutes before injection (known to be effective), EMLA™ applied 5 minutes before injection, placebo cream applied 5 minutes before injection, and nothing. Pain was assessed using a 10 cm visual analogue scale. The data were analysed firstly using two-​way analysis of variance and then using a multiple regression model to adjust for the effect of the age of the patient (Table 12.6). Age did not have a linear relationship with pain; reported pain was highest for the youngest and oldest people and lowest for those in between so the relationship was U-​shaped. To model this, the factors age and age squared (age2) were put into the multiple regression model. Table 12.6  Analysis of variance table for multiple regression in EMLA™ trial Factor

DF

Sum of squares Variance estimate F ratio

P value

Age + age2

2

7.49

3.75

4.40

0.01

Needle size 2

19.39

9.70

11.39