Surviving and Thriving in Postgraduate Research 2nd Ed 9811377464, 9789811377464

Citation preview

Ray Cooksey · Gael McDonald

Surviving and Thriving in Postgraduate Research Second Edition

Surviving and Thriving in Postgraduate Research

Ray Cooksey Gael McDonald •

Surviving and Thriving in Postgraduate Research Second Edition

123

Ray Cooksey UNE Business School University of New England Armidale, NSW, Australia

Gael McDonald RMIT University Vietnam Ho Chi Minh City, Vietnam

ISBN 978-981-13-7746-4 ISBN 978-981-13-7747-1 https://doi.org/10.1007/978-981-13-7747-1

(eBook)

1st edition: © Tilde University Press 2011 2nd edition: © Springer Nature Singapore Pte Ltd. 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Preface

As mentioned in the foreword of the first edition, the genesis of this text goes back to when Gael and I contributed to doctoral workshops for the Australian and New Zealand Academy of Management (ANZAM). These experiences, along with our own experiences in supervising postgraduate research students, led us to assemble a list of the common sorts of questions postgraduate students tended to ask of us. These questions subsequently formed the basis for the chapters and subheadings in this book and with the resulting proposition that we write a book around these questions. We both thought the book was sorely needed and would provide value to postgraduate students. Given the encouraging feedback we have received on the first edition we were not wrong. With the support of Springer, we have now produced the second edition. When considering the timeline associated with initially publishing the book and the production of the second edition, it is difficult to know if the starting point is when we first conceived of the idea, commenced writing or initial publication? Or do we look from when we started the review for the second edition or count from the submission of the fully revised manuscript? We mention this because it actually provides an opportunity for parallel reflection on our lives. Gael started writing the first chapter for the first edition when she found out she was pregnant with twins. They are now 12 years old! Gael has since moved country twice, held two senior academic administration roles and is currently President of RMIT University Vietnam. In the interim, Ray retired from his senior academic role at the University of New England with the title of Emeritus Professor but continues to be active in research support and doctoral supervision. Prior to his retirement, Ray worked with a team that designed a new professional doctorate in innovation research programme from the ground up, culminating in what is now called the PhD.I at the University of New England and some of those experiences are called upon in this edition. Once again, our writing progressed in fits and starts as our workloads allowed and, in the end, we have realised our goal of the second edition. In the academic postgraduate space, a lot has also happened. There is now much closer scrutiny of the timelines for postgraduate completion, greater pressure for students to publish while during their studies, great recognition of well-being issues v

vi

Preface

associated with postgraduate study and, of course, there is considerably more assistance available to students given access to a vast array of digital resources. Gael was acutely aware of this as she sat in the back of the car for an hour-long trip to the airport and was able to access relevant material from a variety of databases for this edition from her iPad. Why did we embark on the second edition? Well, we were encouraged and motivated by the number of students who had indicated how useful they found the first edition of this text, for a range of disciplines including business, management, marketing, education, nursing, psychology, sociology, social policy and other social sciences. Working on the second edition naturally provided us with the opportunity to update material, to take out old links, review the current literature, reflect on changes in postgraduate student experiences, to internationalise and to provide more contemporary support for students. As well, writing the second edition gave us the opportunity to update all discussions of research methodology with a view towards being more systemic, inclusive and pluralist in our coverage. However, in a number of areas, we are intrigued to see how the hallmarks of what constitutes good quality research have not varied significantly. While methodologies and presentation formats for theses, dissertations and portfolios evolve, the basic tenets of high-quality meaningful researching, adhering to principles of academic integrity and the importance of the contribution of findings to both theory and practice, have remain constant. As with the first edition, we balance our coverage of the technical aspects of research (how to carry out a research investigation from start to finish) with the experiential aspects (how to successfully navigate the journey). While the content in this edition has been significantly updated, we have retained the logical ordering of the first edition chapters, which corresponded to different parts of the postgraduate research journey. More specifically, in this new edition, we: • incorporate a new Prelude which sets the context for the book and introduces the metaphor of a research journey, providing the anchoring focus for the chapter structure and content; • implement a unified complex systems approach to undertaking research in the social and behavioural sciences; • go beyond the constraints imposed by mixed methods thinking by adopting a complex pluralist perspective; • offer more extensive coverage of indigenous research considerations; • incorporate considerations and implications associated with professional doctorates as a distinctive point of focus; and • adopt a more international perspective, instead of being focused primarily on the Australian/New Zealand contexts. The chapters in this second edition do not necessarily have to be read sequentially. Depending upon where you are in your research journey, you could choose to commence your reading in those chapters most relevant to that stage or pertinent to a current issue or problem you are confronting. Alternatively, you could commence

Preface

vii

at the start to see if there are strategies and ideas you could still implement, even if you are at one of the later stages. If you are just starting out or even just contemplating undertaking a postgraduate research journey, then the whole of this book should help you to see what lies ahead as well as to prepare for what is to come. From Gael: I would like to thank my fabulous husband Collin who continues to undertake the lion’s share of child minding. I feel truly blessed to have the love and support of this man. Recognition is due to my former PA, Lynn Spray, who had an immense involvement in the preparation of the first edition and also to my current EA, Hoang Trinh Uyen Phuong and my research assistant, Truong Thi Tuyet Ngan. Thank you to the McDonald family, particularly my sister Christine, who continually encourages the achievement of goals. Naturally, appreciation is also given to the many students and academics whose experiences have been woven into the fabric of this book. From Ray: I want to thank my wife, Christie, for her continuing love and support throughout this project, especially as it extended over several years. She is my rock, even in tough times. As in the first edition, the work of a number of master’s, Ph.D. and professional doctorate students I have supervised features prominently in some sections of the book and their journeys to success deserve to be acknowledged here: Peter McClenaghan, Jean Sandall, Michael Muchiri, Michael Braund, Janene Carey, Jenny Harrison, James Hunter, Joanna Henryks, Ravi Pappu, Eugene Ross, Leopold Beyerlein, Keith Wolodko, Valerie Dalton, Peter Fieger, Areej Alfawaz, Ziyad Alghannam, Ahmed Alhosany, Phillippa Kirkpatrick, Geoff Kaine, Fredy-Roberto Valenzuela (deceased), Sujana Adapa, Martin Robson, Lisa Cowan, Kezang Sherab and Wayne Gregson. I was part of all of your journeys and am proud to share aspects of them here in these pages. Armidale, NSW, Australia Ho Chi Minh City, Vietnam February 2019

Ray Cooksey Emeritus Professor Gael McDonald President and General Director

Prelude: The Postgraduate Research Journey

The concept of a journey is a perfect metaphor for postgraduate research in the social and behavioural sciences. Every postgraduate research journey is embedded within contexts that are generally not static but evolving and dynamically changing. Thus, your postgraduate research journey will seldom be a purely linear one; it will not necessarily travel in a direct series of steps from start to finish. Rather your journey will tend to be punctuated by bumps in the road, unexpected obstacles and events, periods of doubt and uncertainty, lapses in motivation and focus, encounters with blind alleys, competition with other events in your life, excitement, enlightenment, disappointment, discouragement, possible choices and movements back and forth between data and interpretation and so on. Sometimes in postgraduate research, it can be more a case of one step forward, two steps back. However, your journey, if well-travelled and completed, will be not only challenging but also very rewarding. Contributing to knowledge is no small achievement; it takes training, thinking, time, perseverance, commitment, resilience and occasionally a bit of luck.

The Main Pathways of the Journey There are actually three versions of the research journey undertaken by every postgraduate researcher: (1) the anticipated/imagined/planned journey which is conveyed in your research proposal; (2) the actual journey that you experience as a researcher, which unfolds in real time and is influenced by a range of events, both anticipated and unanticipated; and (3) the reflective journey where you revisit key decisions and essential aspects of your experiences and learning to assemble and convey a convincing story in a specific research outcome. Each journey reflects a different way of navigating the various processes and stages associated with the research. Figure 1 provides a visualisation of the research journey, which comprises nine identifiable phases embedded within a rich contextual background.

ix

x

Prelude: The Postgraduate Research Journey

• Establish/maintain a place for research in your life: The research journey starts with you, the postgraduate researcher, establishing and then working to maintain a place for research within your life space. Prior to undertaking a research endeavour, you need to think broadly and critically about your motivations, your skills, strengths and weaknesses, choices of and requirements for the specific research programme to which you will commit, your needs (including developmental and skill enhancement needs), intentions and goals, the opportunities and constraints you can see ahead, the research context(s) you will need to work within and the needs/expectations of other stakeholders in your research (including, importantly, your supervisor(s), your family, friends and possibly employer). You need to carefully consider the choice of institution/department under whose auspices you would conduct your research as well as your personal, intellectual, social and financial capacities to commit to a postgraduate programme. Finally, you need to begin cultivating support systems and strategies to help you through the journey. • Prepare for your journey: Your research journey needs to be planned as far as possible. You will need to marshall resources, skills and capacities to support your endeavour and you will need to maintain a detailed journal that records key facets of your journey, including ideas, reflections, contextual notes, encounters with and learning from the literature as well as significant stakeholders, and the choices and adaptations you make along the way. However, throughout your journey, you will need to maintain a healthy respect for the fact that planning will only take you so far. There will be things over which you have no control and things that you cannot plan for or must wait to plan for downstream. Prepare to be flexible and adaptable if circumstances change, which may require you to revisit, rethink, make new choices, adapt, sacrifice or augment; use your journal to record your evolving thoughts. • Contextualise, frame and position your research: To be convincing, you must effectively and fully contextualise your research (i.e. understand the who, what, where and why with respect to your research and where and how it fits with the research of others and with the needs of relevant stakeholders). You will need to frame your research in terms of overall intent and the questions and/or hypotheses to be addressed. You must appropriately position not only yourself (your own context with respect to where you are coming from as the researcher) but also potential and actual research ‘participants’ (their context(s) with respect to where they are ‘coming from’). Part of contextualising and framing your research involves making very clear decisions about the pattern(s) of assumptions you will adopt to guide your research (often referred to as ‘paradigm’ choices). To be effective and to have impact and contribute theoretically and/or practically, your research must be convincing to relevant stakeholders and interested parties. This means that all aspects of your research must maintain close attention to research quality and stakeholder needs. We will see that research quality can be addressed at two levels: ‘paradigm’-specific criteria and meta-criteria. You will find the meta-criteria especially useful in helping you the plan and implement your research. All this activity serves to flesh out the

Prelude: The Postgraduate Research Journey

xi

amorphous light-grey-shaded area in Fig. 1 which then helps you to shape the choices you make throughout your journey.

Establish/maintain a place for research in your life Prepare for your research journey

Contextualise, frame & position your research Configure your research activities Present a convincing proposal Access/connect with your data sources

The “Data Triangle”

Implement your data gathering strategies

Build meaning from your data

Evolving context(s) within which your research is embedded

Create convincing outcome(s)

?

Innovation Thesis/ Publication/ Dissertation Conference/ Portfolio Paper

Major journey pathway Thinking forward: anticipations & planning Thinking backward: reflections, characterisations & reconsiderations Different possible research outcomes Unanticipated influences on the journey

Fig. 1 The research journey visualised

xii

Prelude: The Postgraduate Research Journey

• Configure your research activities: You will need to configure your data-gathering activities so that you can effectively achieve your intended goals and address your research questions. This is done in full awareness of your choices of research frame, guiding assumptions and with consideration given to relevant characteristics of contextualisation and positioning. Configuring research involves deliberations on types of data sources, types of data (quantitative/qualitative) to be generated and specific data-gathering strategies to be employed. Each specific combination chosen comprises a ‘Method Unit’ (MU). Method Units form the basic building blocks for any research configuration. Each Method Unit is accompanied by its own set of procedural considerations that must be planned and resourced before implementation. • Present a convincing proposal: Your research proposal is generally the first research ‘outcome’ you will produce. It is the story about your anticipated research journey, intended to convince your supervisors, peers and other interested parties that you will learn something of value, through the exercise of independent research effort, sufficient to warrant awarding of the degree you are pursuing. This story may be produced in a written or oral form (in many cases, both) and, in many postgraduate programmes, stands as an important hurdle you must clear before embarking on your research journey proper (many institutions may refer to this hurdle as ‘confirmation’ or as ‘confirmation of candidature’). • Access/connect with your data sources: This is one vertex of what is called the ‘Data Triangle’ in Fig. 1. To focus data-gathering activities, you will need to use sampling processes to facilitate accessing or connecting with your data sources, in a manner consistent with your adopted pattern(s) of guiding assumptions and intended research goals. Data sources here may be human or non-human (e.g. texts/documents, YouTube segments, movies, policies, paintings/drawings, cultural artefacts, secondary databases, animals) in nature. Sampling may comprise anonymous choices using numerical identifiers (e.g. random sampling) or specifically identified and chosen data sources (e.g. purposive sampling). If the data sources/participants you want to choose are human, then their ethical right to give informed consent to participate overrides your desire for them to participate, which means that participation is their choice, not yours. This then becomes one of the aspects of your research that you cannot completely control. • Implement your data-gathering strategies: This is the second vertex of the ‘Data Triangle’. Data-gathering/data creation strategies need to be implemented in a manner consistent with your adopted pattern(s) of guiding assumptions with close attention to relevant research quality criteria. You want any data gathered or created to be directly relevant to your research goals. Many data-gathering strategies can work synergistically together to enrich the learning you can achieve and, where appropriate, the participant involvement you seek. For the social and behavioural sciences, data-gathering strategies can be categorised as those that facilitate connecting with people, those that facilitate exploring the handiworks produced by (or valued by) people and those that help the researcher to, in most cases, structure the experiences of people. Some are oriented towards gathering quantitative data and others towards qualitative data. Mixed method

Prelude: The Postgraduate Research Journey

xiii

practitioners argue that both types of data are needed to convey a convincing story, but this thinking is too limiting. Gathering the same type of data using different data-gathering strategies can be equally convincing. Here, we reinforce the value of pluralist thinking rather than just mixed methods thinking. • Build meaning from your data: This is the third vertex of the ‘Data Triangle’. Quantitative and/or qualitative data that you have gathered (or created) need to be analysed in a manner consistent with your adopted pattern(s) of guiding assumptions as well as the Method Unit configuration you have implemented. Your goal is to build meaning from the data so that you can convey the stories they reveal relevant to your research goals and questions. Descriptions, social constructions, patterns, connections, relationships, dynamics and changes over time may emerge from appropriate analyses; models may be tested, and theories constructed. While analyses of quantitative and qualitative data draw upon very different procedures, the end goal is the same—to discover what are the data telling you and to create displays and narratives that convey those stories to others. If the stories that arise from quantitative and/or qualitative data can be integrated to highlight points of convergence, divergence, qualification or deeper understanding, this can add to the convincingness of your research. It is important to note that (1) each vertex of the ‘Data Triangle’ addresses one of the three synergistic characteristics of a Method Unit, (2) relevant characteristics of contextualisation and positioning, including relevant ethical and cultural considerations, underpin every vertex and (3) depending upon your research frame, chosen pattern(s) of guiding assumptions and MU configuration, you may have to navigate all vertices of data triangle many times before you can build convincing meaning from your data. • Create convincing outcome(s): As a postgraduate researcher, you can produce different research outcomes to achieve different kinds/levels of impact (see the dot-dashed arrows at the bottom of Fig. 1). A research outcome typically involves creating an integrated story about your research journey that is convincing, for one or more audiences, with respect to both the quality of the work you have done and the value of what you have learned. For professional and practice-focused doctorates, a research outcome may also be an innovative product or process. Research outcomes may be prospective (e.g. a thesis, dissertation or portfolio proposal, a research grant proposal, planned innovation, policy implication) or retrospective in focus (e.g. a thesis or dissertation, a portfolio, a journal article or conference paper, technical report, presentation, realised innovation, policy or process change). The meta-criteria for research quality should prove especially useful in helping you to produce a convincing outcome. Desired impact will differ depending upon the research outcome being produced. For example, for a proposal, the immediate impact is that you are given the go-ahead to conduct your postgraduate research. For a thesis or dissertation, the immediate impact is that you convince examiners or your supervising committee that you have created important value for learning and deserve the award of the degree you are enrolled in. For a professional doctorate

xiv

Prelude: The Postgraduate Research Journey

portfolio, the immediate impact is that you convince examiners or a supervising committee that you have created important value for learning and deserve the award of the degree you are enrolled in at the same time as you demonstrate an impact on some professional or industry practice or process. For a conference paper, the immediate impact would be acceptance for a presentation or discussion at the conference and the downstream impact might be enhancement or enrichment of your professional network because of the value for learning you have generated for future potential research and collaboration. For a journal article, the immediate impact is acceptance for publication and the downstream impact is creating value for learning for others, as they cite and build upon your research. For an innovation, the immediate impact may be adoption and the downstream impact may be enhanced community or organisational benefit.

The Importance and Limits of Planning As we signalled above, your research journey can be influenced by a diverse range of forces, such as demands, expectations, constraints and opportunities, that emerge from your own context(s) (call these ‘internal forces’) as well as from sources other than yourself (call these ‘external forces’). External forces are depicted in Fig. 1 by the arrows with the question marks, and some may have the power to help shape a research activity, usually by stimulating trade-offs and/or adaptations. The implication here is that you only control some of these forces (mostly internal forces) while being controlled, directed or constrained by other forces (mostly external forces). Some of these forces can be anticipated and managed from the start, whereas others may emerge during your research journey, requiring you to adapt. The existence of external forces creates the need to consider the political and ethical implications and potential applications of your research and may strongly influence your choice of research frame. The final goal of your research journey, of course, is to achieve some outcome that speaks convincingly to one or more target audiences. Any outcome from your research may have impact on, or implications for, not only your immediate research context(s) but also for certain context(s) outside the boundaries of the immediate research project or activity. How well, and how transparently, you navigate this network of connections as well as the forces that create pressures on those connections ultimately influences how convincing your final research outcome(s) will be. Different researchers may navigate very different pathways between the various phases of research, even if they focus on the same issue, problem or topic. Thus, there is no single best way to navigate a pathway through the various phases depicted in Fig. 1; that is, there is no one right way to do social and behavioural research. To successfully complete your research journey, effective planning is essential, and this requires you to engage in systems thinking. Conduct your own thought experiment and imagine that you are conducting your research. Try to see not only the main pathways and decisions you need to make; try also to anticipate what you

Prelude: The Postgraduate Research Journey

xv

might have to do to make certain things happen or what you might have do to offset potential criticisms if you don’t do certain things. Thus, you are basically trying to see not only the ‘trees’ that comprise the elements of your research, but also the ‘forest’ within which those trees are growing. Think ahead about accessing research contexts and data sources as well as about data preparation, storage and analysis processes. Try to anticipate at least some of the internal and/or external forces that might emerge to influence your research and modify your strategic or contextual choices accordingly. Plan for your approach(es) to data analysis, at least to some extent, as certain data-gathering strategies may lead to certain analytical choices. Engaging in such systems thinking can help you to identify your own skill deficits and training/development needs (internal forces) as well as political, ethical and research user expectations (external forces) that you will need to address if your research is to have a good chance of success. However, you should expect false starts, blind alleys, complexities, reversals in fortune, unanticipated events and even emergent opportunities throughout the journey. In most cases, your journey planned will not look exactly like your journey completed, partly because you cannot anticipate everything (you, after all, are not omniscient, you are human!) and partly because you do not control every aspect of your project. In this light, a good practice for effective research planning is to plan as well and as thoroughly as possible, but also plan to be flexible and adapt to changes and unanticipated events, which may eventually mean letting go of aspects of or the entirety of your original plan if circumstances require. Planning for your research journey is intimately connected to but also constrained by your research goals and questions, access to needed resources, the research frame and pattern(s) of guiding assumptions you adopt, research quality criteria and meta-criteria and any decisions and sacrifices you make along the way. Effective research planning cannot occur in a vacuum nor can effective planning anticipate everything that might happen along the way. As we mentioned earlier, it is important to realise that the research journey is different for every researcher. A research project is just one episode in your life (a very absorbing, time-intensive, tiring and potentially rewarding episode, to be sure) and the journey you take will be an intensely personal one. A high degree of involvement and personal investment in your research is essential for (but not determinant of) your success. However, the downside of this level of commitment is that it may make letting go of aspects of your research plan, if circumstances demand, more difficult. Cultivating flexibility and an appreciation for diversity and pluralism in patterns of guiding assumptions and strategy choices is one important way to combat the sometimes-paralysing effects of feeling like you must give away an important part of your research plan because it simply isn’t working or is no longer feasible to implement. The adaptive researcher is one who can (1) see other pathways along the journey to their end goal besides the one originally planned and (2) is willing to switch paths when required, even if it takes your journey in a different direction than you originally intended. Our goal in this book is to support your development into an independent, flexible and adaptive researcher.

xvi

Prelude: The Postgraduate Research Journey

The Structure of This Book The chapters in this book are each linked to one or more of the phases of your research journey. Figure 2 makes these links transparent. In our chapters, we will answer a range of important questions that emerge along your research journey. We know these questions emerge because of our own extensive experiences as postgraduate supervisors as well as from feedback and input we received during numerous postgraduate development workshops we have been involved with over many years. We found that many of these questions were not clearly addressed in standard postgraduate research training experiences, which is why they were uppermost in many postgraduates’ minds. This is the reason the chapter titles are in the form of questions. Working through these chapters can help you to avoid some of the pitfalls and understand how to make better adaptive choices when the unexpected happens. The answers, tools, frameworks and strategies we offer can help demystify and smooth the journey a bit for you. At the same time, your research toolkit will be greatly enlarged and enriched as our goal is to ensure you become alive to the need for respecting diversity and pluralism in approaches to social and behavioural research. How and in what order you work through the chapters depends upon where you are along your research journey. If you are considering undertaking a postgraduate research programme, it would be advisable to start with Chaps. 1 through 8 to get a feel for what lies ahead before you apply to a programme as these chapters explore how you can fit research into your life. Then once you have been accepted into a programme, work through the rest of the book in chapter order. If you are in the very early stages of your journey, start with Chap. 3 and work from there. If you are further along in your journey, perhaps heading towards your proposal and confirmation, then Chaps. 5, 9 through 16 are essential to work through as would Chap. 26. It is also possible, however, to cherry-pick which chapters to work through depending upon not only where you are along your journey but also on specific questions you currently have or specific problems or issues you are experiencing. For example, you may be wondering whether to publish some aspect of your research with your supervisor during your journey and this should lead you to explore Chaps. 4, 23 and 25. If you are wrestling with contextualisation issues and/or with making your research feasible to undertake, Chaps. 9 through 12 should be of great benefit. If you are concerned about navigating the ‘Data Triangle’, then Chap. 14 along with Chaps. 17 through 21 would be of benefit, followed by Chap. 22 which focuses on telling a convincing story. No matter how far along you are in your research journey, you will find Chap. 3 to be an essential read. It is never too late to begin recording your journey in your own research journal.

Prelude: The Postgraduate Research Journey

xvii

Chapter 1: Why am I doing this and what should I expect? Chapter 2: What skills do I need? Chapter 3: How should I record my research journey? Chapter 4: How should I manage my relationship with my supervisor(s)? Chapter 5: How should I manage the research project? Chapter 6: How should I manage my time? Chapter 7: How do I stay on track? Chapter 8: How do I maintain a good work/life balance?

Establish/maintain a place for research in your life Prepare for your research journey

Contextualise, frame & position your research Configure your research activities Present a convincing proposal

Chapter 9: Why should I think about guiding assumptions? Chapter 10: How should I contextualise & position my study? Chapter 11: How do I frame & conceptualise my research problem & questions? Chapter 12: How do I scope, shape & configure my research project? Chapter 13: How should I select, read, and review the literature? Chapter 14: What data gathering strategies should I use? Chapter 15: How do I handle academic integrity issues? Chapter 16: How should I shape and defend my proposal?

Access/connect with your data sources

The “Data Triangle”

Implement your data gathering strategies

Build meaning from your data

Evolving context(s) within which your research is embedded

Create convincing outcome(s)

Innovation Thesis/ Publication/ Dissertation Conference/ Portfolio Paper

Chapter 17: How can I gain access to data sources? Chapter 18: When and how should I deal with measurements? Chapter 19: How do I manage the sampling process? Chapter 20: How should I organise my data? Chapter 21: How should I approach data analysis and display results? Chapter 22: How should I tell a convincing story? Chapter 23: How should I respond to criticisms that arise? Chapter 24: How will my thesis/dissertation/portfolio be examined and judged? Chapter 25: Should I publish as I go? Chapter 26: Common pitfalls

Fig. 2 The research journey linked to the chapters in this book

Contents

1

2

Why Am I Doing This and What Should I Expect? . . . . . . 1.1 Why Am I Doing This? . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Examining Your Motivations . . . . . . . . . . . . 1.2 How Do I Go About Choosing Where to Enrol? . . . . . 1.3 Researching Where to Study . . . . . . . . . . . . . . . . . . . 1.3.1 Considering Supervision . . . . . . . . . . . . . . . 1.3.2 So How Do I Go About Locating a Good Supervisor or Supervisory Panel Members? . 1.3.3 The Pre-meeting . . . . . . . . . . . . . . . . . . . . . 1.3.4 Meeting with Potential Supervisors . . . . . . . 1.3.5 Follow-up . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Getting Registered . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4.1 What Are They Looking for? . . . . . . . . . . . 1.4.2 What Is Required for Admissions Documentation? . . . . . . . . . . . . . . . . . . . . . 1.4.3 How Do I Go About Financing My Doctoral Studies? . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

1 2 3 7 9 12

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

13 14 14 17 17 18

.....

19

..... ..... .....

20 22 24

What Skills Do I Need? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Skills for Postgraduate Research . . . . . . . . . . . . . . . . . . . 2.1.1 Do I Have What It Takes to Get a Postgraduate Research Degree? . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 So, What Is a PhD? . . . . . . . . . . . . . . . . . . . . . 2.2 Prior Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 What Have Already Been Identified as Key Skills? . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Key Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.. ..

27 27

.. .. ..

27 29 30

.. ..

30 32

xix

xx

Contents

2.3.1

What Are the Top 8 Skills Needed to Successfully Complete a PhD? . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 What Additional Skills Are Needed to Successfully Complete a Professional Doctorate? . . . . . . . . . . . 2.4 Skills Audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 How Can I Assess My Skills? . . . . . . . . . . . . . . . 2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix: Ph.D. Skills Questionnaire (Source: Authors) . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

33

. . . . . . .

43 45 45 46 47 48 52

3

How Should I Record My Research Journey? . . . . . . . . . . . 3.1 The Research Journal . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Important Purposes for a Research Journal . . . 3.1.2 Potential Contents for Your Research Journal . 3.2 Mechanics for Maintaining Your Research Journal . . . . 3.3 When to Record in Your Journal . . . . . . . . . . . . . . . . . 3.4 Illustrations of Research Journal Entries . . . . . . . . . . . . 3.4.1 Using the Contents of Your Journal . . . . . . . . 3.5 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

55 55 56 57 57 62 63 68 69 70

4

How Should I Manage My Relationship with My Supervisor(s)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Learn What the Supervisory Requirements Are for Your Postgraduate Program . . . . . . . . . . . . . . . . . . 4.2 Before You Meet—Finding/Attracting Supervisor(s) . . . . . . 4.2.1 How Do I Go About Finding Supervisor(s)?—Seeking Commitment First . . . . . 4.2.2 How Do I Go About Attracting Supervisor(s)?—Applying First . . . . . . . . . . . . . . 4.3 Beginning the Relationship(s)—‘Backgrounding’ and Clarifying Expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 ‘Backgrounding’ . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 How Do I Go About Clarifying My Expectations? . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Maintaining Supervisory Relationship(s) . . . . . . . . . . . . . . . 4.4.1 How Do I Keep the Ship Afloat and Relationship(s) Healthy? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.2 Handling Emerging Problems . . . . . . . . . . . . . . . 4.4.3 What Can/Should I Do if My Supervisor and I Can’t Agree About Issues Involved with My Research or My Supervisor Gives Me Incorrect Advice? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

73

. .

73 74

.

74

.

75

. .

76 76

. .

81 85

. .

85 89

.

89

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

Contents

xxi

4.4.4

What Can/Should I Do if I Have a Supervisory Team Where the Supervisors Don’t Get Along or Have Contrary Views/Opinions/Paradigm Preferences? . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.5 What Can/Should I Do if My Supervisor, in My View, Is Making Unrealistic or Unacceptable Requests or Demands? . . . . . . . . . . . . . . . . . . . 4.4.6 What Can/Should I Do if Interpersonal Conflicts Develop with a Supervisor? . . . . . . . . . . . . . . . . 4.5 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

..

90

..

91

.. .. ..

91 92 95

How Should I Manage the Research Project? . . . . . . . . . . . . . . . 5.1 Planning the Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 How Can I Manage My Study to Be as Efficient as Possible? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Research Planning Tools . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 What Are the Benefits of Planning Your Journey as a Project? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Stage One (Six Months Prior to Enrolment) . . . . . . . . . . . . 5.3.1 What Can I Do Before I Actually Enrol to Adequately Prepare Myself? . . . . . . . . . . . . . . . . 5.4 Stage Two (Year 1, Part-Time/Year 1, Semester 1, Full-Time) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 What Should I Expect in My First Year? . . . . . . . 5.4.2 What Should I Do If I Think that I Am Studying in the Wrong Place? . . . . . . . . . . . . . . . . . . . . . . 5.4.3 What Is Provisional Registration/Probationary Candidature? . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.4 What Other Activities Should I Expect in Stage Two? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.5 How Do You Generate a Research Topic? . . . . . . 5.4.6 How Do I Make Sense Out of the Wash of Ideas that I Have for My Research? . . . . . . . . . . . . . . . 5.4.7 How Do I Develop Research Questions? . . . . . . . 5.5 Stage Three (Year 2, Part-Time/Year 1, Semester 2, Full-Time) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 What Does a Conceptual Framework Look like? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.2 What Is an Interim Report? . . . . . . . . . . . . . . . . . 5.6 Stage Four (Year 3, Part-Time/Year 2, Semester 1, Full-Time) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. .

97 97

. .

98 98

. .

98 127

.

127

. .

128 128

.

128

.

129

. .

129 131

. .

131 132

.

134

. .

135 135

.

136

xxii

Contents

5.6.1

What Are the Main Influences on Data Gathering Strategies? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.2 What Is Meant by Operationalising Concepts? . . . 5.6.3 How Do I Ensure Data Quality? . . . . . . . . . . . . . 5.6.4 What About Pilot Testing or Trialling? . . . . . . . . 5.7 Stage Five (Year 4, Part-Time/Year 2, Semester 2, Full-Time) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8 Stage Six (Year 5, Part-Time/Year 3, Semester 1, Full-Time) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8.1 What Is Involved with Checking Your Thesis/Dissertation/Portfolio? . . . . . . . . . . . . . . . . 5.9 Stage Seven (Final Six Months) . . . . . . . . . . . . . . . . . . . . . 5.9.1 What Do I Need to Do to Finish This Darn Thing? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9.2 As a Part-Time Student, If I Were to Take Time Off from My Employer for Concentrated Study, When Would Be the Best Time(s)? . . . . . . . . . . . 5.9.3 What if I Want to Suspend My Studies? . . . . . . . 5.9.4 What Happens if I Want to Terminate? . . . . . . . . 5.10 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.11 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

How Should I Manage My Time? . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Managing Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 So, Where Does My Day Go? . . . . . . . . . . . . . . . 6.2 Planning and Prioritising . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 But What If I Am Not a ‘List’ Person? . . . . . . . . 6.2.2 But I Need the Panic of an Immediate Deadline Before I Start Anything? . . . . . . . . . . . . . . . . . . . 6.3 Scheduling Research Sessions . . . . . . . . . . . . . . . . . . . . . . 6.3.1 How Do I Find Extra Time When I Have a Pretty Full Life as It Is? . . . . . . . . . . . . . . . . . . . . . . . . 6.3.2 What Additional Considerations Are There for Scheduling Research Sessions? . . . . . . . . . . . . 6.4 Breaking Up a Work Session . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Why Break a Session into Smaller Units? . . . . . . 6.5 Taking Breaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.1 But I Like Wandering About, How Can I Deal with That? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.2 But I Often Feel that I Don’t Deserve a Rest Break and Need to Keep Going! . . . . . . . . . . . . . 6.6 Allocating Tasks Within a Session . . . . . . . . . . . . . . . . . . .

. . . .

136 137 137 138

.

139

.

141

. .

142 143

.

143

. . . . . .

144 144 145 145 147 149

. . . . .

151 151 151 153 154

. .

155 155

.

156

. . . .

157 159 159 160

.

161

. .

161 162

Contents

xxiii

6.6.1

How Should I Organise the Tasks Within a Session? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6.2 My Problem Is Not Working; It Is Knowing When to Stop a Task! . . . . . . . . . . . . . . . . . . . . 6.6.3 So, When Do I Stop a Task? . . . . . . . . . . . . . . . 6.6.4 But How Can I Avoid Getting Held Up, Not by Me, But by Waiting for Others? . . . . . . . . . . 6.7 Establishing a Study Environment . . . . . . . . . . . . . . . . . . 6.7.1 How Do I Go About Creating an Ideal Work Environment? . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7.2 What If I Would Prefer to Work from Home? . . 6.7.3 Make Sure Your Work Station Is Ergonomically Effective for You . . . . . . . . . . . . . . . . . . . . . . . 6.8 Working in Different Locations . . . . . . . . . . . . . . . . . . . . 6.9 Organisation of Materials . . . . . . . . . . . . . . . . . . . . . . . . . 6.9.1 What Are Some Suggestions for Organising My Literature and Accumulated Materials? . . . . 6.9.2 How Do I Handle Unsolicited Material That Is Sent to Me? . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.9.3 What About Organising My Research Data? . . . 6.9.4 How About the Organisation Around My Thesis/Dissertation/Portfolio Material? . . . . . . . . 6.10 Dealing with Interruptions and Distractions . . . . . . . . . . . 6.10.1 How Do I Deal with People Coming into My Work Space and Interrupting Me? . . . . . . . . . . . 6.10.2 How Do I Deal with Distracting Myself from My Work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.12 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

How Do I Stay on Track? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Staying the Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 Do Most People Actually Finish? . . . . . . . . . . 7.1.2 What Commonly Gets in the Way and Inhibits Completion? . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Strategies to Stay on Track . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Use Your Planning Tools and Deadlines . . . . . 7.2.2 Deal with Procrastination . . . . . . . . . . . . . . . . 7.2.3 Avoid Becoming Too Isolated . . . . . . . . . . . . . 7.2.4 Don’t Hide but Engage with Your Supervisor . 7.2.5 Face Problems Head-On . . . . . . . . . . . . . . . . .

..

162

.. ..

165 165

.. ..

166 167

.. ..

167 168

.. .. ..

168 170 171

..

172

.. ..

174 174

.. ..

175 176

..

177

. . . .

. . . .

178 179 181 182

... ... ...

183 183 183

. . . . . . .

184 185 185 187 192 194 197

. . . . . . .

. . . . . . .

xxiv

Contents

7.2.6 Be Persistent . . . . . . . . . . . . . . . . . . . . 7.2.7 Start Writing Early on and Keep Writing 7.2.8 Be Conscious of Security . . . . . . . . . . . 7.2.9 Present at a Conference . . . . . . . . . . . . . 7.2.10 Check in with Your Motivations . . . . . . 7.2.11 Evaluate Your Productivity . . . . . . . . . . 7.2.12 Reward Yourself . . . . . . . . . . . . . . . . . . 7.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Key Recommendations . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

How Do I Maintain a Good Work/Life Balance? . . . . . . . . . . . 8.1 Maintaining a Work/Life Balance . . . . . . . . . . . . . . . . . . . 8.1.1 What Constitutes Work/Life Balance? . . . . . . . . 8.1.2 So Why Is Finding Some Sort of Balance Between These Life Dimensions Important? . . . . 8.2 Employment Dimension . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.1 But What if I Am a Full-Time Student? . . . . . . . 8.2.2 How Do I Go About Turning Down Requests for Additional Commitments? . . . . . . . . . . . . . . 8.2.3 What if I Want to Shift from Full-Time to Part-Time Study or Even Suspend My Study? . . 8.3 Mental Well-Being Dimension . . . . . . . . . . . . . . . . . . . . . 8.3.1 Why Do I Sometimes Feel Out of Control and Really Stressed? . . . . . . . . . . . . . . . . . . . . . 8.3.2 What Exactly Is Stress? . . . . . . . . . . . . . . . . . . . 8.3.3 How Do I Recognise When I Am Stressed? . . . . 8.3.4 How Can I Manage My Stress? . . . . . . . . . . . . . 8.3.5 What Are Some of the Tactics for Managing Stress? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.6 What Should I Be Aiming for? . . . . . . . . . . . . . 8.3.7 What Are Ways of Dealing with Frustration and Anger? . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Physical Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.1 How Can I Ensure My Physical Well-Being While Studying? . . . . . . . . . . . . . . . . . . . . . . . . 8.4.2 What Are Some Strategies for Improving One’s Well-Being? . . . . . . . . . . . . . . . . . . . . . . 8.4.3 Why Is Exercise so Important, When All I Want to Do Is Work? . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Social Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

198 199 205 206 206 207 209 210 212 213

.. .. ..

215 215 216

.. .. ..

216 217 218

..

220

.. ..

221 222

. . . .

. . . .

223 223 224 225

.. ..

226 231

.. ..

231 232

..

232

..

233

.. ..

236 236

. . . . . . . . . .

Contents

xxv

8.5.1 8.5.2 8.5.3

I Am a Single International Student; How Does This Social Dimension Affect Me? . . . . . . . . I Am a Married Student; How Does This Social Dimension Affect Me? . . . . . . . . . . . . . . . . . . . . . I Am a Student with Children; How Does This Social Dimension Affect Me? . . . . . . . . . . . . . . . .

238

. . . .

. . . .

. . . .

239 241 241 244 245

Why Should I Think About Guiding Assumptions? . . . . . . . . . 9.1 What Are Guiding Assumptions? . . . . . . . . . . . . . . . . . . . 9.2 Exploding Some Misconceptions . . . . . . . . . . . . . . . . . . . 9.3 Make Your Guiding Assumptions Clear . . . . . . . . . . . . . . 9.3.1 Arguments for Guiding Assumptions . . . . . . . . . 9.4 How Do Guiding Assumptions Relate to Quantitative and Qualitative Data Preferences? . . . . . . . . . . . . . . . . . . 9.5 Engaging Pluralist Logic—Moving Beyond ‘Mixed Methods’ Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6 Guiding Assumptions Are Associated with ParadigmSpecific Quality Criteria . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.1 Quality Criteria Associated with the Positivist Pattern of Guiding Assumptions . . . . . . . . . . . . 9.6.2 Quality Criteria Associated with Interpretivist/ Constructivist and Other Non-positivist Patterns of Guiding Assumptions . . . . . . . . . . . . . . . . . . 9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.1 Convincingness . . . . . . . . . . . . . . . . . . . . . . . . 9.7.2 Three Domains of Meta-Criteria . . . . . . . . . . . . 9.7.3 Working with the Meta-Criteria . . . . . . . . . . . . . 9.8 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

247 247 257 260 262

..

267

..

267

..

276

..

276

..

280

. . . . . .

. . . . . .

283 285 287 297 300 301

.. ..

305 306

..

308

.. .. ..

312 313 320

8.6 Spiritual . . . . . . . . . . . 8.7 Conclusion . . . . . . . . . 8.8 Key Recommendations References . . . . . . . . . . . . . . . 9

236

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

10 How Should I Contextualise and Position My Study? . . . . . . . . 10.1 Why Is Contextualisation Important? . . . . . . . . . . . . . . . . 10.2 What Kinds of Contextualisation Strategies Can I Employ in My Research? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 What Might Your Contextualisation Strategies Be Linked To? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 Researcher Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Positioning with Other’s Research . . . . . . . . . . . . . . . . . .

xxvi

Contents

10.6

Positioning of Participants and Other Data Sources . . . . . . . 10.6.1 Positioning of Human Participants . . . . . . . . . . . . 10.6.2 Positioning of Indigenous Participants . . . . . . . . . 10.6.3 Positioning of Non-human Data Sources . . . . . . . 10.7 Focusing in on Issues Relevant to Your Specific Research Context(s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.1 Political Contextual Influences in and/or on Your Specific Research Context(s) . . . . . . . . . . . . . . . . 10.8 Illustrative Contextualisation and Positioning Arguments . . . 10.9 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11 How Do I Frame and Conceptualise My Research Problem and Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Choice of Research Frame . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 Action Research Frame . . . . . . . . . . . . . . . . . . 11.1.2 Evaluation Research Frame . . . . . . . . . . . . . . . 11.1.3 Development Evaluation Frame . . . . . . . . . . . . 11.1.4 Case Study Research Frame . . . . . . . . . . . . . . 11.1.5 Survey Research Frame . . . . . . . . . . . . . . . . . . 11.1.6 Descriptive Research Frame . . . . . . . . . . . . . . 11.1.7 Exploratory Research Frame . . . . . . . . . . . . . . 11.1.8 Explanatory Research Frame . . . . . . . . . . . . . . 11.1.9 Cross-Cultural (Cross-National or Comparative) Research Frame . . . . . . . . . . . . . . . . . . . . . . . 11.1.10 Indigenous (Indigenist) Research Frame . . . . . . 11.1.11 Feminist Research Frame . . . . . . . . . . . . . . . . 11.1.12 Transdisciplinary Research Frame . . . . . . . . . . 11.2 What Constitutes a Researchable Problem? . . . . . . . . . . . 11.3 What Tools Can I Use to Help Me Identify and Clarify My Research Problem? . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 The Emergence of Your Research Questions/Hypotheses . 11.4.1 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.2 Research Questions . . . . . . . . . . . . . . . . . . . . . 11.4.3 A Generic Illustration . . . . . . . . . . . . . . . . . . . 11.5 Some Concrete Examples from Recent PhDs and Professional Doctorate Portfolios . . . . . . . . . . . . . . . 11.6 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

327 327 333 335

.

336

. . . .

340 343 345 346

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

349 349 352 354 357 359 361 365 367 368

. . . . .

. . . . .

. . . . .

371 374 377 380 383

. . . . .

. . . . .

. . . . .

389 398 400 401 402

... ... ...

403 412 414

12 How Do I Scope, Shape and Configure My Research Project? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Scoping and Shaping Your Research: Working Within Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

417 418

Contents

xxvii

12.1.1 Research Scoping and Shaping Choices . . . . . . . . Potential Configurations of MUs—A Unifying Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 The Method Unit (MU) . . . . . . . . . . . . . . . . . . . . 12.2.2 Simultaneous Configuration . . . . . . . . . . . . . . . . . 12.2.3 Sequential Configuration . . . . . . . . . . . . . . . . . . . 12.2.4 Hierarchical Configuration . . . . . . . . . . . . . . . . . . 12.2.5 Case-Based Configuration . . . . . . . . . . . . . . . . . . 12.2.6 Longitudinal Configuration . . . . . . . . . . . . . . . . . 12.2.7 Hybrid Configuration . . . . . . . . . . . . . . . . . . . . . 12.2.8 Evolutionary Configuration . . . . . . . . . . . . . . . . . 12.3 Making Trade-Offs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4 Harnessing Synergies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 Building a Conceptual/Theoretical Framework—Do I Need One? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.1 Conceptual Frameworks . . . . . . . . . . . . . . . . . . . 12.5.2 Theoretical Framework . . . . . . . . . . . . . . . . . . . . 12.6 Trialling Your MU Research Activities for Navigating the ‘Data Triangle’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.7 How Can You Reflect Research Scoping, Shaping and Configuring in Your Writing? . . . . . . . . . . . . . . . . . . . 12.8 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

428

. . . . . . . . . . .

438 439 440 443 446 449 452 457 465 468 476

. . .

478 479 484

.

486

. . .

492 498 499

.. ..

503 503

..

504

..

505

..

507

.. ..

507 508

..

510

.. ..

510 512

12.2

13 How Should I Select, Read and Review the Literature? . . . . . . 13.1 The Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.1 I Just Want to Get on with My Study So Why Can’t I Do the Literature Review Concurrently with Data Gathering? . . . . . . . . . . . . . . . . . . . . 13.1.2 So, What Is the Purpose of a Literature Review? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.3 What Are the Common Problems Associated with Literature Reviews? . . . . . . . . . . . . . . . . . . 13.1.4 How Do I Go About Creating a Good Literature Review? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.1 What Strategies Do You Recommend for the Preparation Phase of the Research? . . . . . 13.2.2 What Is the Best Way of Generating Key Words? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 Searching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xxviii

Contents

13.3.1

How Do I Become Acquainted with Everything the Library Can Provide? . . . . . . . . . . . . . . . . . 13.3.2 What Questions Should I Ask? . . . . . . . . . . . . . 13.3.3 What Advice Is There for Web Searching for a Researcher? . . . . . . . . . . . . . . . . . . . . . . . 13.3.4 Can Material on the Internet Be Trusted? . . . . . . 13.3.5 Digital Searching . . . . . . . . . . . . . . . . . . . . . . . 13.3.6 What Is Back-Referencing? . . . . . . . . . . . . . . . . 13.3.7 What Is Cited Reference Searching? . . . . . . . . . 13.3.8 Finding Relevant Grey Literature? . . . . . . . . . . . 13.3.9 How Do I Judge the Quality of the Literature I Read? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.10 How Can I Ensure that I Don’t Miss Something During the Search Process? . . . . . . . . . . . . . . . . 13.3.11 When Do I Call a Halt to Searching the Literature? . . . . . . . . . . . . . . . . . . . . . . . . . 13.4 Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.1 How Do I Cope with the Sheer Volume of Reading? . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.2 What Involved with Skim and Scan Reading? . . 13.4.3 What Is Active Reading? . . . . . . . . . . . . . . . . . 13.4.4 How Do I Engage in Critical Reading? . . . . . . . 13.5 Taking Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.1 What Should I Be Doing When Making Notes on the Literature? . . . . . . . . . . . . . . . . . . . . . . . 13.5.2 Recording Your References . . . . . . . . . . . . . . . . 13.5.3 What Is the Best Way to Record My Summarising and Evaluating? . . . . . . . . . . . . . . 13.6 Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.6.1 What Is Involved with Writing up a Literature Review? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.7 Revision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.7.1 Why Do I Need to Revise? . . . . . . . . . . . . . . . . 13.7.2 How Is a Literature Review Assessed? . . . . . . . . 13.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.9 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 1: Questions You Could Ask During Active Reading . . Appendix 2: Concise Critical Notes: Articles and Papers Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 3: Meta-criteria Research Outcome Evaluation Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix 4: Literature Review Scoring Rubric . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.. ..

512 513

. . . . . .

. . . . . .

514 515 516 518 518 519

..

521

..

523

.. ..

524 524

. . . . .

. . . . .

524 525 526 527 531

.. ..

531 531

.. ..

533 535

. . . . . . .

. . . . . . .

535 541 541 542 542 545 547

..

548

.. .. ..

550 551 552

Contents

14 What Data Gathering Strategies Should I Use? . . . . . . . . . 14.1 Strategies for Connecting with People . . . . . . . . . . . . 14.1.1 Interaction-Based Strategies . . . . . . . . . . . . . 14.1.2 Observation-Based Strategies . . . . . . . . . . . . 14.2 Strategies for Exploring People’s Handiworks . . . . . . . 14.2.1 Participant-Centred Strategies . . . . . . . . . . . 14.2.2 Artefact-Based Strategies . . . . . . . . . . . . . . . 14.3 Strategies for Structuring People’s Experiences . . . . . . 14.3.1 Data-Shaping Strategies . . . . . . . . . . . . . . . . 14.3.2 Experience-Focused Strategies . . . . . . . . . . . 14.4 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . Appendix: Clarifying Experimental/Quasi-experimental Design Jargon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xxix

. . . . . . . . . . .

555 557 559 578 591 591 605 625 627 647 674

..... .....

676 677

. . . . . . . . . . .

. . . . . . . . . . .

15 How Do I Handle Academic Integrity Issues? . . . . . . . . . . . . . 15.1 What Is Academic Integrity? . . . . . . . . . . . . . . . . . . . . . 15.2 What Sort of Academic Integrity Issues Should I Be Aware of? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2.1 Intentional and Unintentional Harm . . . . . . . . . 15.2.2 Cultural Insensitivity . . . . . . . . . . . . . . . . . . . . 15.2.3 Coercion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2.4 Deception . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2.5 Loss of Confidentiality or Anonymity . . . . . . . 15.2.6 Libel and Slander . . . . . . . . . . . . . . . . . . . . . . 15.2.7 Conflicts of Interest . . . . . . . . . . . . . . . . . . . . 15.2.8 Influential Funding . . . . . . . . . . . . . . . . . . . . . 15.2.9 Inaccurate Analysis and Reporting . . . . . . . . . . 15.2.10 Not Keeping Appropriate Records . . . . . . . . . . 15.2.11 Plagiarism . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2.12 Attending to Correct Referencing . . . . . . . . . . . 15.2.13 Copyright . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2.14 Use of a Professional Editor . . . . . . . . . . . . . . 15.2.15 Misappropriation of the Outcomes of Research . 15.2.16 Inappropriate Authorship Attribution . . . . . . . . 15.2.17 Inappropriate Publications . . . . . . . . . . . . . . . . 15.2.18 Manipulation of Intellectual Property Policy . . . 15.2.19 Financial Misappropriation . . . . . . . . . . . . . . . 15.2.20 Ethical Principles . . . . . . . . . . . . . . . . . . . . . . 15.2.21 Ethical Approval . . . . . . . . . . . . . . . . . . . . . . . 15.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

... ...

689 689

. . . . . . . . . . . . . . . . . . . . . . .

691 692 694 696 697 699 702 702 703 704 705 705 706 707 708 709 710 712 714 715 716 716 720

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

xxx

Contents

15.4 Key Recommendations . . . . . Appendix: Information to Include in a Sheet . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . .

.......................

722

Participant Information ....................... ....................... . .

727 727

.

727

. . .

729 730 731

. . . . . . .

731 734 743 745 746 746 749

.

750

. . . . . . .

752 753 753 754 756 757 758

. . . . . . .

. . . . . . .

761 761 762 762 763 763 768

. . . .

. . . .

770 770 775 780

16 How Should I Shape and Defend My Proposal? . . . . . . . . . . . . . 16.1 Preparing a Research Proposal . . . . . . . . . . . . . . . . . . . . . . 16.1.1 Why Do I Have to Do a Research Proposal? I just Want to Get on with the Study! . . . . . . . . . 16.1.2 When Is a Research Proposal Usually Provided? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.1.3 The Process for Developing a Proposal . . . . . . . . 16.2 Structure of a Research Proposal . . . . . . . . . . . . . . . . . . . . 16.2.1 What Is the Normal Structure for a Research Proposal? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.2.2 Digging Deeper into the Research Proposal? . . . . 16.3 Writing Up Your Proposal . . . . . . . . . . . . . . . . . . . . . . . . . 16.4 Self-evaluation/Reflection . . . . . . . . . . . . . . . . . . . . . . . . . 16.5 Proposal Review and Approval . . . . . . . . . . . . . . . . . . . . . 16.5.1 How Does One Get a Proposal Approved? . . . . . . 16.5.2 Presenting Your Proposal . . . . . . . . . . . . . . . . . . 16.5.3 What Are Stakeholders Looking for in a Proposal and How Would They Evaluate It? . . . . . . . . . . . 16.5.4 What Are the Most Common Problems That Crop Up in Research Proposals? . . . . . . . . . . . . . 16.5.5 Proposal Approval Outcomes . . . . . . . . . . . . . . . 16.5.6 Making Changes to Your Proposal . . . . . . . . . . . 16.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16.7 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix: Research Proposal Check List . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 How Can I Gain Access to Data Sources? . . . . . . . . . . . . . . . . 17.1 Gaining Access to Data Sources . . . . . . . . . . . . . . . . . . . 17.1.1 Why Is Gaining Physical Access Difficult? . . . . . 17.1.2 What Is Actually Involved in Gaining Access? . . 17.2 Ten Steps for Gaining Access to Data Sources . . . . . . . . . 17.2.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.2.2 Identify Key Contacts . . . . . . . . . . . . . . . . . . . . 17.2.3 Determine What Is of Value to the Key Contact and Their Organisation . . . . . . . . . . . . . . . . . . . 17.2.4 Make Contact . . . . . . . . . . . . . . . . . . . . . . . . . . 17.2.5 Undertake Follow-up Meetings . . . . . . . . . . . . . 17.2.6 Engaging with Participants . . . . . . . . . . . . . . . .

723 724

Contents

17.2.7 Implement Your Data Gathering Strategies . 17.2.8 Fulfil Your Obligations . . . . . . . . . . . . . . . 17.2.9 Reporting Your Research . . . . . . . . . . . . . 17.3 Staying in Contact . . . . . . . . . . . . . . . . . . . . . . . . . . 17.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.5 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xxxi

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

780 781 782 783 783 785 786

and How Should I Deal with Measurements? . . . . . . . The Process of Measurement . . . . . . . . . . . . . . . . . . . . Operational Definition of a Construct . . . . . . . . . . . . . . Measurement Scales . . . . . . . . . . . . . . . . . . . . . . . . . . 18.3.1 The Nominal Scale . . . . . . . . . . . . . . . . . . . . 18.3.2 The Ordinal Scale . . . . . . . . . . . . . . . . . . . . . 18.3.3 The Interval Scale . . . . . . . . . . . . . . . . . . . . . 18.3.4 The Ratio Scale . . . . . . . . . . . . . . . . . . . . . . 18.4 What Counts as a ‘Good’ Measure? . . . . . . . . . . . . . . . 18.4.1 Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18.4.2 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . 18.4.3 Relationship Between Reliability and Validity 18.4.4 Measurement Sensitivity . . . . . . . . . . . . . . . . 18.4.5 Constructs, Variables and Models . . . . . . . . . 18.5 Choosing/Using a Measure Developed by Other Researchers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18.5.1 Issues to Consider Prior to Adoption . . . . . . . 18.5.2 Issues to Consider Prior to Use in Analyses . . 18.6 Developing and Validating Your Own Measure . . . . . . 18.6.1 Issues to Attend to During Development . . . . 18.6.2 Issues to Attend to Prior to Use in Analyses . 18.7 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

789 790 792 793 793 794 795 796 798 799 805 808 809 809

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

813 813 814 816 816 823 824 825

19 How Do I Manage the Sampling Process? . . . . . . . . . . . . . . . . 19.1 Key Considerations Surrounding Sampling . . . . . . . . . . . . 19.1.1 Focus on Statistical Efficiency = Precision . . . . . 19.1.2 Focus on Statistical Effectiveness = Power . . . . . 19.1.3 Focus on Statistical and Computational Viability and Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . 19.1.4 Focus on Representativeness of Sample . . . . . . . 19.1.5 Focus on Representativeness of Experiences . . . . 19.1.6 Focus on Sufficiency . . . . . . . . . . . . . . . . . . . . . 19.1.7 Focus on Contextual Relevance/Knowledge/ Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

827 827 829 832

. . . .

. . . .

836 837 838 841

..

841

18 When 18.1 18.2 18.3

. . . . . . .

xxxii

Contents

19.1.8 Focus on Transportability . . . . . . . . . . . . . . . 19.1.9 Focus on Expediency . . . . . . . . . . . . . . . . . . 19.1.10 Hybrid Synergies . . . . . . . . . . . . . . . . . . . . . 19.2 Probabilistic Sampling Strategies . . . . . . . . . . . . . . . . . 19.2.1 Simple Random Sampling . . . . . . . . . . . . . . . 19.2.2 Stratified Random Sampling . . . . . . . . . . . . . 19.2.3 Cluster Sampling . . . . . . . . . . . . . . . . . . . . . 19.2.4 Two-Stage Cluster Sampling . . . . . . . . . . . . . 19.2.5 Systematic Sampling . . . . . . . . . . . . . . . . . . . 19.3 Non-probabilistic Sampling Strategies . . . . . . . . . . . . . 19.3.1 Convenience Sampling . . . . . . . . . . . . . . . . . 19.3.2 Quota Sampling . . . . . . . . . . . . . . . . . . . . . . 19.3.3 Volunteer Sampling . . . . . . . . . . . . . . . . . . . 19.3.4 Purposive Sampling . . . . . . . . . . . . . . . . . . . 19.3.5 Snowball Sampling . . . . . . . . . . . . . . . . . . . . 19.3.6 Contextual Sampling . . . . . . . . . . . . . . . . . . . 19.3.7 Theoretical Sampling . . . . . . . . . . . . . . . . . . 19.3.8 Negative Case Sampling . . . . . . . . . . . . . . . . 19.3.9 Sequential/Saturation Sampling . . . . . . . . . . . 19.3.10 Synergistic Combining of Sampling Strategies 19.4 Constraints on Sampling Quality . . . . . . . . . . . . . . . . . 19.4.1 Plan B Sampling Strategy . . . . . . . . . . . . . . . 19.5 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

842 842 843 843 844 847 850 852 854 857 858 860 864 868 873 876 878 881 884 888 889 890 891 893

20 How Should I Organise My Data? . . . . . . . . . . . . . . . . . . . . 20.1 Preparation of Quantitative Data . . . . . . . . . . . . . . . . . 20.1.1 Coding Rules . . . . . . . . . . . . . . . . . . . . . . . . 20.1.2 Data Entry . . . . . . . . . . . . . . . . . . . . . . . . . . 20.1.3 Final Data Checking . . . . . . . . . . . . . . . . . . . 20.1.4 A Final Word! . . . . . . . . . . . . . . . . . . . . . . . 20.2 Preparation of Qualitative Data . . . . . . . . . . . . . . . . . . 20.2.1 Data Creation . . . . . . . . . . . . . . . . . . . . . . . . 20.2.2 Data Transcription . . . . . . . . . . . . . . . . . . . . 20.2.3 Memoing/Transcribing Maps and Multi-media Data Sources . . . . . . . . . . . . . . . . . . . . . . . . 20.3 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

895 896 896 900 904 905 905 905 912

.... .... ....

916 917 919

21 How Should I Approach Data Analysis and Display of Results? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.1 Preliminary Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.1.1 Analyses of Data Inform the Multi-layered Stories that Convey What You Have Learned . . . . . . . . . .

921 921 925

Contents

21.2

xxxiii

Analysing Quantitative Data . . . . . . . . . . . . . . . . . . . . . . . 21.2.1 Understanding Quantitative Analytical Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.2 What Are Some Useful Tips and Considerations for Quantitative Analysis? . . . . . . . . . . . . . . . . . . 21.2.3 What Are Some Useful Tips on Reporting Quantitative Outcomes? . . . . . . . . . . . . . . . . . . . 21.3 Analysing Qualitative Data . . . . . . . . . . . . . . . . . . . . . . . . 21.3.1 What Are Some Useful Tips and Considerations for Qualitative Analysis? . . . . . . . . . . . . . . . . . . . 21.3.2 What Does It Mean to Code Qualitative Data? . . . 21.3.3 What Are Some Useful Tips on Reporting Qualitative Outcomes? . . . . . . . . . . . . . . . . . . . . 21.4 Integrating Quantitative and Qualitative Analyses . . . . . . . . 21.5 Computer Support Systems for Quantitative and Qualitative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.5.1 Some Useful Statistical Packages . . . . . . . . . . . . . 21.5.2 Some Qualitative Data Analysis Packages (Other Packages Are also Available—See www.scolari.com) . . . . . . . . . . . . . . . . . . . . . . . . 21.5.3 Free Online Training Guides . . . . . . . . . . . . . . . . 21.6 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

22 How Should I Tell a Convincing Story? . . . . . . . . . . . . . . . . . . 22.1 Shaping the Major Outcome of Your Postgraduate Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.1.1 The Traditional PhD . . . . . . . . . . . . . . . . . . . . . 22.1.2 Professional Doctorate/Other Forms of Professional Practice & Applied Doctorate . . . 22.1.3 Doctoral Thesis by Publication . . . . . . . . . . . . . 22.2 Taking a Closer Look at the PhD Thesis (or Dissertation) . 22.2.1 What Is a Typical Structure for a PhD Thesis (or Dissertation)? . . . . . . . . . . . . . . . . . . . . . . . 22.3 Some Illustrative Chapter Structures for PhD Theses and Professional Doctorate Portfolios . . . . . . . . . . . . . . . . 22.4 How Do I Make Logical and Empirical Arguments? . . . . . 22.4.1 What Are the Different Kinds of Arguments that I Can Make? . . . . . . . . . . . . . . . . . . . . . . . 22.5 How Should You Write/Edit Chapters for a Convincing Thesis/Dissertation/Portfolio? . . . . . . . . . . . . . . . . . . . . . .

.

934

.

934

.

940

. .

945 959

. .

959 962

. .

966 982

. .

985 986

. . . .

988 990 991 999

..

1003

.. ..

1003 1004

.. .. ..

1006 1007 1009

..

1013

.. ..

1042 1047

..

1050

..

1054

xxxiv

Contents

22.5.1 What Is an Appropriate Writing Style? . . . . . 22.5.2 What About Formatting? . . . . . . . . . . . . . . . . 22.5.3 Resources to Support Your Writing Activities 22.6 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

1055 1057 1058 1059 1060

23 How Should I Respond to Criticisms That Arise? . . . . . . . . . . 23.1 Responding to Criticism . . . . . . . . . . . . . . . . . . . . . . . . . 23.1.1 What Types of Criticism Should I Expect? . . . . . 23.1.2 From Whom Might Criticism Come? . . . . . . . . . 23.2 Criticism from Your Supervisor(s) . . . . . . . . . . . . . . . . . . 23.2.1 How Can I Better Handle the Criticism that I May Receive from My Supervisor(s)? . . . . . . . 23.2.2 What Advice Should I Be Cautious of? . . . . . . . 23.2.3 What if My Supervisor Is Too Soft in Their Feedback? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.2.4 What if I Genuinely Believe that the Criticism Is Unfair? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.2.5 What Are Some Further Strategies for Handling Supervisor Criticism? . . . . . . . . . . . . . . . . . . . . 23.3 Criticism from Your Colleagues . . . . . . . . . . . . . . . . . . . . 23.3.1 How Do I Go About Critiquing Others? . . . . . . 23.4 Criticism from Strangers . . . . . . . . . . . . . . . . . . . . . . . . . 23.4.1 How Do I Handle the Criticism of a Stranger Whose Comment Doesn’t Appear Relevant? . . . 23.5 Criticism from Yourself . . . . . . . . . . . . . . . . . . . . . . . . . . 23.5.1 What Can I Do to Minimise My Self-criticism? . 23.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23.7 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

1063 1063 1064 1064 1065

.. ..

1066 1069

..

1070

..

1070

. . . .

. . . .

1071 1073 1074 1076

. . . . . .

. . . . . .

1076 1077 1077 1078 1080 1081

....

1083

....

1084

....

1084

....

1086

....

1086

24 How Will My Thesis/Dissertation/Portfolio Be Examined and Judged? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.1 Approaches to the Thesis/Dissertation/Portfolio Examination Process . . . . . . . . . . . . . . . . . . . . . . . . . . 24.1.1 What Are the Different Approaches to the Examination Process? . . . . . . . . . . . . . . . . . . 24.2 Examination of Your Final Major Research Outcome by Your Supervisor(s) . . . . . . . . . . . . . . . . . . . . . . . . . 24.2.1 What Is the Role of Supervisor(s) in the First Stage of the Examination Process? . . . . . . . . .

. . . . .

Contents

24.3

Formal Examination Processes . . . . . . . . . . . . . . . . . . . 24.3.1 Examination of Your Final Major Research Outcome by an Internal Panel/Committee . . . . 24.3.2 Examination of Your Final Major Research Outcome by Independent External Examiners . 24.3.3 Examination of Your Final Major Research Outcome via an Oral Defence or Viva . . . . . . 24.4 Examination Outcomes . . . . . . . . . . . . . . . . . . . . . . . . 24.4.1 What Are the Possible Outcomes of the Examination Process? . . . . . . . . . . . . . . . . . . 24.4.2 Do People Actually Fail? . . . . . . . . . . . . . . . 24.4.3 What Is the Most Common Examination Outcome? . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.4.4 What if I Have to Make Changes Before My Thesis/Dissertation/Portfolio Will Be Passed? . 24.4.5 What if I Must Revise and Re-Submit My Thesis/Dissertation/Portfolio Following the Examination Process? . . . . . . . . . . . . . . . . . . 24.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.6 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . Appendix: Illustrative Rubric for Assessing a Research Masters or PhD Thesis/Dissertation . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xxxv

....

1087

....

1087

....

1088

.... ....

1101 1105

.... ....

1105 1106

....

1106

....

1106

.... .... ....

1107 1108 1110

.... ....

1111 1115

25 Should I Publish as I Go? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25.1 Generating Research Outputs . . . . . . . . . . . . . . . . . . . . . . . 25.1.1 Should I Publish During My Research Journey? . . 25.1.2 What Are the Disadvantages of Publishing While Doing My Postgraduate Study? . . . . . . . . . . . . . . 25.1.3 What Are the Advantages of Publishing While Doing My Postgraduate Study? . . . . . . . . . . . . . . 25.2 Common Resistance to Publishing as a PhD Student . . . . . . 25.2.1 What Are the Common Excuses Students Use not to Publish? . . . . . . . . . . . . . . . . . . . . . . . 25.3 Publishing Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25.3.1 What Could I Publish? . . . . . . . . . . . . . . . . . . . . 25.4 The Publishing Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 25.4.1 What Are the Steps for Getting My Topic into Published Form? . . . . . . . . . . . . . . . . . . . . . . . . . 25.4.2 What Are the Common Reasons for a Paper Being Rejected? . . . . . . . . . . . . . . . . . . . . . . . . . 25.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . .

1117 1117 1118

.

1118

. .

1119 1121

. . . .

1121 1122 1122 1126

.

1126

. .

1131 1131

xxxvi

Contents

25.6 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1132 Appendix: Example of a Paper Outline . . . . . . . . . . . . . . . . . . . . . . 1133 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1134 26 Common Pitfalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.1 Common Pitfalls for Postgraduate Students . . . . . . . . . . . . 26.1.1 From a General Perspective, What Are Common Pitfalls for Postgraduate Students? . . . . . . . . . . . 26.1.2 What Are Some Common Pitfalls Specifically in Relation to Research Configuration and Execution? . . . . . . . . . . . . . . . . . . . . . . . . . 26.2 Vacillating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.2.1 What Is Vacillating? . . . . . . . . . . . . . . . . . . . . . 26.2.2 So, How Do I Get Some Focus and Direction? . 26.3 Not Taking Responsibility . . . . . . . . . . . . . . . . . . . . . . . . 26.3.1 I Understand What not Taking Responsibility Means but How Can I Avoid the Problem? . . . . 26.4 Losing Your Confidence . . . . . . . . . . . . . . . . . . . . . . . . . 26.4.1 What Do You Recommend for Dealing with a Loss of Confidence? . . . . . . . . . . . . . . . . . . . . 26.5 Having Problems with Writing . . . . . . . . . . . . . . . . . . . . . 26.5.1 What Are the Common Writing Problems? . . . . 26.5.2 Any Useful Tips for Helping Me with My Writing? . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.6 Not Demonstrating Critical Thinking . . . . . . . . . . . . . . . . 26.6.1 How Can I Judge Academic Quality? . . . . . . . . 26.6.2 What Are the Most Common Criticisms of Academic Output? . . . . . . . . . . . . . . . . . . . . . . 26.7 Being Scared of Developing Theory or Meaningful Interpretations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.8 Not Fully Identifying the Original Contribution of Your Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.8.1 If I Find Someone Else Is Doing a Similar Study, Will that Erode My Potential for Original Contribution? . . . . . . . . . . . . . . . . . . . . . . . . . . 26.8.2 How Do I Demonstrate Originality of My Research? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.9 Not Realising that Doing a Postgraduate Research Degree Is as Much About Lifestyle Changes as It Is About Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.10 Not Establishing a Workable Relationship Early on with Your Supervisor(s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.. ..

1137 1137

..

1137

. . . . .

. . . . .

1138 1140 1140 1141 1142

.. ..

1143 1144

.. .. ..

1144 1145 1145

.. .. ..

1146 1149 1150

..

1151

..

1152

..

1153

..

1153

..

1154

..

1156

..

1157

Contents

26.10.1 What if My Supervisors Disagree with Each Other? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.11 Not Planning for What Will Happen After Your Postgraduate Research Journey Is Complete . . . . . . . . . . . . . . . . . . . . . . 26.11.1 What if I Am Currently in Employment, but Acquiring This Qualification Has Prompted Me to Move to Another Position? . . . . . . . . . . . . . . . 26.11.2 What if I Want a Full-Time Job in Academia? . . . 26.12 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.13 Key Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xxxvii

.

1159

.

1160

. . . . .

1160 1161 1164 1166 1167

Chapter 1

Why Am I Doing This and What Should I Expect?

When a friend got married, the Minister who was performing the service required them to have pre-marriage counselling. During the first session he had the couple list all the reasons why they wished to marry. They quietly wrote as they considered why they wished to embark on this life journey with specifically this person. Upon completion of the exercise, the Minister told them to keep the list in a safe place, and preferably accessible, as they would serve as an excellent reminder when times get tough as to the reasons why they did actually marry. Fortunately, the friend has said that she hasn’t had to pull out the paper to re-acquaint herself with the rationale for her choice of lifelong partner. However, the activity seemed like a sensible one and, on reflection, there are parallels with preliminary considerations you might also undertake before embarking on a postgraduate research journey. On many future occasions you may find yourself questioning your decision to pursue what can be a lengthy commitment. For this reason, it is helpful to know, from instigation, what your motivations are for undertaking such an immense project and to gain some clarity around a few key areas, such as: • • • • • •

being realistic about the goal that you have set; having an understanding of yourself; the expectations of the institution you will be studying in; the nature of the supervisory relationship; the means by which you might support yourself; and the effect your prolonged study may have on your friends and family.

At the outset, it is worth mentioning that this text is intended primarily for doctoral-level research students for whom, in addition to the possibility of preparatory coursework, there is the expectation of a substantial research project and the preparation of an expository research outcome, such as a thesis. However, students undertaking research master’s degree will also benefit from what we explore in this book. We acknowledge that doctoral degrees now come in many different forms depending on the field and profession involved (Gill & Hoppe, 2009). There is the traditional PhD, which typically results in the submission of a thesis and, depending © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_1

1

2

1 Why Am I Doing This and What Should I Expect?

upon the country in which you are undertaking the award, you may (e.g., in the U.S) or may not (e.g., U.K, Australia) be required to complete coursework prior to commencing your research. There are an increasing number of professional and practice-based doctorates which are now being pursued around the world (Lee, Brennan & Green, 2009; Maxwell, 2003; Neumann, 2005). Professional doctorates challenge the existing epistemology of a traditional PhD (self-contained, disciplinecentred, appealing primarily to an academic audience) because the research typically requires contextualised professional, practice-based and/or community interaction, innovation and focus, often with the goal of creating research impact (Bourner, Bowden, & Laing, 2001; Green, Maxwell, & Shanahan, 2001; Fink, 2006). The research outcome from a professional or practice-based doctorate can take the form of a dissertation or a research portfolio that speaks to multiple audiences (Maxwell & Kupczyk-Romanczuk, 2009). Similarly, some students are now undertaking a doctoral degree by publication (Badley, 2009; Jackson, 2013; Nethsinghe & Southcott, 2015), where the research outcome comprises a compendium of published works woven together with contextualising narratives. Despite these variations in approach, students will be actively engaged in project planning, research and the writing process. This text is therefore, intended to assist all those students embarking on a research endeavour for which there will be a considerable commitment of time in pursuit of an intellectual outcome of a dissertation, collection of papers, portfolio or thesis. In addition to this text, there are a number of useful books that you may also find of value in relation to undertaking your research journey (Brown, 2006; Brown, McDowell, & Race, 1995; Burton & Steane, 2004; Churchill & Sanders, 2007; Collis & Hussey, 2014; Cryer, 2006; Delamont, Atkinson, & Parry, 2000; Denholm & Evans, 2012; Dinham & Scott, 1999; Dunleavy, 2003; Elphinstone & Schweitzer, 1998; Fisher, 2010; Graves & Varma, 1997; Marshall & Green, 2007; Petre & Rugg, 2010; Phillips & Pugh, 2015; Powell & Green, 2007; Quinton & Smallbone, 2006; Race, 2007; Rudestam & Newton, 2015; Salmon, 1992; Thomas & Brubaker, 2007; Wellington, Bathmaker, Hunt, McCulloch, & Sikes, 2005).

1.1

Why Am I Doing This?

The extent of the commitment required should not be underestimated and a postgraduate student who is setting the goal to obtain a doctorate needs to be realistic as to what is required of them, and the impact that a postgraduate program may have on themselves and others. If you take the approach “Oh, I’ll enrol and see how it goes”, then you are probably destined to failure, as that level of engagement is just not strong enough to sustain you when you start to hit some of the emotional, methodological, financial or time management bumps along the way. Interestingly, for some students, they are secretly reluctant to set the goal of getting a doctorate as they confess that they see the goal as too lofty and unobtainable and that, in some way, they are unworthy or don’t have what it takes. It is therefore appropriate to bust a few illusions here. The truth is that, as most experienced supervisors will tell

1.1 Why Am I Doing This?

3

you, getting a doctorate is more dependent on organisation and persistence than on brilliance. So, yes, you can do it, but you need to be dedicated to the completion of your project from the beginning. Most people are understandably hesitant about embarking on doctoral studies. As one successful researcher has reflected “I was scared that I may not be up for it, or that I would waste three years of my life, or worse I would quit half way through disappointing all the people involved. Luckily, a dear friend of mine came up with a good piece of advice. He simply said: “Forget about the others and take this opportunity to invest in yourself” (Aliotta, 2011). So, go ahead and set the goal, write it down, and give yourself a realistic date for completion. The time lines will be approximately three to four years if you are planning to be a full-time student and possibly even up to six years if you intend to be studying part-time. To be realistic, now add one more year from a standing start, that is, with no preparation, for pre-registration and registration activity at the beginning, as well as submission and graduation procedures at the end. Once you have established the time lines and the goal, share that goal with others who will be supportive of you such as close family and friends. You will need to have them firmly on board with your plans and also committed, as they will be significantly affected by your studies as a result of your physical and sometimes emotional absence, and pre-occupation with the task. Be sure that your friends and family are aware of not just what you are getting into but also how it will impact on them and diminish the time that you have together, not just for a few months but for a number of years. If you are in employment you also need to communicate your goal of obtaining a doctorate with your manager. If the timing is right, during a performance review process is the best opportunity of raising the subject, as some concessions, such as time release, may be required in the future, particularly at the beginning and the end of the project. Getting early organisational commitment to your personal goal will be helpful.

1.1.1

Examining Your Motivations

With goal setting there is a difference between what you want to achieve and why you want to achieve it. What you want to achieve is the bigger picture of what the goal is; but why is one step before the goal. The “why” are the deep inner feelings that spur your commitment and motivation, therefore, you are encouraged to have good insight into why you want to accomplish the goal as this is the main well from which you will draw your motivation, particularly when you get into the doldrums about mid-way. To be able to draw on this motivation, take time to reflect over a few days and be brutally honest with yourself as you explore all the reasons behind your desire to embark on such a significant project. In examining why students want to obtain a doctorate, Lawton (1997, p. 3) suggests that students start by examining their aspirations and long-term goals. Lawton found that motives can vary considerably between not knowing what else to do in one’s life, remaining as a student,

4

1 Why Am I Doing This and What Should I Expect?

enjoying the academic environment, having the intention to pursue a career in teaching and research, having a particular problem or research issue they wish to investigate, acquiring research skills, or even a desire to achieve the specific personal accomplishment of the highest academic qualification that can be given. Motivation to undertake a professional doctorate might be stimulated by a need to solve a concrete problem in context, influence or change existing practices or develop and evaluate an innovation; in short, to have an impact in or on one’s profession or industry. It has been observed that there is surprisingly little research on what motivates people to start and persevere through a doctorate, and even less about what effect gaining a doctorate subsequently has on their life (Brailsford, 2010; Hegarty, 2011). In their study Leonard et al. (2005, p. 135) noted that their respondents gave more than one reason why they undertook their doctoral studies. For more than a third it was for professional development, or a requirement for their job, although this was rarely first on their list of motives. For example, “I was working as a research assistant and it seemed the next logical step in my career progression as a researcher. Also, I was offered a research job which provided the opportunity to do the doctorate alongside, using some of the data collected as part of the larger project” (Leonard, Becker, & Coate, 2005, p. 138). A third of respondents undertook a doctorate because of having an interest in a particular area, while a quarter mentioned personal development, including the pleasure of learning, testing themselves, gaining confidence, proving themselves, self-fulfilment and the joy of study. For example, “I was stuck at home with three kids under eight, desperate to use my mind but, logically, unable to take on a job as well” (Leonard et al., 2005, p. 138). Other reasons provided were acquisition of research skills such as statistical and writing skills, as well as being able to supervise students in the future. A few international students indicated their motivation was the ability to study while living abroad and improving their language skills. There are a myriad of reasons why you may wish to engage in postgraduate study and they will vary from individual to individual (Fung, Southcott, & Siu, 2017). For international students there are what have been called push (from the source country) factors and pull (drawn to a host country or institution) factors (Mowjee, 2013). For simplicity let us divide the motivations into intrinsic and extrinsic factors. Intrinsic factors relate to motivations that are linked to your feeling of self-worth, or your wish to acquire and/or apply new knowledge, achieve a personal bucket list goal, investigate a critical issue, be a role model to children while extrinsic factors tend to relate more to what postgraduate study attainment might actually achieve or do for you, that is, in the form of better job prospects, better pay, higher status, family recognition, having an impact on professional, organisational or community practice (seeking change for the better), and for international students this could also be exposure to a new culture and possibly migration opportunities. Looking at these reasons in more detail may prompt recognition of your motives. Taking extrinsic motives first, they are usually the easiest to identify and primarily relate to enhanced career choices and the expectations of the occupation you wish to

1.1 Why Am I Doing This?

5

pursue. This is more notable for those who wish to enter, or to stay in, an academic environment, where a doctorate has long been recognised as the entry qualification necessary for a career in the sector and where, without a doctorate, career progression is extremely limited. However, for one of our students, he had no intention of pursuing a teaching career but felt strongly that, as an independent business consultant dispensing advice, the qualification would give him the necessary kudos to get through the door to acquire clients and the credibility to communicate his recommendations. Similarly, for one professional doctorate student, his primary motivation was to improve his organisation’s performance by creating, implementing and evaluating an innovation that could capture ideas for organisational improvements from the staff and volunteers that he managed as Fire and Emergency Services Commissioner in Western Australia. The desire for higher status is probably one of the weaker motivations to hang on to as the prestige associated with obtaining a doctorate can be fairly short-lived. If you are working in academia then you would have noted that virtually everyone has a PhD and the recognition you receive on an aircraft when asked for your assistance as a doctor, soon turns to embarrassment as you indicate you are probably next to useless in an emergency (interestingly, no-one is ever in urgent need of a methodological framework or a business plan). Intrinsic motivations have more of a personal element and, in addition to self-development; there is the intellectual challenge as well as the personal satisfaction and a sense of accomplishment. As one PhD student in her late 30s commented, “I have never really had a career plan but what I have done is set personal goals. I got an undergraduate degree and then a master’s so the next level of personal accomplishment seemed to be a PhD. I didn’t start out with the intention of doing a PhD; it just seemed like the next step if I wanted to get that great feeling of personal achievement. The actual topic was secondary”. A similar sentiment was expressed by another student who reflected “well I guess the primary motivation was the Himalaya syndrome. It was there, it was the final step if you like to a university career as it was available to me and I wanted to do it, it was that last jump” (Brailsford 2010, p. 21). For others, the topic and the intellectual curiosity are their prime motivators, as one student commented, “a strong motivating factor was certainly the wish to come to grips with a defined, substantive body of knowledge and to develop my critical and creative faculties—essentially, reasons to do with personal development” (Connell cited in Dickinson, Connell & Savage, 1997, p. 121). As Francis (1997, p. 18) has pointed out, doing a significant piece of research “changes people, not simply in terms of the technical expertise and knowledge gathered in their field, but also in terms of the way they value themselves and their work”. Upon the completion of your PhD you will become an expert in your area. This has been identified as almost an unavoidable consequence of working for many years exclusively on a specific topic and you may even exceed your supervisor in the knowledge that you have developed (Aliotta, 2011). Intrinsic motivation is considered the dominant factor in the motivation of adult students in continuing postgraduate education (Templeton, 2016). Dinham and Scott (1999, p. 19) identified intrinsic motivation as being more common than

6

1 Why Am I Doing This and What Should I Expect?

extrinsic, when 60 percent of the students they surveyed gave what could be termed as intrinsic reasons for undertaking their doctorate. Mueller et al. (2015) also found that students are significantly attracted by intrinsic motivations such as life aspirations and the intellectual challenge of gaining a doctorate for the intention of further study. It is thought that tapping into one’s intrinsic motivation contributes to a greater level of commitment (Ryan & Deci 2000) with those that have the motivation of personal development obtaining higher learning achievement (Lee & Pang, 2014; Thunborg & Brubaker, 2013). The most common reason given for undertaking a doctorate is a desire to study in greater depth an already interesting and well-liked field of study. However, there is usually not one but a combination of reasons behind the desire to do a doctorate, and during your moments of self-reflection it may be worthwhile listing your reasons. Make it personal, for example, when wanting to lose weight the goal may be the number of kilos by a certain time, but the motivation is because we want to look and feel great. As one American student confided, “in hindsight, I think one of the main reasons I successfully completed the PhD was the fact that I didn’t pass the exam (preliminary doctoral written exam) on the first try. It’s ironic, but life sometimes works in strange ways. That initial failure caused me to answer the basic question [why I was doing this] and provided the mental fortitude to keep going, despite the hurdles and problems I would later face” (Azuma, 2017). Motivations are heightened when they are personal and very dear to our heart. Let’s be honest, there has been many a family member who has been motivated to achieve success in study or a career in order to out-do a sibling. What is imperative is that the motivation(s) is important to you. A Middle Eastern postgraduate student who was struggling with the decision as to whether to do a PhD recently indicated the primary motivation for postgraduate study was family pressure and the need to fulfil family expectations. Intriguingly, he stated that every adult member of his immediate living family had a PhD, including his grandmothers! It does, however, need to be your motivation not someone else’s desire or expectation if it is going to fuel you over an extended period of time. The motivation can be quite personal. For example, one of our students expressed dissatisfaction with their current job and the need for a career change. Another older student, who had been lecturing for many years, indicated their fear of being without a job as the requirement for a doctoral qualification increasingly became apparent as a necessity to stay in their role. The reality is that on your list of motivations there will be a mix of both intrinsic and intrinsic motivations. This is not unusual. Similar to the results found by Mueller et al. (2015), Wiegerová (2016) observed that young PhD students are influenced by both extrinsic and intrinsic motivations specifically relating to their institution, family and their wish to continue to study and research at a postgraduate level. In fact, Templeton (2016) went so far as to indicate that people with only intrinsic motivations do not have enough to keep committed to the program. If you have identified your motivations, write then down and keep them handy. Yes, they are pretty personal, don’t worry, no-one is going to ask you to share them, but it is important that you identify the buttons you are going to need to push along the way to keep you moving forward. You may need to repeat this process

1.1 Why Am I Doing This?

7

periodically as it is not uncommon to find one’s motivation for undertaking a postgraduate research qualification change as you transition through the process. A young female student shared how her motivation changed; “when I expressed my goal of doing a PhD with others, an older male colleague who didn’t have a doctorate chided me and questioned whether I was up to it. It was probably only a throwaway line but, wow, did that motivate me. I was shocked at his comment and, while I can’t say that the emotion stayed with me through the whole process, to be honest, it certainly did sustain me for quite some time in the early years of my research as I wanted to show him I could do it. However, in the latter stages of my study I was more motivated by a need to capitalise on the significant investment of time and resources that I had put into the project. I wanted something to account for all that effort.” So your motivations may change over time as you progress through the research journey, however, you do need to be in touch with what your primary motivations are so that when times do get difficult and your commitment wanes, you are able to remind yourself of why you are doing it in order to get another injection of enthusiasm to carry you over until the next challenge.

1.2

How Do I Go About Choosing Where to Enrol?

Closely related to the choice of where to enrol is the question of whether you should continue your postgraduate education in the university where you undertook your prior qualifications, or should you change to an entirely new institution? As one student concluded, “the decision to return to my Alma Mater was deliberate for, despite the great distance from home and the logistics problems implied; I particularly liked and felt comfortable in London. I had always found it conducive to study, it was where the bulk of the primary resources were located, and I was familiar with the geography of the town and the university. Perhaps, above all, I was keen to work with people I knew and trusted” (Dickinson cited in Dickinson, Connell & Savage, 1997, p. 116). There may be some ethnic preferences as Smith (2007) found that for minority ethnic students in the UK ‘familiarity breed contentedness’ with student concerned with proximity as well as staying with a previously experienced institution for their postgraduate studies. While it is sometimes tempting to undertake your doctoral studies in the same environment in which you undertook your undergraduate or Masters qualification, because of the desire to remain in a secure environment where you know the people, the library and the topography but, from an academic recruiter’s point of view, there is commonly a subtle desire to see some variation in the degree awarding institutions. A movement to another institution and the successful completion of a qualification demonstrates a degree of mobility and adjustment, while staying in one’s home institution does promote questions of overt patronage. You may, therefore, wish to demonstrate your flexibility by seeking another institution to do your doctoral studies. If,

8

1 Why Am I Doing This and What Should I Expect?

however, you are constrained by geographic location, that is, there is only one institution in your town, funds are too limited to pursue a research program at another institution or, you have family commitments which restrict your ability to move away to study, alternatively, the expertise in your field resides in your home location, clearly, you would be wise in the face of those constraints to stay where you are. Taking a moment to consider the choice of institution further, and with an eye for what will enhance a CV, some students’ desire is to attend the most prestigious institution they can get into, and they are hopeful that there will be an appropriate supervisory match with their intended topic. Others may place greater emphasis on the supervision and will enrol in an institution where they perceive that the greatest level of knowledge is in their subject area. For part-time students, the choice of institution may be driven by pragmatics as they seek the most accessible institution to their work or home while, for others, they may reside in a small city in which there may be only one immediate choice of tertiary institution. Reflect on these alternative perspectives when choosing where to study, and consider which approach resonates with you. Interestingly, Dinham and Scott (1999, p. 24) have provided a ranking of the reasons most commonly given by Australian doctoral students when choosing a university at which to complete their degree. These are in rank order: • • • • • •

Geographic proximity to family Quality of the doctoral program Desire to work with a particular supervisor The reputation of the university Financial assistance provided Having worked or studied there previously.

In a similar discussion, Das (2015) outlined 7 factors potential doctoral students should consider when choosing research institution for their study: the match of the research interests of the candidate and the research activities of the faculty, available funding, peers and co-workers with whom the candidate will be interacting, the publication track record in the department, the supervisors working style and expectations, other areas with a related research focus, and the prevailing work ethic. A point worth noting is that undertaking postgraduate research is an independent activity and, given the power of email and other forms of digital communication, as well as external access to library resources, proximity to your data sources is probably more important than proximity and the location of your supervisor(s) and home University. Our further recommendation is that where you do have choices, keep your options open and do due diligence on more than one institution by initially obtaining information and then, if possible, visiting the institution. For international students, the latter may mean telephone calls and/or internet investigation. Start this process at least six months to a year before you intend to enrol.

1.3 Researching Where to Study

1.3

9

Researching Where to Study

List the institutions that interest you and, initially, look up information on them on the internet. In the first instance, as most universities appear in ranking tables start broad and see where the institutions are on any comparative league tables, although noting international assessments tend to be broad assessments of the institution rather than discipline-specific. The following websites may be of value for this purpose: • https://www.timeshighereducation.com/world-univresity-rankings • https://www.usnews.com/best-colleges/uiuc-1775/overall-rankings • https://www.topuniversities.com/university-rankings/world-university-rankings/ 2018 • https://www.socialcapitalgateway.org/eng-rankings.htm. If you want to look specifically at the research standing of the universities that you are interested in, then you may need to locate national databases. For example, for universities in the U.K., the UK Research Excellence Framework assesses the quality of research in universities and colleges in Britain and reports the findings (https://www.ref.ac.uk). To obtain information on actual doctoral programs and to undertake further evaluation of the research within your intended department, as well as the individuals within that department, you will need to become more institution-specific and look at actual university websites. However, be aware that, as most people do not like seeing large chunks of text on a website, the information may be limited. Don’t be swayed by the lovely pictures of students demonstrating ethnic and gender diversity and having fun in the classroom, as these are often either stock photos sourced by the promotions department, or orchestrated photos of selected staff and students. Surprisingly, searching organisational websites is not as easy as one would expect but what you are initially looking for is an indication of: • The structural layout of the organisation—what are the faculties and departments? Is there a graduate school? • Who are the key personnel, such as the Director of the Doctoral Program and the Head of Department in the discipline area you wish to study? • What existing research projects are being undertaken by staff and students? • Who are the key researchers and what are their areas of research interest? • Is there a student union and/or student support services for graduate students? • What is the nature of the doctoral program? • Is there a downloadable map of the campus for use when you visit? • What organisational and industry links do they have as this may be useful for future data collection? Start up a file on each institution you are interested in and phone the course information office to get further documentation. Realise, however, that educational marketing usually works on two or three levels. Given the cost of printing the first

10

1 Why Am I Doing This and What Should I Expect?

level of material, it is usually a beautiful “teaser” brochure which has great pictures but not much detail. The more robust information is contained in the second and third levels of promotional material. This information is often easier to obtain when you actually visit the campus and engage with the program staff. If you are able to visit then don’t only go to the course information centre but also try to connect with the actual academic department where your subject area is taught, as you will find them more knowledgeable about the program. If possible, endeavour to visit the institution twice, the first time to get a feel for the place and the resources, and the second time to actually meet with key people with whom you have made an appointment, people such as the program directors and academics who are in the field you are interested in studying. Obviously if you are an international student visiting the university may not be possible so see if you can locate someone who has studied the same program there before and now lives by you. The alumni office might be helpful in making that connection for you. In getting more detailed information on the program find out not only the obvious, such as what, if any, course work is required, you should also obtain information on the relevant policies and regulations that govern the degree you would be enrolling in, as well as on registration requirements, the examination process, the processes for ethics approval and so on. This information should be documented and available as part of the program regulations, however, it doesn’t often sit on the promotional racks, so you may have to ask for it, indicating that you are looking into studying at the institution. It is here we should make an emphatic note on the importance of establishing a good relationship with the administrator/department secretary and staff. They are the backbone of any academic department and are particularly knowledgeable. Make a point of learning their names and establish rapport. I learnt this from personal experience. When I arrived at the LSE seeking relevant information, the department secretary was in a real flap as she was trying to organise a postgraduate function for 5.00 pm and was way behind in the preparation. I could see that I wasn’t going to get any sensible answers from her so, as most antipodeans would do, I offered to help, rolled up my sleeves, opened umpteen bottles of wine, set up tables and played bartender. Later, when she had calmed down she came over to me and said, “now what can I do for you?” From then on, she was immensely helpful and continued to provide all manner of insight and support for the next few years until her retirement. The next stage in your investigative process is to identify what resources might be available to a postgraduate research student, such as, in the form of study locations—do they have study spaces and how busy are they; the availability of software, database, equipment, external library access, library training. Do they charge for inter-library loans? Do they have holdings related to your area of interest? What is the function of the research office or postgraduate office and, more importantly what support programs do they offer. Greenspon (2013), a first year PhD student in Applied Physics, made a sage observation that it is more about the importance of the structure of the program, the support available, the facilities and the location rather than the reputation of the institution. It may be especially useful

1.3 Researching Where to Study

11

to visit the campus library to check out the resources it offers. As well, you could check into their holdings of prior postgraduate research theses, dissertations and/or portfolios in your area of interest and have a look at some of these. A further location to review as part of your investigative process is the relevant student union. They can alert you to any issues that may have arisen in the past in the academic department where you intend to study, as well as also giving further insight into some of the supporting mechanisms available for postgraduate students at the institution. Most institutions, either through the student union or the postgraduate centre, will also have a scholarships office where personnel can provide you with advice regarding scholarships. If you are an international student check out the international office and ask about their pastoral care program for overseas students. While you are on site, it would also be a good idea to try to interact with some of the other postgraduate students, who will be able to share their experiences regarding the institution, the program, the administration, the support services and to comment on potential supervisors. Postgraduate students talk amongst each other regarding their supervisors, so it’s not too difficult to get an indication of the reputations of various academic staff in relation to their supervision, whether they are hands-on/hands-off, around or not around, and so on. A quick word to other postgraduate students can often provide some useful information although, remember, it is anecdotal and may not be entirely accurate, so treat such information as only one possible piece of the informational jigsaw when determining who might be an appropriate supervisor. If time is limited, then ask the postgraduate students you meet for their email addresses and follow up later as this may be more appropriate and will also give you more time to ask them about their research. Having thoroughly digested the course information on the second visit, make an appointment with the doctoral program director/dean of postgraduate students (or a similar title) as well as one or two academic staff whom you believe could be potential supervisors. The purpose of your meeting with the program director should be to clarify any questions you have from reading the program documentation and to obtain additional information that is unlikely to be found in the written material. Specifically, you may wish to: • Determine how large the department is in terms of numbers of students, faculty members and administrators. Larger departments tend to have more resources and support training, however, if it is too large, there is the potential that you could get lost amongst a sea of other graduate students. • Find out the program completion rate. This information is now being collected more routinely because it is related to government funding, so don’t be fobbed-off when they say they don’t have the information available. • Determine the procedures for aligning students with a supervisor, and/or supervisory committee and gauge to what extent a postgraduate student can request a specific supervisor. Also inquire whether it is common to have more

12

•

• • •

• • • •

1 Why Am I Doing This and What Should I Expect?

than one supervisor, and whether the secondary or co-supervisor can be in another institution. Know how supervision time is allocated, that is, the usual time allocation for an academic staff member who is supervising a doctoral student. Ascertain whether there is a maximum number of students an academic staff member can supervise. While there may not be a standard answer, there may be a range. The fact that you have asked the question does alert the department to appropriateness of supervision as being an important consideration for postgraduate students before embarking on their study. Express your interest in the program and provide a quick précis of your intended research area. Identify the names of potential supervisors in your likely topic area whom you will later visit and talk to. Should a communication breakdown occur, or the relationship deteriorate between a supervisor and their postgraduate student, ascertain the mechanisms available for resolving the situation. In addition to the academic supervisor, find out whether there is a mentor or buddy system operating in the department. What resources are available to postgraduate students? Is there support for data collection and attendance at conferences? What are the policies governing processing time lines for getting ethics approval? Also ascertain what interim reports are required to qualify for candidature or during the course of study. Identify the next steps and time frames in regard to registration. Ask whether they could suggest names of current, or even past, postgraduate students who may be studying in a similar area to you.

1.3.1

Considering Supervision

They say that one of the keys to being successful in life is first to choose your parents very carefully. Well, the same applies to supervisors or supervisory panel/ committee members but, unlike choosing your parents; this time around you might actually get to have some influence on the selection of individuals in a critical relationship which could significantly impact on your well-being and the likelihood of completion of your doctoral studies. Remember, not all academics are necessarily good at, or like, supervising postgraduate students. Some academics find postgraduates distract them from their own research, while others consider them to be resources available to assist their own research, which could ultimately distract you from your research. In a blistering account of their experience published online in the Times Higher Education in 2013 (and has since been taken down), one student commented “I found the supervisors remote and odd. A couple of them tried

1.3 Researching Where to Study

13

to block the submission of the theses to my institution. Indeed, on three separate occasions in my career, academics informed me that if I submitted this thesis, it would fail. The results that followed these warnings were a Master of Arts passed with distinction, a Master of Education with first-class honors and a dean’s award, and a PhD passed without correction. I was left with the impression that these supervisors had no idea what they were doing. The worst supervisors share three unforgivable characteristics: they do not read your writing, they never attend supervisory meetings, they are selfish, career-obsessed bastards”. Fortunately, many consider good supervision an important and part of their job and are dedicated to supporting their students in achieving their doctoral aspirations. These individuals meet regularly with their students, are diligent in their review of material, provide constructive feedback, are open to theoretical and methodological choices that vary from their own and constructively mentor their students. It is distracting, time-consuming and embarrassing to have to change supervisors, so you want to make sure you get the right one/s at the beginning of the project. In some circumstances where you are applying from a distance and assigned a supervisor on the basis of your topic, the decision is made for you. For other students, there is the potential for some investigation and personal selection of a supervisor or at least a short list of preferred supervisors. In those circumstances you will be active in identifying your supervisor(s).

1.3.2

So How Do I Go About Locating a Good Supervisor or Supervisory Panel Members?

It is widely acknowledged that successful doctoral candidature relies heavily on the supervisor-candidate relationship (Nethsinghe & Southcott, 2015) so getting the right person or persons is important. In some Universities rather than a single supervisor you could be appointed a primary and secondary supervisor even a supervisor panel/committee made up of senior academics. In the later instance, it is not uncommon to have the panel members allocated to you and for the members to have a spread of expertise related to your research. However, if you are able to have a say in who might oversee your doctoral supervision then here is some advice you might find useful. When looking for a suitable supervisor(s) let’s divide the process into three stages: (1) The pre-meeting stage which entails topic reflection and preliminary investigations; (2) Meeting with potential supervisors to ascertain the likely personal and academic fit; and (3) The final follow-up stage where you express common courtesies and a subtle indication of your supervisory preferences.

14

1.3.3

1 Why Am I Doing This and What Should I Expect?

The Pre-meeting

At the beginning of postgraduate study students do differ in their entry point, with some students having a very firm idea of what they want to do, the research questions and the approach they wish to take, while others are more ambiguous, even in regard to the topic area and possibly having a number of alternative fields of interest they are considering. To focus your search for potential supervisors, it is a good idea to spend some time reading and deliberating in order to narrow down your topics of interest to a general field of study. Having decided on the discipline and/or problem area(s) of interest to you, go to the university website and look up associated academic staff and their specific expertise. From this list you will be able to identify an academic who may be interested in your intended research or, alternatively, academics who are currently undertaking research which is of interest to you. You may also have identified some key academic staff mentioned to you after talking to the program director and administrative staff. If time and costs permit, it is also a good idea to attend a local conference in your field of interest, as the attendee list may yield local academics that could make potential supervisors. Another source of potential supervisors is http://www.scholar.google.com using key words related to your study, your country location or your preferred country location. Not only will it generate names of people researching in your field of interest, but you will also be able to source current papers on the topic. Prior to meeting a specific individual, it is also useful to use Google Scholar (enter the academic’s name) to read the full range of relevant papers/abstracts they have written. That way will you come across as being reasonably informed and with a strong indication that you have done your homework.

1.3.4

Meeting with Potential Supervisors

Once you have isolated a number of key individuals who could be potential supervisors, make an appointment for an informal discussion with them regarding their research interests, and your interest in doctoral study in their area of expertise. As Peters (1997) has succinctly stated, “My own gut feeling is that you have got little to lose and much to gain by writing a respectful letter or making a pleasant phone call. The worst that can happen, provided you don’t come off as a complete dolt, is that the professor won’t be helpful. Indeed, the professor’s response or lack of one can tell you something about whether or not he has the time or temperament to be a good advisor” (Peters 1997, p. 40). Yes, this often requires cold-calling, as you don’t know the person, but that is a skill you need to develop if you are going to gain access to research data sources and also to future, potential employers. When you do meet with potential supervisors indicate your interest in postgraduate study and, potentially, in doing research in their area. Introduce into the conversation, a past paper that the academic has published to demonstrate your

1.3 Researching Where to Study

15

propensity for the subject area and that you have done some background checking. Academics love to talk about their work and what they are doing so, by having a discussion with the supervisor regarding their past and current research; it may also widen your scope of potential research topics and questions. In the case of professional doctorate programs, one requirement might be for a non-academic practitioner or industry/profession person to be appointed to a supervisory team. You would want to have a conversation with relevant people to gauge their interest in and capacity for joining your supervisory team (they may reside, in fact, in your own workplace and you might need to converse with them about the implications of that). In these instances, you would be talking, not about their own research, but about their contextualised views of problems and possibilities for where applied, practice-based, professionally-focused research might be needed. In particular, your conversation might focus on areas where both you and they think changes, improvements and/or innovations are required. About 20 min into this conversation is where you may then be able to shift the direction of the conversation towards your research ideas. When meeting the potential supervisor and communicating your ideas for your doctoral research, try not to come across as too dogmatic. The supervisor is there to guide you in your research journey and, should they feel that you are inflexible or indifferent to their suggestions, that will flag to them that, potentially, you may be a difficult student. Possible questions to ask of a potential academic supervisor: • A question to which it is sometimes difficult to find a reliable answer and, hence, one you should ask, not only of the program director but also of a potential supervisor, is what is the time allocated by the department for lecturing staff who are supervising postgraduates? • How many postgraduate students is the supervisor currently supervising? How many have completed, and did their students complete on time? Experienced supervisors get people through, so you want someone who has done this many times before. • Have they had any students who have not completed? Naturally enough, not all doctoral students complete their study but, should a supervisor have a disproportionate number of students who have dropped their studies, this may indicate poor interaction between the supervisor and student. • What are their expectations of a student in the first year of their doctoral study? Where do they expect you to be after the first year will give a good indication of their understanding of the research journey and will also give you clear expectations. • How frequently do they anticipate meeting with a doctoral student? You are looking for a response that preferably indicates weekly or fortnightly meetings and not the experience of one student who recounted “I knew a supervisor whose idea of supervision was a once-a-semester meeting in a bar where he would order three bottles of red wine and start drinking. The meeting ended when the wine finished”.

16

1 Why Am I Doing This and What Should I Expect?

• Last year, how many research-related international trips did they do? This may sound like an odd question but what you are trying to ascertain is whether your prospective supervisor will actually be around to supervise you. High-flying researchers often have great reputations but may rarely be on the ground and available to you. • Are they intending any significant long-term absences from the country, a sabbatical or study leave for instance, or long service leave or retirement within the next three to six years? • How would they describe their supervising style (supervisory styles will be discussed in more detail in Chap. 4)? How does that style change during the different phases of doctoral study? You want someone who is engaged and will also hold you accountable to milestones. • What are their views on a student publishing or presenting at a conference during their study, or would they prefer the student to complete their studies and then publish? What are their views on co-authoring? You don’t want a supervisor who will expect automatic co-authorship on all your work. • Can they give an example of a time that they had to advocate for their student? You want someone that is prepared to go to bat for you if necessary. • How would they reconcile differences of opinions with co-supervisors? If you are assigned a co-supervisor, it is imperative that they are on the same page or life can get very difficult as you are left to arbitrate. • What are their key industry connections? These may be helpful when looking for data sources. • What grants to they currently hold or intend to apply for? This may be potentially positive for you if you are able to align your research and be employed as a research assistant on an externally funded project. Possible questions to ask of a potential non-academic supervisor: • Have they had any experience supervising other professional doctorate students? • How much time could they devote to working with you in a supervisory capacity and would they be happy to participate in supervisory team meetings with academic supervisors. • If your project generates useful intellectual property, such as an innovation or change process, how would they view the balance in allocation of intellectual property rights between student, university and the industry, profession and/or the organisation in which the research is situated? • What kind of resources could they provide you with access to? • What kinds of project feedback would they like to receive that might be shared with other members of the industry, profession and/or the organisation? • Who, in their industry, profession and/or the organisation, do they think might be useful for you to contact with respect to your research project and intentions? Always be respectful of their time and keep the meeting to around 45 min. Don’t ask a potential supervisor general questions relating to the program. Such

1.3 Researching Where to Study

17

information should have been obtained previously from relevant administrators or program documentation.

1.3.5

Follow-up

Out of courtesy, email and thank the potential supervisor for their time and input. It is a nice touch to consider providing something of intellectual value. For example, if you discussed a recent paper that you have read and that possibly the potential academic supervisor was not aware of; you may wish to provide a copy or link, as most academics are always interested in new material in their field. From your prior discussions with the program director, you will have some indication of how the allocation of supervisors is undertaken in the department. Be mindful of this process and that you can, in no way, demand that a particular staff member is your supervisor, but you can politely indicate to the program director a preference which you would like to be taken into consideration should you be accepted onto the program. However, remember that just because you like that supervisor, it doesn’t necessarily mean that they will be available for you, as most universities operate now with work load models. That supervisor may, in fact, already have a number of postgraduate students and is, therefore, not able to take any more on. Similarly, the decision to appoint that person as a supervisor is often done by the level above, that is, their manager or Head of School in conjunction with the research/postgraduate office and the potential supervisor. Where there is more than one supervisor to be allocated, the skills that each supervisor will bring need to be balanced and consideration given as to how they might work with each other.

1.4

Getting Registered

Following getting the documentation on regulations for your intended qualification, you will have become fully acquainted with the program expectations, time lines and so on. You will have met with potential supervisors and spoken with other postgraduate students, so, the final stage in the pre-registration process is completion of the relevant applications and, possibly, formal interviews. This is not a job that should be rushed, and you may be required to assemble ancillary information such as academic transcripts, letters of recommendation and maybe a tentative research proposal. All of this will require some co-ordination. If you are not sure what is being requested, ask. A simple call to an administrator (the one that you have established rapport with) may iron out an issue that could otherwise hold you back. While some institutions allow applications any time in the year, others are more specific given that course work will commence at the start of each semester. Provide all information accurately and attach all requests for supporting documentation. It really

18

1 Why Am I Doing This and What Should I Expect?

annoys administrators when they get incomplete information and have to initiate contact with a potential student, and request material that should have been provided in the first instance. Where possible, it is a good idea to append an example of your own writing, either a previous research paper you have written or some other document you have produced in a university context, which can show your capabilities in academic writing. In many cases, you will be interviewed by the department or graduate school before being admitted as a doctoral student and, quite possibly, will be interviewed by the doctoral program director whom you would have met previously. This meeting will be more formal than your last, and the interviewer will now be the main person asking questions (previously, it was you). At this stage, it would be ideal if you have a broad topic area that could be framed into a research question(s). Without being dogmatic, and remaining open to the learning process, some students will have an indication of likely data sources as well as some thoughts on the research frames (e.g., action research, evaluation research, survey research; see Chap. 11 for more details) and data gathering strategies they favour (e.g., questionnaires, interviews, experiments, observation; see Chap. 14 for more details). Demonstrating more than a passing acquaintance with the existing body of knowledge and related theories will impress your interviewer. It is, therefore, suggested that you prepare well for the meeting in order that the interviewer will have confidence in your ability to perform the task of completing the qualification.

1.4.1

What Are They Looking for?

Universities are looking for dedication to completion of a postgraduate degree. Universities are judged on their postgraduate ‘completion rates’ and, therefore, wish to enrol students into the program who have a high probability of successfully progressing at the right pace, completing all required course work and submitting an appropriate research outcome. They will be looking at your academic qualifications and any prior research you may have undertaken, as well as your personal commitment. It is very sad for all concerned when a student drops out of doctoral studies, given the considerable investment in time by all parties. Therefore, those deciding who is accepted into a doctoral program are looking for individuals who not only can develop the necessary research skills at that level, but who will also persevere and last the distance. Reflect on your past experiences and what circumstances you have overcome where you have previously committed to and completed a significant project. It may be at the master’s level, or it may be some work you have undertaken in your private life. They are looking for a student who will weather the storms, stick with the project over a number of years, and can meet the challenges inherent in all stages of the postgraduate research journey. At the next level they will also be looking at your receptivity to new concepts, ideas and approaches, and for intellectual enquiry and flexibility in order that your research can be developed. One further point worth mentioning, which is often

1.4 Getting Registered

19

overlooked by doctoral candidates, is that, while undertaking doctoral research is an autonomous activity undertaken on an individual basis with the assistance of one or more supervisors, you will also, by association, be part of the academic group within the department. They will, therefore, also be looking at you from the perspective of what you would be like as a member within a community of learners, and how you will interact with others given the numerous opportunities for graduate student seminars and workshops during the course of your study. Are you someone who will not only successfully complete your own research, but will you also be supportive and encouraging of others, and will you contribute in a positive way to the research environment of the department or school? If you are undertaking a doctoral program at a distance education university, they may look for your commitment to periods of attendance on campus each year to meet with your supervisor (s) and contribute to seminar programs and the like. Some universities may be influenced by your potential research topic, whether it is related to the research themes or foci of the department. They are particularly attracted to topics that are seen as being hot or emerging areas of interest. Finally, they will look at your prior academic record and particularly your grade point average if you are applying for a scholarship. If you are doing a professional doctorate, people in the work or professional setting in which you want to do your research may need to be convinced as to your intentions and wonder what they will get out of the project. They may harbour suspicions as to your or your university’s motives. They may look for some commitment to continuing dialog between themselves, you, as candidate and the academics involved. In short, undertaking a professional doctorate commits you to having multiple conversations with multiple stakeholders, not just university staff members. Be prepared to have to deal with potentially conflicting demands as well as to take advantage of any emergent opportunities and synergies. Obviously, the university will remain the primary stakeholder in your research as you will, hopefully, be selected for and enrolled in a program they control, and the university will ultimately award the degree to you, if you are successful. However, you must also realise that professions, organisations, communities and/or workplaces, who stand to potentially be influenced or even changed by the research you do, are also key stakeholders, whose interests must be safeguarded as well.

1.4.2

What Is Required for Admissions Documentation?

It is not uncommon for admission documentation to require you to provide a covering letter or essay to support your application. This can be somewhat daunting but not, however, if you put yourself in the shoes of the person reading it. Provide them with: • a statement of why pursuing this qualification aligns with your career goals or personal aspirations;

20

1 Why Am I Doing This and What Should I Expect?

• a statement of your field(s) of research interest and, in the case of a professional doctorate, what impacts you’d like your research to have in an industry, profession, organisation, community and/or workplace; • a statement of how your research interest may align with the school or department and, in the case of a professional doctorate, with a particular industry, profession, organisation, community and/or workplace; • evidence of your prior academic, and where appropriate, relevant workplace, industry or professional, experience; • a statement highlighting the logistics and support available to you that enhances your likelihood of completion; and • evidence of your personal traits of diligence, persistence and interpersonal skills. If you draft out a paragraph from each of the above statements, you will quickly have a reasonably well-structured supporting document. Be mindful, however, that if the application requires answers to specific questions, rather than a more general approach, you would be advised to align your responses to the specific questions being asked. Keep the statements simple and focused. Hi-brow comments such as, “the advancement of knowledge for the betterment of mankind” can sound pompous and unrealistic. Have your documentation read over by someone else to check for any typos, spelling errors or omissions. Remember, they will be looking at your ability to write and what might be the contribution of your research to the domain of study. Commonly, your application will require supporting recommendation letters. We suggest getting letters from an academic in the field (they carry more weight), and from someone who can attest to your qualities of persistence, team work and diligence. If you are undertaking a professional doctorate, a letter of recommendation from someone familiar with your connections and relevance to and capacities in the context in which you plan to conduct your applied research will be very useful. The final piece of advice here is to photocopy all of your registration material and forms. Yes, things do get lost and, in the event that you may suddenly have to recreate all that material in a matter of days, it is advisable to have an easily accessible spare copy.

1.4.3

How Do I Go About Financing My Doctoral Studies?

Financing your doctoral study will be largely dependent on your mode of study. Will it be part-time or full-time, or a combination of both at different times over your years of study? As a consequence, there are a number of means by which postgraduate study can be financed. These are as follows: • Government grants: This may come in the form of a Higher Education Loan Program to assist students pay for their studies.

1.4 Getting Registered

21

• External scholarships and grants (by external, we mean external to the institution): These are available to students from a variety of tertiary institutions. As a consequence, they are highly contestable but worth investigating. For international scholarships look at Scholarshipnet (2018) (https://www. scholarshipnet.info). However, you will find that most international scholarship bodies operate independently so will need to be researched and applied for on a case-by-case basis. For international students from developing countries, it is a useful exercise to contact the Embassy or Consulate of the countries where you would like to study and speak to their Educational Advisors as there is frequently an annual round of scholarships. • Internal scholarships: These are typically offered by a university for their own postgraduate students. Generally, there is a limited number available and, as with external grants, they have specific dates for submission of relevant paperwork, well in advance of the year in which a student is studying, so it is best to find this information out very early from the scholarships office. When applying for scholarships, there are often a number of evaluative criteria that are used but, from our experience of being on postgraduate scholarships committees, they tend to rely heavily on grade point average and the letters of recommendation, so do make sure you give your referees an adequate briefing and time to write the letters of support. • Research Assistant positions: Institutions often have positions available for Research Assistants or Teaching Assistants who will undertake seminars and tutorials. They are paid on academic salary scales for Research Assistants and Teaching Assistants. While these are highly competitive, the advantages are that you can get close to the academic environment. The disadvantage is that sometimes that becomes your main job and your own research can fall by the wayside. • Employer, profession or industry support: Some employers/professions/ industries will, if requested, agree to fund or support, in kind, postgraduate research. This is more likely to occur where they can see beneficial longer-term return from that support. In professional doctorate programs, candidates will frequently rely, at least partially, on employer/profession/industry support for their candidature. • Self-funding: Partial or full self-funding is the default funding mode for many postgraduates, when students are in full-time employment and are financing their research, either directly from their salary or with additional assistance given by their employer. • Entrepreneurial efforts: We have known the occasional student who has run a business on the side in order to generate income for their studies. Admittedly, it was web-based, which gave them some flexibility, however one needs to be cautious that not all of one’s time is spent on the fledgling business at the expense of postgraduate study. Which is the most common form of funding? According to a UK study (Leonard et al., 2004, p. 373), 39% of British doctoral students were self-funded, and 29%

22

1 Why Am I Doing This and What Should I Expect?

were self-funded with some support from their employer or family. A smaller number, 21%, received scholarships, grants or studentships. It is worthwhile anticipating what the cost of the postgraduate study is going to be taking into account tuition fees, living costs, probable data collection expenses, conference attendance outlays and so on. Work out whether you are going to study part-time or full-time? Are you going to take time out, that is, take leave without pay to work on, say, the final write-up? Is your partner contributing to the cost of your program? Are you planning on having a family, which may take one person out of the earning equation? What additional money could be generated? You may also wish to think about working for one or two years before you start in order to build up your resources. In this way, should you need to take time out to concentrate on your postgraduate studies or, specifically, to undertake the preliminary literature search, the data collection process or the research outcome write-up, you will have the resources to do so.

1.5

Key Recommendations

It has been noted that students who enter postgraduate programs for specific career goals are more likely to graduate than those with vague plans (Peters, 1997, p. 8). However, in the headlong rush to get going on your doctoral study, it is worth pausing at the beginning and ensuring that you have done the necessary preliminary work. This includes a full information search, investigating all your options for programs, supervisors, potential topics, modes of study and financial support. For those students seriously considering embarking on their doctoral studies, let us recap on some of our key recommendations: • At the beginning of your research journey, reflect on what you wish to achieve, from both a personal and a learning perspective, and why you want to do it. Name your goal and what you want to achieve, as well as know your motivation (s) for undertaking the degree. • Bring others into your process. Share your goals with others. Let other people know what you want to achieve—parents, friends, and employers so, it is hoped, they will be understanding when at times your postgraduate study takes precedence over other activities. • A key element for motivation is obviously your interest. Therefore, when choosing your topic, be cautious of being steered in a direction which you do not find appealing by an over-zealous potential supervisor. You need to have ownership of your study, and the topic will need to sustain your interest over a long period of time, if you are to remain motivated and able to maintain your focus. • Before talking with academics in potential institutions, take time to read and reflect in order to distil your topic from a broad area of interest to a general topic, and preferably to a problem/research question(s). This initial thinking

1.5 Key Recommendations

•

•

•

•

• •

•

•

•

23

time, exploration, reading and reflection will pay off well when you initiate discussions with potential supervisors. Keep your options open regarding where you intend to study and do due diligence on more than one institution by initially reading their web material. If possible, visit each institution twice, the first time to get a feel for the place and the resources, and the second time to actually meet key people with whom you have made appointments. When meeting with potential supervisors don’t be afraid to ask about their past supervision history. In this regard, you want to know how many students they have supervised but, more importantly, how many students they have supervised to completion. Make sure you obtain a copy of your program regulations. Program regulations will contain such information as the course requirements, minimum and maximum periods of candidature, thesis/dissertation/portfolio length, requirements for ethics approval, periodic reporting requirements, confirmation processes and examination processes. Pick the people, not the school. In other words, the supervisor relationship, the quality of the expertise in your area, and the academic training you will receive will be more important than the actual school or department where you are enrolled. Start early, as supporting documentation such as prior degree transcripts and reference letters will need to be sourced and it all takes time. While waiting for your application to be confirmed, don’t waste time. Keep reading and formulating your thoughts regarding your future study. As you wait for the administration process to churn on slowly, we suggest you continue reading material in the intended area of your study, or the methodological approach you wish to use as the preparation will be helpful in acquainting you with the literature and will provide more insight into the likely direction of your research. If you want to gain confidence and a clearer understanding of expectations, it is always a good idea to look at other recently completed theses/dissertations/ portfolios undertaken in the department in which you intend to study. These are usually available in the university library and more commonly these days have been digitised. If not, you may wish to look at the recent research outcomes supervised by relevant academic staff. Most supervisors proudly display the completed theses/dissertations/portfolios of their students on their office shelves and, as a consequence, they are easily accessible. If you are applying to undertake a professional doctorate, start conversing early on with relevant people in the context(s) in which you intend to conduct your research and for which you aspire to have some kind of impact. Get a feel for where they see as the problems and needs for research in their context and lay the groundwork for approaching potential non-academic supervisors. Remember, you are not doing this alone; others are there to support you in the form of personnel in the graduate office, your supervisor, advisors and fellow students. However, there are others you also need to consider and who are an

24

1 Why Am I Doing This and What Should I Expect?

integral part of your research journey. These are your family and friends. You need to ensure that they can, and will, support you in a task that will take you away from them for large slabs of time over the next few years. Be up-front and ensure that they, too, have a realistic understanding of what is involved. • At the outset, start developing a network of trusted colleagues and peers. This is critical to thriving and surviving during the postgraduate experience. Be prepared to contribute to a give-and-take set of obligations of mutual support during your research journey. • In order to achieve a goal, you have to fuel your goal with action which requires extra commitment and changes to your routine. As one colleague of mine who was working full-time confided, he would never have completed his PhD if he hadn’t got up at 5 a.m. each morning for three-and-a-half years. • Believe that you can do it! This may sound like ‘guru speak’ but, essentially, while you may lack confidence what you can develop is a ‘can-do’ attitude. Yes, there will be areas where you will stumble, but have confidence in your ability to work through the challenges and come triumphantly out the other side.

References Aliotta, M. (2011). Ten good reasons for doing a PhD—Part I. Academic life—Resources for aspiring (and established) academics. https://marialuisaaliotta.wordpress.com/2011/11/14/phd_ 1-1/. Azuma, R. T. (2017). So long, and thanks for the PhD! http://www.cs.unc.edu/*azuma/hitch4. html. Accessed 6 Feb 2018. Badley, G. (2009). Publish and be doctor-rated: The PhD by published work. Quality Assurance in Education, 17(4), 331–342. Bourner, T., Bowden, R., & Laing, S. (2001). Professional doctorates in England. Studies in Higher Education, 26(1), 65–83. Brailsford, I. (2010). Motives and aspirations for doctoral study—career, personal, and inter-personal factors in the decision to embark on a history PhD International Journal of Doctoral Studies, 5(1), 16–27. Brown, B. R. (2006). Doing your dissertation in business and management the reality of researching and writing [SAGE Study Skills Series]. London: Sage Publications. Brown, S., McDowell, L., & Race, P. (1995). 500 tips for research students. London: Kogan Page. Burton, S., & Steane, P. (2004). Surviving your thesis. London: Routledge. Churchill, H., & Sanders, T. (2007). Getting your PhD: A practical insider’s guide. London: Sage Publications. Collis, J., & Hussey, R. (2014). Business research: A practical guide for undergraduate and postgraduate students (4th ed.). London: Palgrave Macmillan. Cryer, P. (2006). The research student’s guide to success (3rd ed.). Buckingham, UK: McGraw-Hill Education. Das, A. (2015). Factors to consider when choosing a lab for PhD training. An international forum for cell biology. http://www.ascb.org/compass/compass-points/factors-to-consider-whenchoosing-a-lab-for-phd-training/. Accessed 20 Jan 2018. Delamont, S., Atkinson, P., & Parry, O. (2000). The doctoral experience success and failure in graduate school. London: Falmer Press.

References

25

Denholm, C., & Evans, T. (2012). Doctorates downunder: Keys to successful doctoral study in Australian and Aotearoa New Zealand (2nd ed.). Camberwell, Victoria: ACER Press. Dickinson, H. W., Connell, H., & Savage, J. (1997). Student experiences. In N. J.Graves & V. P. Varma (Eds.), Working for a doctorate: A guide for the humanities and social sciences (pp. 113–130). Routledge, London. Dinham, S., & Scott, C. (1999). The doctorate: Talking about the degree. Kingswood, NSW: University of Western Sydney, Nepean. Dunleavy, P. (2003). Authoring a PhD: How to plan, draft, write and finish a doctoral thesis or dissertation. Basingstoke, UK: Palgrave Macmillan. Elphinstone, L., & Schweitzer, R. (1998). How to get a research degree: A survival guide. St Leonards, NSW: Allen and Unwin. Fink, D. (2006). The Professional doctorate: Its relativity to the PhD and relevance for the knowledge economy. International Journal of Doctoral Studies, 1(1), 35–44. Fisher, C. (2010). Researching and writing a dissertation: An essential guide for business students (3rd ed.). New York: Financial Times/Prentice Hall. Francis, H. (1997). The research process. In N. J.Graves & V. P. Varma (Eds.), Working for a doctorate: A guide for the humanities and social sciences (pp. 18–34). Routledge, London. Fung, A. S. K., Southcott, J., & Siu, F. (2017). Exploring mature-aged students’ motives for doctoral study and their challenges: A cross border research collaboration. International Journal of Doctoral Studies, 12, 175–195. Gill, G., & Hoppe, W. (2009). The business professional doctorate as an informing channel: A survey and analysis. International Journal of Doctoral Studies, 4(1), 27–57. Google Scholar. https://scholar.google.com/. Accessed 6 Feb 2018. Graves, N. J., & Varma, V. P. (1997). Working for a doctorate: A guide for the humanities and social sciences. London: Routledge. Green, B., Maxwell, T. W., & Shanahan, P. (2001). Doctoral education and professional practice: The next generation?. Armidale, NSW: Kardoorair Press. Greenspon, A. (2013). 9 things you should consider before embarking on a PhD: The ideal research program you envision is not what it appears to be. https://www.elsevier.com/connect/ 9-things-you-should-consider-before-embarking-on-a-phd. Accessed 15 Jul 2018. Hegarty, N. (2011). Adult learners as graduate students: Underlying motivation in completing graduate programs. The Journal of Continuing Higher Education, 59(3), 146–151. Jackson, D. (2013). Completing a PhD by publication: A review of Australian policy and implications for practice. Higher Education Research and Development, 32(3), 355–368. Lawton, D. (1997). How to succeed in postgraduate study. In N. J. Graves & V. P. Varma (Eds.), Working for a doctorate: A guide for the humanities and social sciences (pp. 1–17). London: Routledge. Lee, P., & Pang, V. (2014). The influence of motivational orientations on academic achievements among working adults in continuing education. International Journal of Training Research, 12 (1), 5–15. Lee, A., Brennan, M., & Green, B. (2009). Re-imagining doctoral education: Professional doctorates and beyond. Higher Education Research & Development, 28(3), 275–287. Maxwell, T. W. (2003). From first to second generation professional doctorate. Studies in Higher Education, 28(3), 279–291. Maxwell, T. W., & Kupczyk-Romanczuk, G. (2009). Producing the professional doctorate: The portfolio as a legitimate alternative to the dissertation. Innovations in Education and Teaching International, 46(2), 135–145. Leonard, D., Becker, R., & Coate, K. (2004). Continuing professional and career development: The doctoral experience in education alumni at a UK University. Studies in Continuing Education, 26(3), 369–385. Leonard, D., Becker, R., & Coate, K. (2005). To prove myself at the highest level: The benefits of doctoral study. Higher Education Research & Development, 24(2), 135–149. Marshall, S., & Green, N. (2007). Your PhD companion: A handy mix of practical tips, sound advice and helpful commentary to see you through your PhD (2nd ed.). Oxford, UK: How to Books.

26

1 Why Am I Doing This and What Should I Expect?

Mowjee, B. (2013). Are postgraduate students ‘rational choosers’? An investigation of motivation for graduate study amongst international students in England. Research in Comparative and International Education, 8(2), 193–213. Mueller, E. F., Flickinger, M., & Dorner, V. (2015). Knowledge junkies or careerbuilders? A mixed-methods approach to exploring the determinants of students’ intention to earn a PhD. Journal of Vocational Behavior, 90, 75–89. Nethsinghe, R., & Southcott, J. (2015). A juggling act: supervisor/candidate partnership in a doctoral thesis by publication. International Journal of Doctoral Studies, 10, 167–185. Neumann, R. (2005). Doctoral differences: Professional doctorates and PhDs compared. Journal of Higher Education Policy and Management, 27(2), 173–188. Peters, R. (1997). Getting what you came for: The smart student’s guide to earning a Master’s or a PhD (Rev ed.). New York: Noonday Press. Petre, M., & Rugg, G. (2010). The unwritten rules of PhD research. Open up study skills (2nd ed.). Maidenhead, UK: Open University Press. Phillips, E., & Pugh, D. S. (2015). How to get a PhD: A handbook for students and their supervisors (6th ed.). Maidenhead, UK: Open University Press. Powell, S., & Green, H. (Eds.). (2007). The doctorate worldwide. Maidenhead, UK: Open University Press. Quinton, S., & Smallbone, T. (2006). Postgraduate research in business: A critical guide. London: Sage Publications. Race, P. (2007). How to get a good degree: Making the most of your time at university (2nd ed.). New York: Open University Press. REF. (2014). Research excellence framework. http://www.ref.ac.uk/. Accessed 6 Feb 2018. Rudestam, K. E., & Newton, R. R. (2015). Surviving your dissertation: A comprehensive guide to content and process (4th ed.). Thousand Oaks, CA: Sage Publications. Ryan, R., & Deci, E. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. Salmon, P. (1992). Achieving a PhD: Ten students’ experience. Staffordshire, UK: Trentham Books. Scholarshipnet. (2018). http://www.scholarshipnet.info/. Accessed 6 Feb 2018. Smith, H. (2007). Playing a different game: The contextualised decision-making processes of minority ethnic students in choosing a higher education institution. Race, Ethnicity and Education, 10(4), 415–437. Socialcapitalgateway. (2018). Academic Rankings. http://www.socialcapitalgateway.org/ resources/academic-rankings. Accessed 6 Feb 2018. Templeton, R. (2016). Doctorate motivation—An (auto) ethnography. Australian University Review, 58(1), 39–44. TheHigherEducation. (2018). World University Rankings. https://www.timeshighereducation. com/world-university-rankings. Accessed 6 Feb 2018. Thomas, M. R., & Brubaker, D. L. (2007). Theses and dissertations: A guide to planning, research, and writing (2nd ed.). Thousand Oaks, CA: Corwin Press. Thunborg, C., Bron, A., & Edström, E. (2013). Motives, commitment and student identity in higher education—Experiences of non-traditional students in Sweden. Studies in the Education of Adults, 45(2), 177–193. Topuniversities. (2018). QS world university rankings 2018. https://www.topuniversities.com/ university-rankings/world-university-rankings/2018. Accessed 6 Feb 2018. USNews. (2018). Best colleges rankings. https://www.usnews.com/best-colleges/uiuc-1775/ overall-rankings. Accessed 5 Feb 2018. Wellington, J., Bathmaker, A. M., Hunt, C., McCulloch, G., & Sikes, P. (2005). Succeeding with your doctorate. London: Sage Publications. Wiegerová, A. (2016). A study of the motives of doctoral students. Procedia—Social and Behavioral Sciences, 217, 123–131.

Chapter 2

What Skills Do I Need?

2.1 2.1.1

Skills for Postgraduate Research Do I Have What It Takes to Get a Postgraduate Research Degree?

With the exception of some supremely confident individuals, lurking in the back of the mind of most postgraduate students is the question of whether they have what it takes to undertake and complete a postgraduate qualification. Given that it is not just about knowing research methodology and being able to write but is also about planning and managing yourself. There is a wide range of skills that you will need to draw upon in order to successfully come out the other end of the process. It should be comforting to know that doing a postgraduate research degree is inherently a learning experience, so, while you may not have all the attributes at the beginning, they will most certainly develop as you go along. Clearly, however, the more adept you are in demonstrating some key competencies the greater the likelihood that you will complete the degree. Regrettably, however, there is no stated threshold of skills needed in the beginning apart from the admissions criteria or, say, a prior research degree, nor is there any guarantee that possession of essential key skills will necessarily ensure smooth sailing throughout the project. Fortunately, the area of what skills are required both for and upon conclusion of a postgraduate research degree, has been a topic of discussion with considerable debate regarding the ability to be able to produce generic and transferable skills as outcomes of research higher degrees (Gilbert, Balatti, Turner, & Whitehouse, 2004). By generic skills, we mean skills which all postgraduates should possess upon graduation (sometimes called Graduate Attributes). Zeegers and Barron (2006) identified six attributes a PhD student should have: • ability to make contributions to knowledge; • ability to undertake independent research; • ability to communicate research findings; © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_2

27

28

2 What Skills Do I Need?

• ability to demonstrate intellectual and professional integrity; • ability to undertake future research and employment; and • ability to participate in life-long learning and engage with the community. More attention has been given to the development of a broader set of skills, so-called transferable skills. Transferable skills are defined as skills developed in one context (in this case, the academic context) that can be also used in other contexts, for example future employment, whether that is in research, government or business (Scholz, Klein, Behrens, & Johansen-Berg, 2009). Given the growing realisation that with a combination of increasing numbers of doctoral graduates being produced and only a small proportion actually finding employment long term in the university sector (many being employed elsewhere), transferable skills play an important role in the future career paths of graduates (Walsh, Seldon, Hargreaves, Alpay, & Morley, 2010). Government and industry leaders have also complained that PhD graduates lack the skills required for current labour markets and have challenged the relevance of the PhD (Mowbray & Halse, 2010). Consequently, there has been great emphasis placed on the importance of transferable skills and transferable skills training programs are now a familiar element of research degree programs in many countries, hopefully resulting in potentially better future career opportunities for graduating PhD students. This has been successful with doctoral training programs in the sciences which have been providing trainees with transferrable skills that “are in many ways meeting the increasing demand to prepare graduates for a wide variety of careers” (Scholz, Klein, Behrens, & Johansen-Berg, 2017, p. 11). Development of transferrable skills is also a hallmark of the emergence of professional doctorates, which emphasise being able to connect with a wider range of stakeholder audiences in ways that permit them to influence the direction, contextual relevance and even the substance of knowledge creation (e.g., Fink, 2006; Maxwell, 2003). Some universities websites have endeavoured to describe what they mean by transferable skills, e.g.: PhD Transferable Skills: https://careercenter.umich.edu/article/phd-transferableskills. PhD Transferable Skills: https://grad.msu.edu/phdcareers/career-support/skills. But it is not just about skills necessary for future careers. In the short term, it is about skills necessary to ensure successful completion. The focus has therefore been on the skills to enable students to undertake well-rounded research and to cope with the personal challenges that their study entails, as well as preparing them for future employment, either in or outside of academia. The result has been considerable deliberation about what the appropriate skills for postgraduate study are. In order to get a handle on the skills required let’s look at what a PhD is, what have already been identified as key skills, what do we recommend, and how you could go about auditing yourself to gain a good idea of how you are placed and which skills you may have to work on.

2.1 Skills for Postgraduate Research

2.1.2

29

So, What Is a PhD?

To gain an appreciation of the skills involved in a PhD, let’s first identify what a PhD is. The PhD has been described as “the pinnacle of university learning and scholarship” (Mowbray & Halse, 2010, p. 653). While acknowledging that PhDs by thesis/dissertation, professional and practice-based doctorates and PhDs by publication vary in their focus, the purpose of a PhD has been well-documented. The American Council of Graduate Schools (Stewart & Stewart, 1993, p. 1) states it succinctly—a Doctor of Philosophy program is designed to prepare a student to become a scholar, that is, to discover, integrate and apply knowledge as well as communicate and disseminate it. The British are a little more detailed in their description of what a PhD is and, from the UK perspective, the intention of a PhD is two-fold: • The first is, “to enable young people of high intellectual ability to develop and bring to fruition as far as possible the quality of originality, to contribute new and significant ideas, and to make a positive contribution to knowledge and creativity in their respective disciplines”. • The second is, “to provide training in research methods which makes them capable subsequently of assuming the role of independent scholars and research workers at the highest level, capable of planning and carrying to completion a well-conceived plan of research directed toward a given objective without the necessity of supervision from experienced people” (Committee of ViceChancellors and Principals (PVCP), 1998, cited in Pole, 2000, p. 96). Essentially, a PhD is aiming at developing you as an independent researcher with the ability to critique large volumes of material in order to identify new research themes. Subsequently, it should enable you to design and execute independent research programs, contribute to the body of knowledge by reporting on those programs, competently discuss your research with others, both students and staff in the academic community and, oh yes, upon completion also to get a full time job or perhaps progress further in your career trajectory if you are already employed. What about a professional doctorate? Professional doctorates (e.g., DBA, EdD, DPsych) have evolved from earlier forms that basically involved coursework and a thesis that was focused on professional practice to more modern forms where the nature of the entire research endeavour took on new structures and shapes (e.g., a portfolio) and typically involved a more concerted integration of influences from not only academia but also workplace, professional and practice-based communities (Maxwell, 2003, referred to these new forms as second-generation professional doctorates). Thus, a professional doctorate tends to focus on generating knowledge relevant not only to academia but also to some profession or community of practice. In many cases, the intention of professional doctorate research is to influence practices or facilitate change in a

30

2 What Skills Do I Need?

particular professional or workplace context. With respect to second generation profession doctorates, Maxwell said “(i)n these (i.e., professional doctorates) the realities of the workplace, the knowledge and the improvement of the profession and the rigour of the university are being brought together in new relationships” (Maxwell, 2003, p. 290). As an example, the PhD (Innovation) at the University of New England in Australia is a professional doctorate with a focus on innovation. Adding value to the label of PhD, the PhD.I has a deliberate emphasis on research that develops and demonstrates the relevance, utility and potential of an innovation (be it technology, software, social policy or program) in some professional or industrial context. The research outcome produced is a portfolio that chronicles this innovation journey.

2.2 2.2.1

Prior Research What Have Already Been Identified as Key Skills?

It has been noted that there is actually no formal theoretical skills framework (Manathunga, Lant, & Mellick, 2006) and with an absence of a framework there are ever increasing lists, often derived in ad hoc manner (Platow, 2012), but interestingly with many descriptions including skills such as: research methodology, project management, teamwork, and communication (Wardenaar et al., 2014). To distil our own thinking and to steer you in the direction of undertaking your own skills audit, it is appropriate to look at what others have identified as necessary skills required for the completion of a PhD. An analogy has been drawn between undertaking a PhD and traditional cabinet-making. In the past, an apprentice cabinet-maker was required to make a cabinet which demonstrated all his skills; in the academic context, the PhD is the cabinet and the requisite cabinet-making skills for the PhD student are: • use of the academic language, structure and design, punctuation and academic writing style; • knowledge of background literature, seminal pieces of work, current literature both in and from related disciplines and the ability to organise and critique the material, identifying conceptual relationships, themes and gaps; • knowledge of research methods appropriate to the discipline(s) reflected in your project, relevant to data gathering and data analyses, as well as the ability to critically evaluate what you learn; and • the ability to demonstrate a level of maturity and independence in using these skills which would include necessary skills such as tact and diplomacy (Petre & Rugg, 2010, p. 4).

2.2 Prior Research

31

It has been noted that successful doctoral candidates often emphasise skills over knowledge in the outcome of the doctoral process. (Pole, 2000, p. 106) subsequently refers to a typology of both skills and knowledge, these being: • Knowledge acquired through the discipline focus of their qualification. • Technical skills developed as a result of conducting the research, this includes data gathering, use of software, statistical and qualitative analysis skills and associated interpretive skills. • Craft knowledge which denotes a capacity to manage a research project through various aspects. It is more holistic than technical skills and incorporates the entire research process and interaction with key networks, as well as project planning. • Personal skills which relate to communication, teamwork and, in the personal dimension, building one’s self-confidence and self-identity. The Council of Australian Deans and Directors of Graduate Studies initially identified over 50 generic skills which PhD students should acquire during their candidature (Zeegers & Barron, 2006). These appear to have been truncated in 1999, with the main skills required for research and thesis preparation, being: • communication—writing, oral, presentation, electronic; • information skills—identifying and searching information sources, managing information, data analysis and presentation, bibliographic compilation; • project skills—project management, working with ethics, safety and intellectual property guidelines, regulations, team work, leadership, negotiation, decisionmaking, grant application skills, dealing with outside agencies, business planning, marketing and entrepreneurship; • cognitive skills—analysis, evaluation, synthesis, application of arguments and evidence, research design and methods, language skills; and • professional development—career preparation, tertiary teaching skills, development of employment and career opportunities (Council for Australian Deans and Directors of Graduate Studies, https://www.ddogs.edu.au). In a similar list generated from interviewing actual final year full-time PhD students, Mowbray and Halse (2010) classified their skills into 7 categories: • • • • • • •

research; cognition; written and oral communication; workplace and career management; project management; leadership and organisation; and personal resourcefulness.

In general terms, the skills articulated by the students parallel the skills that policy makers consider desirable outcomes of doctoral education with the exception of personal resourcefulness. Personal resourcefulness rarely appears in university

32

2 What Skills Do I Need?

and government policies or advocated by industry, yet this was the category of skills most valued by students. Personal resourcefulness involves “developing the confidence, discipline, intrinsic motivation, resilience, tenacity and interpersonal skills that enabled students to balance the institutional, professional and personal responsibilities occurring during the PhD” (Mowbray & Halse, 2010, p. 656). In regard to the latter dimension of personal skills, Phillips and Pugh (2015) do not refer specifically to skills, but to the psychological dimensions required of a student engaged in the PhD process, and highlight the need for enthusiasm, ability to deal with isolation, self-confidence, managing frustration, time management, not getting side-tracked, and moving towards being an independent researcher who can get the job done. The terminology has become even more varied and, in relation to the field of education, Gardner, Hayes, and Neider (2007) interviewed PhD students to determine the skills, habits of mind and dispositions needed to obtain a PhD. While skills and abilities such as the ability to analyse, synthesise, evaluate and conduct research in a variety of research traditions are more tangible and observable, habits of mind were seen as intangible attitudes, values and characteristics that cannot be casually observed and involve “a quest for knowledge, independence and humility” (Gardner et al., 2007, p. 294). Similarly the skill of developing a growth mindset, instead of fixed mindset, in relation to the PhD journey has been advised. Given that ones’s mindset influences motivation and how you think about failure “with a growth mind-set, you would analyse your failure in more detail to figure out what went wrong and why, so you know what to focus on in the future” (Deconinck, 2015, p. 362).

2.3

Key Skills

With such a variety of skills needed to complete a PhD—where does one start? If you go even further into the research, you will be surprised that upon reviewing British, American and Australasian material, and despite geographic and program diversity, there is a degree of similarity in what have been distilled as the central skill requirements for completing doctoral studies. We are interested in the skills necessary to undertake the qualification, as well as skills that students may acquire along the way. Our focus, therefore, is to examine what skills will enhance your likelihood of successful completion. As a caveat, the terminology relating to this area is quite diverse, with a variety of descriptors such as: generic skills, transferable skills, core competencies, knowledge base, attributes, traits, dispositions and habits of the mind, to mention but a few. For the purpose of simplicity, we will treat them as being relatively synonymous. A further comment worth noting is that the selection of titles for key skills is not easy, given the inevitable overlap in related skills. We have, therefore, chosen what we consider are the most appropriate titles but have also highlighted significant and related skills or traits. We present one list of skills for undertaking a PhD, then augment that list with another list adding skills useful in undertaking a professional doctorate.

2.3 Key Skills

2.3.1

33

What Are the Top 8 Skills Needed to Successfully Complete a PhD?

Intellectual curiosity Sitting at the top of our list is intellectual curiosity and a genuine desire to explore. Undertaking a doctoral degree is about asking questions to which you do not know the answers, then undertaking research to investigate those questions. Some individuals may be less interested in the questioning but merely want to impart their knowledge on a subject area without the research underpinning. If this is the case, then a doctorate, which has a fairly structured format and expectations, is not for them and, perhaps, writing a book rather than undertaking postgraduate study would be a more appropriate endeavour. Doctoral students, who demonstrate humility, acknowledge their lack of knowledge without being intimidated by it, are receptive to new ideas, new approaches, new material, and new methodologies will find the journey interesting even when it is challenging. This works really well if you are willing and able to be creative and to use a different approach to examining an issue or problem. Intellectual curiosity is about thinking systematically and holistically about existing knowledge, guiding assumptions, opportunities and constraints, choices and implications and to look for new angles. It is about being able to think laterally and inventively, to develop original approaches to solving problems. If you are one of those people who want to lift the bonnet, get to know how things work, poses questions, comfortable that you don’t know the answer but excited to look into it, you will do well. With a genuine desire to find answers and natural curiosity this will lend itself to the independent pursuit of knowledge which is part of the doctoral process. Being intellectually curious has benefits: (1) A higher level of curiosity will, no doubt, result in more creativity in one’s approach; by asking questions you will start to uncover some new ground which will contribute to the originality of your research; (2) Curiosity will power you through some pretty tedious times when, for example, you are trawling through large volumes of related, or possibly not so related literature, and volumes of data that are yet to be analysed. Being interested and engaged will, undoubtedly, help as you seek to discover what is actually being communicated and how it might assist your research; (3) Intellectual curiosity will position you well for receiving and digesting critical feedback. Rather than viewing it as negative and becoming defensive, a curious student will be open and receptive and will consider how they can use the information rather than acting on an initial impulse to discard the advice. (4) One more benefit, and more directly related to your research, is that intellectual curiosity will assist you with framing your research questions, evaluating your data and discerning the theoretical or practice implications of your findings.

34

2 What Skills Do I Need?

Critical thinking Critical thinking has been recognised as the most central of cognitive skills as it involves “the ability to scrutinise and synthesise ideas and information, recognise different points of view, appropriate theory and use more sophisticated theoretical insights to interpret data and support analyses and conclusions” (Mowbray & Halse, 2010, p. 660). It involves dispassionately critiquing your own thinking, the literature, previous research and your own research process and results. Before tracking down one line of enquiry, be open to considering alternative perspectives and to contemplating issues utilising different theoretical or conceptual tools. Consider issues from a range of angles and draw upon appropriate concepts in arriving at a critical assessment. If you have not already done so, you will develop skill in reflecting on a subject from a variety of different viewpoints utilising appropriate typologies and concepts to draw out relevant and insightful assessment, and in doing so acquire the ability to develop sound academic arguments. Critical thinking requires you to overcome a passive stance to the information you are reviewing, either from prior literature or your own data, and to be prepared to go further in reflecting on the material as well as reflecting on your research journey. Having located relevant information from a variety of sources, critical thinking involves bringing it together and present in a coherent argument. You are looking to creating evidenced-based views and opinions that in time could become developed into theoretical contributions. To do this involves developing a sense of when it is time to think broadly across a range of issues, concepts and literature, and when it is time to work more deeply into specifics in one or more of these areas. Achieving the right balance between breadth and depth of focus is a dynamic process and there will be periods within your doctoral study where a shift in the balance will be needed. For example, when you commence your journey, you will need to adopt a much stronger emphasis on breadth rather than depth, in order to begin shaping your research problem. As your problem takes shape, the balance shifts to less breadth and more depth as you focus more tightly on what your specific research questions will be and how you will actually address them. Going deep too early can leave you at risk of neglecting an important issue or area of knowledge, whereas focusing broadly for too long can leave you at risk of floundering in a sea of information and uncertainty as to how to chart a course through. Learn to recognise some of the signals indicating that a shift in the balance is required such as (1) when your research problem needs to be narrowed down into specific research questions (this will need to occur before you produce your proposal and marks a need to shift from breadth of focus toward depth of focus) and (2) when you are drawing the conclusions from your research and begin linking them to wider implications as well as to what has been learned before (this will need to occur as you are writing your concluding chapter and marks a need to shift from depth of focus toward breadth of focus). Flexibility An open attitude and willingness to adapt and learn go hand in hand with flexibility, and the ability to cope with ambiguity. To be flexible and open-minded often

2.3 Key Skills

35

requires a student to be comfortable with a lot of loose ends hanging around before they can be threaded together into a defined theoretical or conceptual framework. Specific methodological choices will need to be made and a coherent story told, however, for a while you will be concurrently entertaining multiple approaches. For this you will need to be “someone who is or can be vulnerable and comfortable with ambiguity” (Gardner et al., 2007, p. 292). Essentially, handling ambiguity is the ability to cope with a lot of “I don’t knows”, and, if you are to take advantage of situations, you will need to be adaptable and willing to change through your research journey. For example, one student I know had their heart set on researching a particular organisation in which they used to work. However, a chance meeting with a business owner at a conference opened up an entirely new opportunity. Fortunately, the student was flexible enough to realise what was being presented. Following discussion with his supervisor, he was able to take advantage of the opportunity provided to access a new research context, despite having already invested time in preliminary but somewhat fraught data gathering in relation to the first organisation. Something else to consider is being able to problem solve in order to overcome obstacles and to effectively manage obstacles that will inevitably come your way and not let these changes overcome you. By way of example, following the untimely death of my supervisor, I was allocated a new supervisor two-thirds of the way through my PhD. I realised at that point that it could become an impediment to completing my studies and vowed to remain flexible and open to the new supervisory relationship which, in time, proved to be extremely rewarding and positive. Essentially, it is recommended that you remain open and even opportunistic, throughout your research journey. In practical terms this will require that you are responsive to feedback, not only to the feedback you are receiving in the early stages and during the formation of your research frame and configuration, but also as you are approaching organisations, groups or individuals for data gathering purposes. As your data are coming in, stay open to seeing where the data and the findings are heading. It is not uncommon to identify a significant new dimension of your research as your results unfold. This may require you to adapt, and re-focus on your initial research questions, in order to achieve greater coherence throughout your research journey from the research questions through to the findings. Project management Specifically, the skills that relate here are at the other end of the intellectual scale but without these doctoral students will struggle. This includes being able to set priorities, plan effectively, make decisions, meet project related milestones, and manage multiple activities concurrently. Essentially this is about goal setting, prioritising tasks, careful planning in order to achieve results on time and within budget. Specifically, we are focusing on project management, organisational and time management skills; skills which we will look at again in later chapters. However, it is appropriate to provide an initial introduction to their relevance and the need to either have or develop skills in these areas.

36

2 What Skills Do I Need?

Doing doctoral research can feel like you are engaged in an unstructured piece of work for a very long time. Deadlines can easily be missed, whereupon the project becomes further elongated. The trick is therefore for you to overlay some structure and to be vigilant with respect to the timing associated with the many stages of the process. To achieve this, you will need to develop and implement effective project management skills. Utilising either a paper-based or preferably an electronic project management system, you will need to take a holistic approach to your project, capture the essence of what is required, the key dimensions (we will help with this part) as well as all the university requirements. In practice, you will need to plan out the next few years and each stage of your research process. As you approach each stage, you will further consider what is required in terms of resources (e.g., time, money, software and other technology), and you will need to plan the next steps in even more detail—this is the straightforward part. The challenging part will be continuing to monitor your progress against your plan, to find solutions to challenges, resolve hold-ups and to make reasonable and realistic revisions to your plan. In addition to project management skills, you will also need to develop fairly high-level organisational skills. You will be amazed at the amount of material and data you accumulate through the duration of your studies: registration and program documentation, your proposal, prior literature papers, relevant books and reports, sample theses and other research outcomes, correspondence, notes, data such as questionnaires, tapes and transcripts, data files, your research journal, drafts of chapters, papers in preparation or published papers, presentations, and so on. It all quickly mounts up and, unless you are organised and have a systematic approach, you will quickly become swamped and be unable to locate material when needed. With doctoral study spanning many years it may seem as if you have plenty of time, but the reality is that there are defined stages which you need to move through and for which momentum must be maintained. Project management helps overlaying structure onto the entire project but, on a weekly, daily and even hourly basis, it is up to you to carve out appropriate research-focused time while keeping other commitments such as your health, your family and friends, and your job ticking over nicely. It will require a considerable amount of time management if you want to have a balanced life and remain happy throughout the journey. Time management deals with the ability to block out usable time on a very regular basis for all aspects of your life and work, prioritising activities, working on the important elements of your project, maintaining productivity and meeting self-imposed deadliness, for which sacrifices will possibly need to be made in the form of foregoing other activities. Apart from helping students complete their doctoral programs within a reasonable time, these are “skills students will use in managing multiple responsibilities for the rest of their professional lives” (Conn et al., 2014, p. 29), hence these key skills are important not just for doctoral completion but also for one’s future career. Self-management While we have talked about intellectual curiosity, don’t fall into the trap of believing that you must be incredibly bright to do a doctorate. Yes, you need to be

2.3 Key Skills

37

able to operate at an advanced postgraduate level but many an intelligent student has fallen by the wayside because they were not able to identify one of the key requirements necessary for doctoral study—to manage themselves, and to remain committed for the duration of the journey. This is sometimes referred to as self-management because of the need to handle not only the intellectual rigour of the process but, equally importantly, the ability to handle yourself in relation to the project—which is to effectively manage your personal, social, emotional as well as financial resources. Regrettably, no-one else is going to do the doctorate for you, so the research process and write-up will require many hours of working alone, establishing a work schedule and sticking to it even when what you are working on doesn’t appear to be coming together, you are struck by boredom or are just plain fed up with the whole exercise. As a consequence, not only should you be comfortable with a very autonomous work style, you should also have the self-discipline necessary to persevere with your project. Unlike your prior studies, which have had defined time lines such as semesters or academic years, punctuated with prescribed assessments in the form of assignments and exams to measure performance, a doctorate can be a very vague beast. Hence you need to possess or develop your ability to work independently, often in unfamiliar environments and under pressure, where you are not the only one setting the demands, while working your way through what can feel like a very amorphous piece of work. Some students find the unstructured working environment difficult. It can feel a bit like jumping in the deep end of a very large swimming pool and not being able to reach or touch the edge. You feel as if you are flapping around and treading water for quite some time until you are able to identify the length and breadth of the pool. To continue with this analogy, the length and breadth of the pool contain the body of knowledge and knowing where it starts and finishes can sometimes be a little tricky. Being able to deal with ambiguity as you personally work your way through, trusting in the fact that ultimately one day you will be able to gauge the size of the pool, will help you stay calm through the process and in the wee hours when it is just you dealing with the project. While you will have one or more supervisors, most of the time you will be working without supervision and in locations that may be new or unfamiliar to you, such as in different libraries or working in a variety of organisations or circumstances collecting data. You will be doing this on your own. In addition, if you are looking for reassurance, a pat on the back and a grade at the end of each semester, it’s not going to happen. Sometimes, the lack of recognition or encouragement can be a bit demoralising, so it is an excellent skill to be able to pull yourself out of the doldrums and to actively engage in self-motivation. Give some thought to how you might be able to create your own forms of recognition by identifying specific rewards you could give yourself (possibly a Sunday binge session on Netflix or day off at the beach or sporting ground) and attach them to your achievement of specific accomplishments associated with your research project. Essentially, you are the one who needs to take the initiative and to be actively responsible for your learning, the process and the outcomes. This may require that

38

2 What Skills Do I Need?

you undertake a frank assessment of you own strengths and weaknesses and seek to improve where ever possible. Communication In relation to doctoral students, communication skill is a broad term used to describe the ability to communicate your research in a variety of different formats and settings at different stages through the research process and with different audiences. This will involve the ability to communicate effectively in all modes (e.g., written, oral, email, messaging, visual and non-verbal). As a postgraduate student, you will be communicating in many different forms and arenas, both formally when submitting scholarship applications and research proposals, seeking access to data sources, gathering your data, participating in workshops, submitting journal papers, and writing your research outcome. In more informal settings, you are likely to be establishing rapport with administrative and support people, making friends with other postgraduate students, establishing/maintaining relationships with your supervisor(s), presenting a proposal, emailing your progress reports, engaging with sources of data and asking for assistance from others with relevant expertise. As the principal research outcome (but not the only written one) of a doctorate is a thesis, dissertation or portfolio, one of the critical areas is the ability to write effectively. This is one step further than just writing well. To write well, you must follow the rules of academia such as presenting material in grammatically correct form and following appropriate protocols and formatting styles such as the academic structuring of the chapters of the thesis (more on this in later chapters). One must also “obey academic conventions about referencing and constructing bibliographies which some may find extremely irksome” (Lawton, 1997, p. 3). To write effectively however is to write in the style appropriate for the purpose and the audience. It could be in communication by email with your supervisor, writing for a journal, or writing a research outcome. If writing is a perceived area of weakness for you, fortunately, in addition to the material that we present in this book, there is a variety of texts that are tailored for the postgraduate student that can assist with appropriate academic structure and techniques, as well as how to construct coherent arguments in a logical and well-formed way, which will enable you to communicate to a range of audiences (see, for example, Benn & Benn, 2006; Bolker, 1998; Dunleavy, 2003; Single & Reis, 2009; Thody, 2006; Zerubavel, 1999) For a doctoral student there will also be a lot of verbal communication such as presenting at doctoral colloquia, engaging in workshops, and presenting at conferences and negotiating access to data sources. To do this successfully in addition to the obvious art of public speaking is the skill of active listening. By active listening, we mean being open to and able to hear what is being communicated without overlaying your own prejudices or assumptions. In practice this will mean genuinely endeavouring to understand what is being communicated, asking considered questions, and offering respectful advice where appropriate. In the broader context of communication are interpersonal skills which encompass everything, for example, from being able to ‘play nicely’ with other

2.3 Key Skills

39

postgraduate students through to being able to defend and graciously receive feedback on your research from supervisors and in academic symposia or workshops. It also refers to providing constructive feedback to peers and accepting feedback from them, contributing to the research and learning of others, working effectively with your supervisor(s) and asking for help when needed, and promoting your research. Interpersonal skills also involve working in teams, facilitating discussions, and possibly engaging in conflict resolution. As you can gather, interpersonal skills are even broader than the traditional forms of academic engagement and relate to the way you work with people who are directly or indirectly involved in your research. This could be the administrative staff within the department, the library staff, and statistical advisors, and mentors, participants in your study and others who, in effect, become your networks. This would also extend to relating to and communicating with key stakeholders in your research, something that is especially important in a professional doctorate as we will see below. While the emphasis in most of the literature is on the solitary and independent nature of a doctorate, one area worth mentioning is that you will frequently require advice and input from other individuals. You will therefore need skills in building networks and in making and maintaining connections as you build relationships with others, such as researchers, participants, stakeholders, gatekeepers, resource providers and fellow students. Interpersonal communication is important in the building of relationships and a key relationship is with your supervisor or committee members. If you wish to avoid embarrassment at a future meeting where the supervisor expects work to have been done, a paragraph to be re-drafted, a questionnaire to be piloted, or a chapter to be modified, and it has not been done, you need to have developed open and clear communication. If you are unsure of what is expected of you, always seek further clarification. Building relationships will also be necessary as you solicit help from a variety of different sources during the times when you hit a patch where you need additional assistance. When this happens, you need to take the initiative to locate and tap into relevant support and advice. Be aware that the sources may not reside entirely in your own institution. It is important that you are extremely active in seeking out needed information and don’t passively hope that someone will see you in difficulty and come to rescue you. However, when you do seek help, don’t overburden people with the expectation that they will do your work for you. Demonstrate that you recognise it is your primary responsibility, but that some assistance or direction would be welcomed. Your interpersonal skills define the way you relate to those involved in your research journey. Treat them in a positive way without making them feel that you have been overtly demanding of their time or resources, or potentially disrespectful of their views and opinions (some may call this ‘organisational politics’). The interpersonal skills you develop here will hold you in good stead for later in your career, particularly the ability to listen effectively, acknowledge and, where appropriate, taking on board differing perspectives.

40

2 What Skills Do I Need?

Academic skills You were probably wondering when we would get around to the skills and techniques related to your ability to actually conduct research, but we have deliberately placed this further down the list in order to emphasise that it is the absence of the less traditionally-perceived skills that can actually trip you up if you are not aware of their importance. In relation to academic skills, most universities focus on the more pragmatic skills necessary for undertaking research, such as methodological, statistical and other data analysis skills and thesis writing. However, there is a lot more to consider, ranging from having a broad conceptual understanding of the research environment and appropriate research skills to address the research problem in question through to the practicalities of data gathering, analysis and use of supporting technology. Ability to think and conceptualise systemically would also be included here, where you need to consider carefully the assumptions that will guide your research and how best to frame and contextualise your research (we will have much more to say about these matters in later chapters). Immediately relevant to a new doctoral student is an understanding of the relevant institutional policies, expectations and requirements particularly around candidature, confirmation, submission, ethics, rights and obligations and reporting requirements as well as knowledge of resources and support systems. At a broader level is knowledge of the research environment. This is the ability to conceptualise your work within the general research environment, and to be able to demonstrate an understanding of the context in which your research takes place at local and international levels. It also involves being involved in significant academic and professional networks related to your research, being acquainted with sources of funding, being able to judge standards of good research practice and being aware of the issues relating to intellectual property and commercial exploitation of research results. This may also extend to understanding relevant professional and societal expectations and where you stand in respect of these. This could be in regard to publication standards, cultural expectations as well as career expectations and goals. Research skills most commonly spring to mind when reflecting on the skills required for successful completion of a doctorate (Garcia-Perez & Ayres, 2012). These skills cover the key elements of the research process and specifically relate to the ability to: develop original research questions from problems identified; have a comprehensive and critical knowledge of both the current and historical literature, theoretical foundations, theoretical concepts and relevant methodologies and techniques; frame and configure a research project and be cognisant of ethical requirements in relation to conducting research; justify the research approaches taken; negotiate research obstacles; analyse and synthesise data; evaluate findings; develop coherent arguments; present findings, document, report and reflect on progress; and recognise the means by which research is evaluated. Wardenaar et al. (2014) listed “seven skills needed to work as an independent researcher, namely the ability to (1) formulate a good research question, (2) apply properly research methods and techniques, (3) link own work to relevant theories within academic specialization, (4) develop and maintain relations with colleagues in the wider

2.3 Key Skills

41

research community (5) work independently, (6) show leadership and (7) reason analytically” (Wardenaar et al., 2014, p. 114). At the practical end of the spectrum are the skills associated with the mechanics of a doctorate. If the research process is the ability to drive the car, the technology skills are the nuts and bolts of how the car actually works. By technology skills, we are referring to either having, or developing, suitable technological and analytical capabilities applicable to your project. They largely centre on instrumentation, hardware, software, data management, analysis and storage associated with the need to acquire, record, collate, store, retrieve, manipulate and analyse your data. One area where students often lack confidence, if it is required by the nature of their research project, is in their analytical abilities. They are, therefore, advised to seek out advisory support. For example, if you gather quantitative data, the statistical procedures you will use are dependent on the configuration of your research and quality of your measurements and the implementation of those procedures will differ depending upon the computer software system you use. If you are not sure how to best analyse such data and/or what computer software system to use, it is imperative that you seek advice early in your doctorate, prior to your data collection (we offer some further advice on these issues in Chap. 21). You may otherwise end up backing yourself into a corner where you are unable to enact the appropriate statistical techniques because you have not collected your data in a format appropriate to a specific type of analysis. To avoid this problem, spend some time with a statistical advisor (one may be available through the graduate office or a psychology, business or statistics department) and double check that you are not limiting your options—go with your proposed methodology in mind to ensure that the data collected, both qualitative and quantitative, can be appropriately analysed. For example, if you propose to use a questionnaire, take an example of the questionnaire and also preferably some pilot test data. Similarly, if you gather qualitative data, it will be useful to seek advice on different ways of handling analyses, often using a computer support system, to help you to achieve the appropriate depth of learning you want to achieve (again, we will explore these issues further in Chap. 21). The growth of archival and analytical tools available has intensified the need for information skills in academic and professional research and you will need to develop those skills early in your research journey. As there is likely to be an abundance of material relating to your topic from a variety of predominantly digital resources contained in libraries, organisational records and online databases, it is in your interests to acquire skills in e-literacy. This is one step beyond undertaking internet searches, to actually utilising databases and recording literature sources through some form of archival system. It is a good idea to get to know the library and its electronic resources and search facilities early on. Participate in a course or workshop, if necessary (most libraries run programs at the start of each academic semester). Essentially, e-literacy skills relate not only to resources for acquiring and collating data, and data management, but also to the presentation of results as well as the writing up of a research outcome like a thesis.

42

2 What Skills Do I Need?

Belief in yourself Although not directly a skill per se, belief in yourself is a desirable asset and invoking it, even when you don’t feel like it, is an essential skill. Insecurity is perhaps one of the most common and debilitating emotions felt by research students. Insecurity can be manifested in concern about whether their writing is up to scratch, whether their research will be original or, possibly, just a strong desire to receive the approval of the supervisors and others. The way to handle this is to be persistent, persevere with your work and steel yourself to deal with setbacks or just the sheer tedium of many of the tasks you must undertake. Remain outcome-focused yet receptive and open to any critique of your work (remember, the feedback is being given on the ideas and the material, not on you). For most postgraduate students, it is determination and application rather than brilliance that are needed to complete a doctorate (Phillips & Pugh, 2015). If the opportunity arises, speak to a recently completed doctoral candidate. Ask them about some of the problems that cropped up in the course of their research, what were some of the obstacles they were able to overcome and what were some of the obstacles they had to recognise as being insurmountable and had to work around? More importantly how did they persevere during this process? What you will probably find is that they just hunkered down and demonstrated their tenacity. This particular characteristic is also necessary when you are reaching the final stages and writing up your research outcome, when you are often hitting the boring parts of having to polish the outcome and work through the various drafts and the feedback from your supervisor. Just when you think you have it right, your supervisor may require additional changes. You will need to be tenacious, to stick with the process, even though you are becoming heartily sick and tired of it. Don’t underestimate the emotional highs and lows you will go through in conjunction with the research process. Some days you will feel on top of it and that it’s all coming together nicely; other days you may receive devastating news such as an organisation or individual who does not wish to participate with your data gathering. So, let’s spell it out straight away—you will, at different times feel euphoric, when you receive positive reviews, chronically uncomfortable when you are presenting at a conference for the first time, devastated when you are subject to the scathing critique of a newly-minted PhD who wishes to grandstand at a research seminar, or extremely disappointed at data being lost or corrupted. I could go on with more examples as there is a litany of experiences which could swing your moods. The important thing is to recognise that they will occur and that it is up to you to manage them effectively. Going into a funk for weeks will not help. Somewhere along the line you need to be resilient to these issues and to move on. You will almost certainly encounter problems that will set you back. If you calculate the time loss, it could take just an afternoon or months of work to rectify. At this time, you can either give up, mope around and become depressed or, alternatively, tap into your resilience, re-frame the situation and get on with the job of completing your research journey. This is part of the learning process. But it is

2.3 Key Skills

43

not only the problems that arise that can derail you, it can also be the lack of excitement as you struggle through the many routine tasks associated with your research. As one young actor commented, he gets paid not for the acting but for the immense amount of time spent standing around. Similarly, a doctoral student will find that between the periods of excitement and insight, there are large chunks of tedious downtime doing a number of routine tasks (e.g., entering data is boring, typing in bibliographic notes can be tedious, but it needs to be done). Trying to maintain your level of motivation at such times can be difficult, but it helps to recognise that valleys will occur amongst the peaks and with a lot more regularity than you might anticipate at the start of your research process. Be professional. In practical terms having a professional demeanour refers to a number of different types of circumstances where specific behaviour is expected of you. This includes, for example, adopting a respectful approach towards individuals (especially those who hold divergent views to you), punctuality, observance of meeting protocols, being socially aware, empathetic and conscientious, being positive and supportive and (at the risk of sounding like your mother) presenting yourself in a professional, business-like manner in regard to your dress. Lastly believing in yourself also involves respecting and looking after yourself. This will involve endeavouring to maintain a balanced life where there is time for work, research, exercise, friends and family. Perhaps not as much time as you would like but you will be attempting to maintain your health and wellbeing and personal life along with your doctoral studies.

2.3.2

What Additional Skills Are Needed to Successfully Complete a Professional Doctorate?

Over and above those skills listed above, we would add the following skills for postgraduates undertaking a professional doctorate, drawing upon a broad reading of professional doctorate research, including Banerjee and Morley (2013), Doncaster and Lester (2002), Doncaster and Thorne (2000), McWilliam et al. (2002), Neumann (2005) as well as on our own supervisory and doctoral program development experiences. Critical reflexivity for learning While undertaking a PhD requires a certain degree of critical reflexivity (especially captured in the research journal), this skill really comes into its own in a professional doctorate. Here we are talking about an ability to critically reflect on your own work as well as from feedback and data from other sources. Such reflexivity requires a capacity to remain open-minded when reflecting on what you may or may not have learned (including an active appreciation for diverse world views), a capacity to change and adapt when evidence and circumstances warrant and a capacity to maintain a learning attitude throughout your research journey. Critical

44

2 What Skills Do I Need?

reflexivity needs to focus not only on the academic aspects of the research problem but also on professional/practice-oriented and organisational/community aspects as well as on intersections between those contexts. In a sense, critical reflexivity requires systems thinking in that you are trying to accumulate learning within as well as between relevant contexts and, in general, this contextual connectivity is supposed to be much deeper in a professional doctorate than is it is for most PhD research. Willingness to work in multidisciplinary/transdisciplinary spaces In many cases, professional doctorate research, because of its deeper contextual connectivity and problem-centred orientation, moves into multidisciplinary and/or transdisciplinary spaces. Multidisciplinary research involves reliance on two or more discipline areas, but with each maintaining its own autonomous identity (Leavy, 2011, p. 18). Transdisciplinary research works across two or more discipline areas but in such a way as to realise and capitalise upon the synergies that emerge between them (Leavy, 2011, pp. 24–26). As a consequence, new conceptualisations and approaches may emerge to inform a more holistic orientation. This requires openness to new ideas and to new ways of thinking. Another implication of this skill, arising from the fact that professional doctorate research spans academic and non-academic contexts, is that unitary disciplinary approaches can seldom span the divide. Here, you need to think beyond the boundaries of a single discipline in order to enhance your connectedness with multiple contexts. Ability to connect/communicate with diverse audiences This skill is strongly associated with both critical reflexivity and willingness to work in multidisciplinary/transdisciplinary spaces but focuses on how the connections between and within academic and non-academic contexts are established and maintained. This is indeed a communication skill, but with an order of magnitude more complexity built in. This is because you are connecting with diverse audiences (say, other academic researchers, community members, organisational employees, policy makers) in pursuit of a problem solution or an innovation and each audience may use different language and concepts that you have to relate to. For example, you will not get very far in researching a potential solution to a community problem if you can’t connect with members of the community. However, at the same time, you will not successfully earn your professional doctorate if you can’t connect with members of academia. This extends not only to managing verbal and non-verbal communication in conversations with members of these diverse audiences, but also to written communication as you write up your research outcome (e.g., a research portfolio) in a way that will speak to those diverse audiences. For some professional doctorates, like the PhD.I at the University of New England, examiners will be drawn from these diverse audiences, which will make this more complex communication skill even more imperative. This rounds off our two lists of key skills and, as you can see when you include related abilities, the list is extensive. It is now up to you to demonstrate

2.3 Key Skills

45

self-awareness as to what skills you currently possess and, more importantly, those skills that you need to develop, especially in light of the type of doctorate you are undertaking.

2.4 2.4.1

Skills Audit How Can I Assess My Skills?

As we have reinforced before it is worth taking the time to learn about yourself, your strengths, and your weaknesses. Become aware of your skills and capacities, cognitive abilities, preferences, as well as limitations. Research has indicated that what postgraduates themselves have identified as clear deficits are “their ability to work in teams, to work to an interdisciplinary context, to manage projects and budgets and to write grants” (Westen & Lawson, 2008, p. 27). However, you might be aware of other potential skills deficits specifically relating to yourself. Take a piece of paper and list what you perceive, from the discussion above, are your strengths and, in a different column, what might be your weaknesses. It is the weaknesses column you should be most attentive to. It is incumbent on you to look for opportunities for workshops, training sessions, training seminars or other students who can assist you with improving those weaker skills. If you are wanting a more structured approach to investigating your current skill levels in order to identify areas for further development you may want to check out the skill audit form from Queensland University of Technology (QUT), which could be helpful (HDR Skills Audit for HDR Candidates https://cms.qut.edu.au/__ data/assets/pdf_file/0008/636155/skills-audit-research-application.pdf). To assist you with strategies for skill improvement, another excellent tool can be found at the University College London website which contains a Professional and Personal Development framework https://www.ucl.ac.uk/ppd centred around: • • • •

Academic skills. Self-management skills. Communicating skills. Working with others/Interpersonal skills.

In addition to web-based resources for 24 sub-skills, there is also a downloadable PDF containing a Professional and Personal Development Booklet: https://www. ucl.ac.uk/ppd/docs/Student_PPD_Booklet_2016.pdf. Finally, we have designed a simple questionnaire (see Appendix) for conducting a skills audit based on the discussions in this chapter. By completing and scoring the questionnaire as instructed, you should gain at least a rough idea of what your strengths and weaknesses are and where you might need some development work.

46

2.5

2 What Skills Do I Need?

Conclusion

There is a wealth of related terminology in reference to skills required by postgraduates and, for the purposes of our discussion, we have been using the term ‘skills’ broadly to encompass attributes, traits and competencies. A keen observer will note that, essentially, the skills required for postgraduate study can be divided into three categories: • Personal skills—e.g., project management, time management, personal motivation, enthusiasm, pro-activity, your openness and receptivity to new ideas, problem solving, resilience, budgeting, creative thinking, stress management, as well as networking and team work, which is largely the ability to work constructively and positively as a member of a research community, managing relationships with supervisors and others, perhaps from very diverse groups. • Professional skills—e.g., possessing an understanding of the environment within which you work and of the impact that you might have on this environment as a researcher, the ability to write clearly, undertake critical evaluation, construct coherent arguments, demonstrate reasoning skills, defend and promote your research, contributing to public understanding of one’s research field, give constructive feedback to peers, listening skills and abiding by research ethics protocols. • Technical skills—e.g., synthesising prior research to gain an understanding of the current body of knowledge, problem recognition and refinement, identification of appropriate methodologies, data gathering, your ability to manage and analyse your data appropriately and to present that data, theoretical development, dealing with large documents, grant writing and the ability to transfer research skills into a work environment. To go back to the original question that you are, no doubt, asking yourself, “Do I have what it takes?” The answer is, yes, and if you don’t, you will acquire it during the course of the research journey. You will, however, need to be honest about which areas you will need to work on and to actively seek out training, tools, techniques and experiences which will enhance any skills deficits you have identified. It is likely that your university will have a formal training program for research students for the development of necessary research, professional and personal skills. If it hasn’t, or the program is weak, then take you will need to take the initiative for developing these skills. Be open to learning where and how you can acquire those skills you need. An inability to perform a certain task will hold you up and, ultimately, could de-rail you. It is, therefore, important to face up to something you can’t do and seek help, and to acquire the skills necessary to move on to the next stage. Whatever your starting position, it is important that, as a postgraduate student, you are conscious of what is needed, what your level of ability is, and what you might do in order to acquire the necessary skills. The responsibility for acquiring those skills must reside with you or with you in conjunction with your supervisor(s).

2.6 Key Recommendations

2.6

47

Key Recommendations

To conclude this discussion, our key recommendations are: • Do your own skills audit; find out what you are good at, and what you need to develop further. Upon evaluating your skill level, keep a copy and date it, work out strategies for focusing on particular areas you might perceive as being weaknesses, and undertake the audit again later to reflect on how your competencies may have changed over time. • If you are doing a professional doctorate, remember the augmented skill set you will need to manage communications with a more diverse range of audiences and disciplines and remember to engage in critical reflexivity as you do. • Confront your fears and concerns, whether they are public speaking, sitting examinations, effective academic writing, or receiving negative feedback or criticism (they are the very common fears). Part of confronting your fears is developing positive strategies to cope with them; if you need it, seek help and advice in developing such strategies. • When assessing your skills, do not be delusional; be realistic about your strengths and weaknesses, and do not hesitate to seek support for strengthening those areas that may hold you back. • Students who are intellectually curious, draw on a wide academic spectrum of thought, critically evaluate prior research and data and are adaptable will do well. • Doing a doctorate is an apprenticeship—you are not expected to have all the answers, so don’t feel embarrassed if you seem to be asking a lot of questions. Remember, the only dumb question is the question you didn’t ask. • This is probably one of the largest projects you will ever undertake so there is a necessity to plan the project out, get organised and be willing to re-arrange your schedule and reduce some activities you cherish to free up time. You may need to negotiate some activities and time commitment with significant others so that you don’t do damage to valued relationships as you progress through your research journey. Remember that while it is your research journey, others, who are important to you, will be along for the ride as well. • You are not always going to receive instruction from your supervisor(s) and often you will be working things out on your own. You will therefore need to take the initiative, self-motivate, make decisions and power yourself through the project. • Avoid reliance on others to do research tasks in the areas where you lack skills, particularly in areas such as data transcription and analysis. You need to be responsible for all aspects of your research project and, once again, remember that your research journey is just as important as the research outcome, whether it be a thesis or portfolio. That is, the academic and other skills you acquire during the process will hold you in good stead for performing as a researcher on future projects.

48

2 What Skills Do I Need?

• In addition to the mechanics of data collection and analysis a doctorate is largely about effectively communicating your research journey. Think of it as a telling a multi-layered story and be able to communicate this succinctly both verbally and in written form. • Have confidence that with commitment and stamina you will be able to complete this project. You will also need a healthy dose of resilience when dealing with criticism, setbacks, mistakes, loss of data and the general frustrations that arise during your research journey.

Appendix: Ph.D. Skills Questionnaire (Source: Authors) Step 1: Respond to the statements below in relation to your honest perceptions of your own abilities: use 1 for strongly disagree, 2 for disagree, 3 for neither agree nor disagree, 4 for agree, 5 for strongly agree. Circle the appropriate number for each statement. Strongly disagree

Disagree

Neither agree nor disagree

1. I possess a genuine desire to explore.

1

2

3

4

5

2. I am able to dispassionately critique the literature as well as my own research. 3. I have an open attitude and am opportunistic.

1

2

3

4

5

1

2

3

4

5

4. I am able to plan effectively, set priorities and manage multiple activities.

1

2

3

4

5

5. I am very good at managing my personal as well as financial resources.

1

2

3

4

5

6. I have a talent for writing effectively.

1

2

3

4

5

7. I fully understand the university policies and support services available to me.

1

2

3

4

5

8. I have the gift of being persistent and persevering with my work.

1

2

3

4

5

9. I am good at asking insightful questions. 10. I have the ability to synthesise information to create an evidence-based argument.

1 1

2 2

3 3

4 4

5 5

11. I possess a willingness to adapt and learn.

1

2

3

4

5

12. I have the ability to project manage and meet related milestones.

1

2

3

4

5

13. I am self-disciplined and able to work independently in unfamiliar surroundings.

1

2

3

4

5

14. I am able to verbally communicate my ideas and actively engage in discussions.

1

2

3

4

5

15. I have knowledge of the research environment in which I will be operating.

1

2

3

4

5

16. I would describe myself as tenacious.

1

2

3

4

Agree

Strongly agree

5

(continued)

Appendix: Ph.D. Skills Questionnaire (Source: Authors)

49

(continued) Strongly disagree

Disagree

Neither agree nor disagree

Agree

Strongly agree

17. I demonstrate humility and am aware of what I don’t know.

1

2

3

4

5

18. I am able to bring information together to present a coherent argument.

1

2

3

4

5

19. I have the knack of coping with ambiguity and managing change.

1

2

3

4

5

20. I am good at making decisions and monitoring my progress to achieve results.

1

2

3

4

5

21. I am self-motivated and able to take initiative.

1

2

3

4

5

22. I am an active listener and can contribute to the research and learning of others. 23. I possess all the appropriate research and analytical skills for my project.

1

2

3

4

5

1

2

3

4

5

24. I have the capacity of being resilient and can rebound from disappointments.

1

2

3

4

5

25. I have the talent to be creative and to look at things from different perspectives.

1

2

3

4

5

26. I am good at drawing on appropriate concepts to arrive at a critical assessment.

1

2

3

4

5

27. I am able to problem solve and overcome obstacles.

1

2

3

4

5

28. I am a very organised person.

1

2

3

4

5

29. I believe I am fully responsible for my learning, the process and the outcomes.

1

2

3

4

5

30. I work effectively with my supervisor and ask for help when needed.

1

2

3

4

5

31. I am capable of using computer applications appropriate to my project.

1

2

3

4

5

32. I pride myself on being professional at all times.

1

2

3

4

5

33. I am able to think inventively and to develop original approaches to problems.

1

2

3

4

5

34. I am able to achieve the right balance between breadth and depth of focus.

1

2

3

4

5

35. I am responsive to feedback.

1

2

3

4

5

36. I am very effective at managing my time.

1

2

3

4

5

37. I have undertaken a frank assessment of my own strengths and weaknesses.

1

2

3

4

5

38. I am good at networking, maintaining connections and promoting my research. 39. I am able to effectively analyse data to achieve research outcomes.

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

40. I am good at respecting and looking after myself.

(continued)

50

2 What Skills Do I Need?

(continued) Strongly disagree

Disagree

Neither agree nor disagree

Agree

Strongly agree

41. I am able to critically reflect on information from academic as well as non-academic audiences.

1

2

3

4

5

42. I do not have a rigid allegiance to a single discipline.

1

2

3

4

5

43. I am comfortable interacting with academics as well as non-academics.

1

2

3

4

5

44. When I receive unexpected or contrary (to your views) input or feedback, I critically consider it rather than dismiss it.

1

2

3

4

5

45. I can draw upon different disciplines when I consider a research problem.

1

2

3

4

5

46. I know I can modify my communication content when I interact with multiple audiences.

1

2

3

4

5

47. I am open to new perspectives that I encounter.

1

2

3

4

5

48. I can comfortably interact with others who come from different disciplines.

1

2

3

4

5

49. I respect the relative value and importance of academic and non-academic professional/practice-oriented perspectives.

1

2

3

4

5

50. If I discover my perspective to be incorrect, I modify it accordingly.

1

2

3

4

5

51. I can balance my focus on academic disciplinary matters as well as professional/practice-based matters which may invoke different disciplines.

1

2

3

4

5

52. I can tailor my use of jargon to best meet the needs of my audience.

1

2

3

4

5

53. I am able to think systemically, i.e., I can see the “forest and the trees” in my professional workspace.

1

2

3

4

5

54. I am comfortable questioning the boundaries of my own discipline. 55. I respect the value of non-academic input into research.

1

2

3

4

5

1

2

3

4

5

The following questions are primarily for postgraduates pursuing or intending to pursue a professional doctorate, but may also be relevant to PhD students, depending upon the nature of your research project:

Appendix: Ph.D. Skills Questionnaire (Source: Authors)

51

Step 2: In each skills table, record your score for each of the identified item numbers, then add up the scores in each table in order to identify your areas of strength and weakness. Intellectual curiosity Item

Critical thinking Score

Item

Score

Flexibility Item

Project management Score

Item

Score

1

2

3

4

9

10

11

12

17

18

19

20

25

26

27

28

33

34

35

36

Total

Total

Total

Total

Self-management

Communication

Academic skills

Belief in yourself

Item

Item

Item

Item

Score

Score

Score

5

6

7

8

13

14

15

16

21

22

23

24

29

30

31

32

37

38

39

40

Total

Total

Total

Score

Total

Critical reflexivity

Trans/ multi-disciplinarity

Complex communication

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

Total

Total

Total

Step 3: Rank your skills scores: What were your three highest scoring skill(s): ————————————————————————————— ————————————————————————————— ————————————————————————————— What were your three lowest scoring skill(s): ————————————————————————————— ————————————————————————————— —————————————————————————————

52

2 What Skills Do I Need?

Step 4: Consider what actions you can take to improve your lowest scoring skill areas. [If you are undertaking a professional doctorate or are undertaking a practice-based PhD research project, consider how you might improve your skill set if the Critical Reflexivity, Trans/Multidisciplinarity or Complex Communication dimensions were amongst your lowest skill scores and/or were absent from your highest skill scores.]

References Banerjee, S., & Morley, C. (2013). Professional doctorates in management: Toward a practice-based approach to doctoral education. Academy of Management Learning & Education, 12(2), 173–193. Benn, K., & Benn, C. (2006). Professional thesis presentation: A step-by-step guide to preparing your thesis in Microsoft Word. New Zealand: Pearson Education. Bolker, J. (1998). Writing your dissertation in fifteen minutes a day: A guide to starting, revising and finishing your doctoral thesis. New York: Holt Paperbacks. Conn, V. S., Zerwic, J., Rawl, S., Wyman, J. F., Larson, J. L., Anderson, C. M., et al. (2014). Strategies for a successful PhD program: words of wisdom from the WJNR editorial board. Western Journal of Nursing Research, 36(1), 6–30. Council of Australian Deans and Directors of Graduate Studies. (2018). Retrieved July 22, 2018, from https://www.ddogs.edu.au/. Deconinck, K. (2015). Trust me, I’m a doctor: A PhD survival guide. The Journal of Economic Education, 46(4), 360–375. Doncaster, K., & Lester, S. (2002). Capability and its development: experiences from a work-based doctorate. Studies in Higher Education, 27(1), 91–101. Doncaster, K., & Thorne, L. (2000). Reflection and planning: essential elements of professional doctorates. Reflective Practice, 1(3), 391–399. Dunleavy, P. (2003). Authoring a PhD: How to plan, draft, write and finish a doctoral thesis or dissertation. Basingstoke: Palgrave Macmillan. Fink, D. (2006). The professional doctorate: Its relativity to the PhD and relevance for the knowledge economy. International Journal of Doctoral Studies, 1(1), 35–44. Garcia-Perez, A., & Ayres, R. (2012). Modelling research: a collaborative approach to helping PhD students develop higher-level research skills. European Journal of Engineering Education, 37(3), 297–306. Gardner, S. K., Hayes, M. T., & Neider, X. N. (2007). The dispositions and skills of a PhD in education: Perspectives of faculty and graduate students in one college of education. Innovative Higher Education, 31(5), 287–299. Gilbert, R., Balatti, J., Turner, P., & Whitehouse, H. (2004). The generic skills debate in research higher degrees. Higher Education Research & Development, 23(3), 375–388. Lawton, D. (1997). How to succeed in postgraduate study. In N. J. Graves & V. P. Varma (Eds.), Working for a doctorate: A guide for the humanities and social sciences (pp. 1–17). London: Routledge. Leavy, P. (2011). Essential of transdisciplinary research: Using problem-centered methodologies. Walnut Creek: Left Coast Press. Manathunga, C., Lant, P., & Mellick, G. (2006). Imagining an interdisciplinary doctoral pedagogy. Teaching in Higher Education, 11(3), 365–379. Maxwell, T. W. (2003). From first to second generation professional doctorate. Studies in Higher Education, 28(3), 279–291.

References

53

McWilliam, E. L., Taylor, P., Thomson, P., Green, B., Maxwell, T., Wildy, H., & Simons, D. (2002). Research training in doctoral programs - What can be learned from professional doctorates? Canberra, Australia: Commonwealth Department of Education, Science and Training (DEST). Mowbray, S., & Halse, C. (2010). The purpose of the PhD: Theorising the skills acquired by students. Higher Education Research & Development, 29(6), 653–664. MSU. (2018). PhD transferable skills. Michigan State University. Retrieved August 12, 2018, from https://grad.msu.edu/phdcareers/career-support/skills. Neumann, R. (2005). Doctoral differences: Professional doctorates and PhDs compared. Journal of Higher Education Policy and Management, 27(2), 173–188. Petre, M., & Rugg, G. (2010). The unwritten rules of PhD research. Open up study skills (2nd ed.). Maidenhead: Open University Press. Phillips, E., & Pugh, D. S. (2015). How to get a PhD: A handbook for students and their supervisors (6th ed.). Maidenhead: Open University Press. Platow, M. J. (2012). PhD experience and subsequent outcomes: A look at self-perceptions of acquired graduate attributes and supervisor support. Studies in Higher Education, 37(1), 103–118. Pole, C. (2000). Technicians and scholars in pursuit of the PhD: Some reflections on doctoral study. Research Papers in Education, 15(1), 95–111. QUT. (2018). HDR skills audit for HDR candidates. Queensland University of Technology. Retrieved August 12, 2018, from https://cms.qut.edu.au/__data/assets/pdf_file/0008/636155/ skills-audit-research-application.pdf. Scholz, J., Klein, M. C., Behrens, T. E., & Johansen-Berg, H. (2009). Training induces changes in white-matter architecture. Nature Neuroscience, 12(11), 1370–1371. Sinche, M., Layton, R. L., Brandt, P. D., O’Connell, A. B., Hall, J. D., Freeman, A. M., et al. (2017). An evidence-based evaluation of transferrable skills and job satisfaction for science PhDs. PLoS ONE, 12(9), e0185023. Single, P. B., & Reis, R. M. (2009). Demystifying dissertation writing: A streamlined process from choice of topic to final text. Sterling: Stylus Publishing. Stewart, A. L., & Stewart, R. D. (1993). Proposal preparation (2nd ed.). New Jersey: Wiley. Thody, A. (2006). Writing and presenting research. London: Sage Publications. UCL. (2016). Learning is personal: A guide to your personal and professional development at UCL. University College London. Retrieved August 12, 2018, from https://www.ucl.ac.uk/ ppd/docs/Student_PPD_Booklet_2016.pdf. UCL. (2018). UCL Personal and Professional Development. University College London. Retrieved July 22, 2018, from http://www.ucl.ac.uk/ppd. UMICH. (2018). PhD transferable skills. University of Michigan. Retrieved August 12, 2018, from https://careercenter.umich.edu/article/phd-transferable-skills. Walsh, E., Seldon, P. M., Hargreaves, C. E., Alpay, E., & Morley, B. J. (2010). Evaluation of a programme of transferable skills development within the PhD: Views of late stage students. International Journal for Researcher Development, 1(3), 223–247. Wardenaar, T., Belder, R., de Goede, M., Horlings, E., & van de Besselaar, P. (2014). Skill development in collaborative research projects: A comparison between PhD students in multi-actor research programs and in traditional trajectories. In T. Wardenaar (Ed.), Organizing collaborative research (pp. 108–129). Rathenau Institut: The Netherlands. Westen, M. & Lawson, A. (2008). Doctorates ailing on the world stage. The Australian, 5. Zeegers, M., & Barron, D. (2006). Generic skills training. In C. Denholm & T. Evans (Eds.), Doctorates down under (pp. 88–94). Camberwell: ACER Press. Zerubavel, E. (1999). The clockwork muse: A practical guide to writing theses, dissertations, and books. Cambridge: Harvard University Press.

Chapter 3

How Should I Record My Research Journey?

3.1

The Research Journal

A valuable social and behavioural research practice is to maintain a research journal (or diary or notebook) that records your entire research journey (Nadin & Cassell, 2006; Glaze, 2002; Ortlipp, 2008; Saunders, Lewis, & Thornhill, 2012, pp. 13–15; Stevens & Asmar, 1999, pp. 39–40). Your research journal would commence with your initial research thinking and ideas accompanied by critiques and summaries of key literature sources to help background and flesh out some of those ideas. Your journal can also be used to record any thoughts, hunches, propositions and theoretical ideas, reminders, critical thinking and annotations about the literature, considerations about positioning, details about relevant research contextualisation, arguments for positioning the research (including guiding assumptions and research goals and questions/hypotheses), details about planning and configuring the research, mindmaps, diagrams, sketches, constant reminders about quality criteria and meta-criteria and how you are travelling against them as well as to record the trade-offs, decisions and adaptive choices, with their attendant reasoning, that you made during the course of your journey (Ortlipp, 2008, discusses the importance of this particular use for a research journal in qualitative research; Carcary, 2009, refers to this purpose as maintaining a “research audit trail”). This means that for any specific research project, your research journal could potentially cover a journey spanning several years. If you use your journal to also record data, such as participant observations, interview details and contextual notes, important conversations (e.g., with supervisors, stakeholders and gatekeepers) and so on, it will essentially take on the role of field notes. A well-maintained research journal will thus provide you with a rich and unique source of textual and visual data that are analysable, which means your research journal can serve as a qualitative data source in its own right (Glaze, 2002; Lamb, 2013). In the end, your journal will provide an invaluable memory and reflection aid for producing specific research outcomes such as a postgraduate thesis, dissertation or portfolio, conference presentation or journal article. © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_3

55

56

3.1.1

3 How Should I Record My Research Journey?

Important Purposes for a Research Journal

Apart from providing a permanent record of your research journey, the process of writing and maintaining a research journal helps you to: • reflect more deeply on the evolution of your research using notations that clearly record contextual and literature-based observations as well as any thoughts that help you to situate your research more clearly; • reflect on patterns or critical events associated with your life more generally (day-to-day as well as longer-term) and on how they are influencing your research thinking and activities; • critically analyse and reflect on research outcomes (e.g., published and unpublished research) produced by others that help you to build the platform and context(s) for your own research; • clarify your own role(s) and expectations, which becomes very important when framing, positioning, planning and configuring your research, choosing data gathering strategies and identifying data sources; • clarify the roles and expectations of participants and relevant details about other data sources in the context of your research, which also becomes important when working through the processes of gaining ethics approval (where required) for your research and negotiating your access to research settings, research participants and, possibly, important contextual documents; • monitor where you are in your own research cycle(s), including milestones, accomplishments, inspirations and dialogues with colleagues, peers and other sources of advice and information, all of which are things that can help you to maintain your motivation throughout your research journey and, if your journal is properly chronologically maintained, you can review the status of your research at any point in time; • realise that there are potentially many different paths to achieving your research goals, and that negative outcomes aren’t necessarily fatal—something that will be reflected in the on-going record of the decisions and research trade-offs you make (including the limitations imposed or created by your choices), blind alleys identified, bumps in the road, unforeseen events, obstacles, unexpected outcomes from your analyses, emerging opportunities and changes of direction pursued as a result of what you are learning; • focus your attention on making careful and defensible choices with respect to the constraints you confront, intended starting and ending points for the research, pattern(s) of guiding assumptions (i.e., ‘paradigm’ choices) you adopted and why, strategies and approaches you decided on or rejected, type(s) of data needed to address your research questions, and planned strategies for approaching the analyses of data—all of which can help you to continually reflect on and perhaps re-appraise the choices you have made along the way and the reasons behind them;

3.1 The Research Journal

57

• reflect on your emerging analysis results, interpretations and conclusions as a path toward achieving a convincing story, i.e., what you are learning from your data; • reflect on specific observations and reflections amassed during your data gathering activities to help contextualise each data gathering event; • access and maintain a ready source of qualitative data to reflect on and analyse in support of your research, i.e., the contents of your journal can be analysed as easily as participant responses to survey questions or participant statements obtained through an interview; and • construct a more convincing anticipatory story of your research journey (i.e., a research proposal) and/or a more convincing holistic account of your research journey and the learning it yields in whatever research outcome you are producing (Engin, 2011, argued that this is an essential part of the scaffolding process for building up your own research knowledge).

3.1.2

Potential Contents for Your Research Journal

Your research journal entries should be chronologically ordered, detailed in content but flexible and adaptable to your changing needs. You can use your journal to record virtually anything that might be relevant to your evolving research project. Many of these things will prove useful down the track in helping you to address various research quality criteria and meta-criteria. Possible content for entries in your research journal and the function(s) they can perform are explored in Table 3.1 (note that the contents of the table are representative of the kinds of things you could record, but do not exhaustively list all possibilities).

3.2

Mechanics for Maintaining Your Research Journal

There are many possible ways to organise and structure a research journal, but you should always tailor your journal requirements to meet both your own needs and preferences as well as to meet the requirements of your project. Your journal should be portable and easy to access and maintain at any time. Some researchers still prefer to keep a handwritten journal, but digital journals are becoming more popular to maintain and are available for multiple software platforms (e.g., Windows, iOS (Mac), Android) and devices (desktop computers, laptops, tablets, smart phones) (see, for example, http://lifehacker.com/5246819/five-best-journaling-tools). Walsh and Cho (2012) did a comparison of paper lab notebooks (PLN) and electronic lab notebooks (ELN) for laboratory researchers and found that “ELNs easily surpass their paper competitors with the ability to quickly and easily store, search, access, and share information electronically” (p. 234) but that some “PLN users cited the

58

3 How Should I Record My Research Journey?

Table 3.1 Possible content for research journal entries and the function(s) they may perform Function Project Management

Personal Reflections on the Journey Serving as an Additional Source of Data

Working with Literature

Working with Literature

Research Planning & Configuration Systems Thinking & Adaptation

Research Planning & Configuration Project Management

Researcher Positioning Systems Thinking & Adaptation

Possible content/entries You can record your project timeline (see Chap. 5) as part of your journal so that you can continually track your progress and clearly link recorded entries to key points along your timeline where necessary. You can also record key goals that you have set, including reminders of your research objectives and key milestones that lie immediately ahead. You can record your own day-to-day feelings, emotions, moods, worries, doubts, needs, motivational highs and lows and any other information relevant to living your life as you progress through your research journey. You could think of this as recording how your life journey intersects, affects and is affected by your research journey. These entries can serve as additional data to consider as you formulate data interpretations and construct your final research outcome story. For example, if you are gathering qualitative data via interviews, recording such personal reflections may help you to appropriately characterise the quality of your interview interactions and their resulting data. You can record keywords and details of databases and search engines you use for your internet and electronic searches of literature and the web addresses of multimedia resources so that you can reproduce, or update searches as needed. You can record summaries of key articles and reports, theories and approaches as well as your thoughts and reactions to these and how you might see each piece connecting with your project. These could provide valuable information for any literature review components of planned research outcomes and can also help you to set up the arguments for where your research sits against the backdrop of past and current research. Things to record about any particular article, paper or report you might read could include: pattern(s) of guiding assumptions, key outcomes and implications, theories used/tested/constructed, other articles to follow up on, how you might potentially use the ideas or methods in the article, any quotes you might like to use (along with the page number that the quotation is taken from in the original source), and where you have stored the original article. You could draw a mindmap of an article or paper if you prefer that method of organising what you have read and learned. You can document your evolving thinking about the research frame(s), pattern(s) of guiding paradigm assumptions, Method Unit (MU) configuration, choices of data sources and data gathering strategies you want to use along with your reasoning behind your choices. Recording such thinking can help you to maintain consistency in your choices and understand their downstream implications. You can also record your thinking about the limitations and possibilities associated with your choices of research frame, guiding assumptions, MU configuration, data sources and data gathering strategies. You can record your thoughts regarding the resources you might need to make your research project feasible and practical to complete and what you need to do to access them. You can also document the processes you follow to clear various hurdles before you can commence your research, including your proposal presentation and the ethics approval process. You can document precisely where you are situated as a researcher in the context of your research. This could include recording your thoughts on your evolving relationships with various organisations and the participants or other data sources you need or want access to, any opportunities you took advantage of to gain such access, your values and expectations regarding the research context(s), your preferences for pursuing possible research outcomes as well as noting any knowledge or skill deficits you may need to overcome. In short, you can document how you see yourself fitting into the research context and what you might need to do to fit even better.

(continued)

3.2 Mechanics for Maintaining Your Research Journal

59

Table 3.1 (continued) Function Positioning of Participants & Other Data Sources Research Planning & Configuration Serving as an Additional Source of Data

Systems Thinking & Adaptation

Systems Thinking & Adaptation Research Planning & Configuration

Learning from the Data Building a Convincing Story Systems Thinking & Adaptation Building a Convincing Story Systems Thinking & Adaptation Research Planning & Configuration Systems Thinking & Adaptation Building a Convincing Story

Research Planning & Configuration Personal Reflections on the Journey Systems Thinking & Adaptation

Possible content/entries In your journal, you should document very carefully your thinking about potential participants and other desirable data sources (e.g. secondary databases, documents, media stories, minutes of meetings, and literature) in terms of who or what they are, where they are, their role and/or purposes and why you think they are appropriate to include in, or can contribute to, your research. Such content can constitute data, especially for research guided by an interpretivist/constructivist pattern of assumptions. You would do this so that all your data sources—human and non-human—can be appropriately situated in the context of your research. You can also record your thoughts about how you might choose data sources for inclusion, which will inform the development of your sampling strategies. You can and should document any constraints you had to live with and your reactions to them as your research project unfolded, including resource and access constraints, pressures to complete the project, unresolved issues between you and your research associates or supervisor(s) regarding the research and its conduct, response rate issues, and emergent methodological problems that you had to confront. To help you tell the story of your research methodology, it could be handy to record the sorts of problems you anticipated might emerge during your research, and to document your thinking and actions taken when addressing those problems. For example, anticipating a poor response rate for a questionnaire might lead you to employ additional incentives to encourage greater participation, and you could record your reasoning behind your adoption of the specific encouragement strategies. Your journal can be used to record your emerging interpretations and conclusions from each analysis you are employing as well as setting out reminders about further analyses you might think useful to pursue (even if not initially planned for). This will provide fertile material for constructing a convincing story in your intended research outcome. Your goal here is to flesh out your thoughts on what you are learning from the data you gather and how you might begin to bring various data-related stories together into a more complete, coherent and integrated larger story. To help make your research outcome story more convincing, you may find it useful to record your thoughts on what you learned from the research that you didn’t expect to learn. In other words, were some outcomes surprising? Why were they surprising? How can you make sense of or explain why they occurred? How could you or someone else deal with them in future research? As part of your research planning process, it is good practice to record your reasoning behind every research frame, configuration and methodology choice you made and any trade-offs you had to make to get keep your research on track. For example, you may have originally planned to conduct an systematic interview study in two capital cities, but due to resource constraints and your own family commitments, you had to settle for conducting your research in one city (i.e. the one where you live). In your journal, you could document this trade-off, and the reasons why you had to make it, and write explicit reminder notes to ensure you factor this trade-off into any generalisations made in your research outcome story. You may find it useful to record questions for relevant colleague(s), your supervisor(s) or other key people, such as gatekeepers, as they occur to you, and perhaps map out meeting agendas including issues to discuss with them. You would then also record the answers to those questions in your journal. You may find that some of these discussions become political (e.g., “I’ll do X for you if I get Y in return”) and create implications for conducting your research. Your journal can then become an important space for you to fully work through those implications.

(continued)

60

3 How Should I Record My Research Journey?

Table 3.1 (continued) Function Building a Convincing Story

Contextualisation Building a Convincing Story Data Recording & Management

Research Planning & Configuration

Systems Thinking & Adaptation Building a Convincing Story

Data Recording & Management Serving as an Additional Source of Data

Possible content/entries As your research unfolds, you may have some little gems of ideas that you think might help you make your research outcome story more convincing. For example, you might be worried about a specific limitation in your research. Then you suddenly realise that if you tell a clear story about this limitation, it will immediately give you a very powerful way to showcase your reasoning skills as a researcher and provide you with at least one future direction emerging from your research. Rather than hope you will remember these little ideas when it comes time to write, note them down immediately in your journal so that you don’t lose track of them. In your journal, you can make extensive notes on research context(s), whether in a laboratory, in the field or on a computer. These notes could cover anything from noting historical observations on specific organisations you are researching to recording specific features of workplace tasks you are trying to simulate with a task you are constructing for a laboratory experiment. Such notes will be invaluable to you in not only telling a richer story about your research, but also in ensuring that your conclusions and implications are appropriately sensitised to the context(s) in which they emerged. Your notes about research contexts may also help you to make sense of your results, especially the surprising ones. You can use your journal as a staging ground for developing your preliminary ideas (and their later refinements) on your approaches to gathering specific types of data and on your approaches to helping you analyse and make sense of those data. As part of the latter considerations, you can also make notes regarding how you will prepare data for analysis. One outcome from this type of documentation, if done well, is that your thinking in the journal can be translated in a straightforward way into the formal protocols and rules that will be necessary to guide either yourself or someone else (e.g. a professional transcriber or data entry specialist) in the data preparation process. You record thoughts in your journal with respect to how far you would like to generalise your outcomes or transport your learning to other people, groups, times and contexts. It is critical early on to get your thinking clear on this, because it can influence how you frame and configure your research, how you manage your sampling and data gathering strategies, how you approach data analyses and how you would like to conclude the overall story in your research outcome. You can thus use your journal notes and thinking to set clear boundaries around just how far you want to push your data, and it is likely that some of the trade-offs and constraints you have documented will influence your thinking. If you are conducting research under an interpretivist/constructionist pattern of guiding assumptions and are collecting qualitative data using strategies such as participant observation or in-depth semi-structured or unstructured interviews, then you will find your research journal (or a template variant of it, depending upon your needs) invaluable in helping you to record contextual observations and notes as well as to record your reflections on those observations and notes. For this purpose, your research journal takes on the role of field notes. Thus, for each qualitative interview, you could record information about the conduct of the interview (where, how, when) as well as how the interview dynamics unfolded, and you could then reflect on how you felt before, during and after the interview was completed. These notes would add richness as a second source of qualitative data to consider, alongside the actual interview transcript, when analysing and interpreting what happened and what was said.

3.2 Mechanics for Maintaining Your Research Journal

61

inability to freehand into a notebook as the number one reason for not switching to an ELN followed by the comfort and familiarity of using their current [PLN] method” (p. 230). Potential media for recording journals, many of which are technologically-driven or supported, include: • Handwritten: the oldest and for many, most comfortable and safest-feeling form of journal keeping, minimally requiring just pen and paper. An A4- or A5-size loose-leaf notebook would be preferable to a spiral-bound notebook because it is expandable through the addition of extra pages. Pages can be formatted anyway you like and drawings can be easily incorporated (even in multiple colours, if you have a set of coloured pens handy). • OneNote (https://www.onenote.com/): a general note taking and keeping software program, available for all operating system platforms (Windows, iOS, Android) and devices. OneNote can also work with Livescribe (described below). It can organise and manage text and image entry and storage and can support multi-user sharing of files. On some platforms (e.g., Microsoft Surface tablets), OneNote supports handwritten and hand-drawn inputs as well as typed input (it can also convert handwritten text to typed text). A free downloadable version of the program is available. This would help to offset the “inability to freehand” disadvantage of an ELN cited by some PLN users identified in Walsh and Cho (2012). • WordPress (https://wordpress.org/): a free, web-based, blogging software system that can be used to maintain a personal journal. • Evernote (https://evernote.com/): a multi-platform program designed to facilitate organising tasks and writing and works well on computer, tablet and smart phone platforms. As well, it can sync files across platforms. All files are stored online “in the cloud”. The basic version is free; upgraded versions have a yearly fee attached. Note that the Walsh and Cho (2012) study, cited above, used Evernote as their ELN comparison software package. • Word (https://products.office.com/en-au/word): a standard but highly flexible word processing program that comes with Microsoft Office, which also includes PowerPoint, which can be used to draw diagrams and visuals and Excel, which can be used to create various tables, including project management diagrams. Files can be stored locally on your computer as well as on an online cloud drive called OneDrive. Word works best for desktop computers, laptops and some tablets (such as Microsoft Surface). Microsoft Office software does incur an annual subscription fee. • Google Docs (https://www.google.com.au/docs/about/): another standard but highly flexible word processing program; along with Google Slides (PowerPoint-type program for producing drawings) and Google Sheets (Excel-type program for producing spreadsheets), offers similar capabilities to Microsoft Office, albeit for free. All files are stored online in a cloud drive called Google Drive (which can also have a local version resident on your computer or tablet). Google Docs works well for laptops, tablets and smart phones.

62

3 How Should I Record My Research Journey?

• DavidRM The Journal (http://www.davidrm.com/): a Windows-based program for setting up and keeping a journal. The software can manage images and offers high-level password protection. A free 45-day trial version is available, after which software must be purchased to continue use. Files can be stored locally on a computer or in a cloud drive, such as DropBox or Google Drive. • Livescribe (http://www.livescribe.com/au/): an integrated pen-and-notepad software system available for the iOS and Android platforms. Livescribe couples a Smartpen (which not only writes but also records and plays back audio) with special dot paper notebooks to record and convert handwritten notes and diagrams into digital form and tag those notes to recorded audio. Livescribe is essentially a hybrid approach to maintaining a journal, offering the comfort of handwriting, valued by many, with the security of electronic storage and connections between handwritten notes and audio recording. This would also help to offset the “inability to freehand” disadvantage of an ELN cited by some PLN users identified in Walsh and Cho (2012). Some might argue that, in today’s world, maintaining a hardcopy handwritten version of your research journal is a bit antiquated and that keeping an electronic version would be more desirable. However, many would argue that keeping a hardcopy journal makes the research continuously real and tangible (rather than ‘virtual’), and greatly reduces the potential risk from technological failures (hard disk drives and USB drives/flash drives don’t last forever, and cloud servers can be hacked). Certainly, you can also maintain your research journal in electronic form as this may enhance flexibility, portability and ease of potential incorporation of the contents into a qualitative data software support system, such as MAXQDA, NVivo or dedoose, for analysis purposes. It is even possible to keep your research journal in the form of a ‘blog’ on a website designed for that purpose (recall WordPress above). However, if you do adopt an electronic journal format, remember that you should always backup your electronic entries and files (daily, if possible, especially if you are recording entries daily) and keep a hardcopy of everything—just in case (for handwritten journals, keeping a duplicate stored in a different location is not a bad practice).

3.3

When to Record in Your Journal

Daily recording in a research journal will work for some of you, particularly if you are already in the habit of keeping a journal as part of your everyday life. Daily recording obviously requires some discipline to maintain the momentum. However, if you can maintain this discipline, you can minimise the influence of memory on your entries. For others of you for whom keeping a journal is an unfamiliar practice or where daily entries are not your preference, you may find that recording entries

3.3 When to Record in Your Journal

63

works better on a more episodic basis. For example, you may record entries focusing on events such as meetings or other working sessions, library or internet-search sessions, pilot tests and trials of your methodologies and specific data gathering activities (e.g., interviews, observations or questionnaire administrations). Whatever recording frequency and style you choose, you need to develop a system that works best for you and that you can commit to. Recording entries should be an activity done with a minimum of distraction in order to maximise your recall likelihood and accuracy. It will help your consistency if you work on your journal in the same workspace and at the same times of the day, wherever possible. A good practice to adopt is to reserve a block of time, say toward the end of the day, where you can do journal recording in a concentrated burst. This is because, once you start recalling the day’s events relevant to your research and recording things about those events, other associations, recollections and ideas will begin to emerge and you want to be able to capture these as well. If you try to do this task in very small bursts throughout a day, you may miss chances to capitalise on the rich associative nature of your own memory system. If you are recording episodically, try to record your thoughts as immediately as possible after the event occurs. Even if you are already part of the way through your postgraduate research, it is not too late to start building up your journal. You will have to do this retrospectively up until the time you start, of course, and this may be subject to memory limitations or distortions. However, if you can get as much down on paper/stored in computer about decisions you made along the way, avenues you decided to pursue or not pursue, important ideas you or your supervisor(s) had along the way, key conversations you can recall with your supervisor(s), and so on, you will at least have some sort of working document to fall back on when it comes time to write. If you have to build up your journal, at least partly retrospectively, a good idea would be to reserve a block of time where you can do the retrospective part in a small number of concentrated bursts. Again, this is because once you start recalling and recording, other associations and recollections will begin to emerge, and you want to be able to capture these as you go. This retrospective process can also work well if you have gaps in your journal where you either forgot to make some entries or did not have access to your journal to make the entries. The trick is to make those omitted entries as soon as access to your journal becomes available.

3.4

Illustrations of Research Journal Entries

We will illustrate some of the possibilities for a research journal, focusing mainly on a handwritten version. However, instead of simply filling a notebook with words on lined paper, try creating a more structured notebook page template. One possibility is illustrated in Fig. 3.1a. Such a template provides regions on the page for

64

3 How Should I Record My Research Journey?

(a) Possible blank journal page template Date: ___________

Time Spent on Project Today: ______ hrs

Page: ____

___________________________________________________________________________________

Notes

(b) Page recording journal article summary, personal thoughts about the article and its potential use and evolving research mindmap 29/10/15 3 Date: ___ ________ Time Spent on Project Today: ______ hrs

Ideas/Reflections/Actions

Notes

Ideas/Reflections/Actions

Omodei, M. M., & Wearing, A. J. (1995). The Fire Chief microworld generating program: An illustration of computer-simulated microworlds as an experimental paradigm for studying complex decision-making behavior. Behavior Research Methods, Instruments, & Computers, 27(3), 303316. ----------Computer simulation microworlds study under positivist guiding assumptions; describes a dynamic decision making software simulation system that attempts to represents the world of a fire chief fighting a forest fire in a laboratory situation. Many parameters of the microworld are controllable’ participants are to take the role of fire chief and make decisions for how best to allocating firefighting resources under dynamically changing contextual conditions. Software is comprehensive (Turbo Pascal language) with an extensive manual available. Authors convincingly discuss how representative the microworld is of real world tasks. Developed in Australian universities.

Maps & Sketches

17 Page: ____

Literature Review Article & Critique

Read this article today and found it relevant to what I am trying to do in my research. I am convinced that the simulation is fairly represented of a reallife dynamic decision task (but without participant exposure to actual risks). However, the study is 20 years old – would the software still be available, maintained, usable on current Windows operating system??? I would like to use this type of microworld simulation for part of my research and would prefer not to have to develop it from scratch. Reminder: check on pprogram g availability y and viability y ffor use

Maps & Sketches

research goals

(c) Page recording some questionnaire design decisions as well as some theoretical reasoning about construct relationships 7/7/13 Date: ___________

2.5 hrs Time Spent on Project Today: ______

6 Page: ____

Notes

16/8/11 Date: ___________

I feel like the questionnaire is coming together. Will try to produce a full draft this weekend and get feedback on it next week I am not sure what prediction to make about how OCBs would relate to normative commitment – need to look at some more literature here.

4 Time Spent on Project Today: ______ hrs

Notes

Ideas/Reflections/Actions

Interview was conducted at 2:30 pm in the interview room at Worldwide. Interview lasted 32 minutes. Jean came into the room, almost at a run, closed the door behind her and sat down next to me. She said “I’m so glad some one is actually interested in what I have to say about this restructure, no one has listened to me.” She reached over to the digital recorder and switched it on and said “I’m ready”. Once I asked how she felt about the restructuring of Worldwide, her story came out in a rush, full of emotion and sadness. She started tearing up as she spoke about one of her colleagues having to see a psychiatrist – I got the sense it was a close friend. She recovered very quickly without pausing and continued her story with what I took to be a stiff upper lip, tightly controlled, yet letting her emotion come through.

Jean was my second interview on the day and I was already feeling very tired. She was very full on in her drive to get her story out. I found her story very moving and had trouble not reaching out to comfort her when she started to tear up. However, I remembered my supervisor’s admonition not to get too involved or drawn into their stories because that could influence what happens and could mean I am losing my researcher perspective, and didn’t reach out.

Maps & Sketches

Maps & Sketches Demographics (see p. 4 of journal)

Aff f ective OC

+ OC items

Normatiive OC Normative Continuance OC

OCBs

-

OCB items

9 = very strongly agree 8 = strongly agree 7 = moderately agree 6 = slightly agree 5 = neither agree nor disagree 4 = slightly disagree 3 = moderately disagree 2 = strongly disagree 1 = very strongly disagree

Open-ended question about what OCBs mean to questionnaire participant?

7 Page: ____

Interview with Jean - Notes & Observations

Ideas/Reflections/Actions Felt good about today’s meeting. Deciding on which measure of OC to use was a big hurdle for me and it was good to finally decide.

Lab simulation software?

(d) Page recording event and setting details, participant details and observations about a qualitative interview

Discussions about questionnaire design and theoretical relationships Talked with supervisor today. We discussed which measure of organisational commitment we should include in the questionnaire instrument. Decided to go with the Meyer & Allen (1991) construct because it explicitly separates affective from normative and continuance commitment. Their instrument seems to have reasonable validation evidence, It seems likely, based on research by Podsakoff and others that our other important construct, organisation citizenship behaviour, will be strongly positively related to affective commitment (emotional attachment to org more likely to lead to behaviours beyond what is expected). However, we anticipate that OCBs will be negatively related to continuance commitment (staying with an org because you have to would be less likely to stimulate behaviours beyond what is expected in the job). We also began to lay out the structure of the questionnaire and the format of the items. We are going with a 9-point agree-disagree Likert-type scale for the OC measure because I wanted finer rating distinctions than a 5-point scale could give for showing change over time.

my dynamic decision study

Recorder

door

Jean chose to sit here

Me

female middle manager in former Accounting Dept; in her former job role 4 years; now in the new Accounting & Budgets Dept; her job role as a chief financial accountant was substantially reworked as a consequence of staff shedding in her department. Her new role now reflects aspects from the previous roles of two other accounting colleagues who worked with budgets as well as aspects from her own previous role. She now has responsibility for producing budget reports.

Fig. 3.1 Illustrative pages from a research journal: a blank template; b literature summary/ learning example c record of discussion with supervisor; d contextual details about an interview. Illustrative research journal pages (concluded) e literature summary/learning example in electronic form

3.4 Illustrations of Research Journal Entries

65

(e) Screenshot of the literature review journal page as it might appear in Microsoft OneNote

Fig. 3.1 (continued)

recording specific types of information. The illustration shown in Fig. 3.1a contains four distinct regions: (1) top of page for recording entry-specific information such as the date, length of time you have spent on the project that day (a motivational device), page numbers and a broad-level heading for the entry; (2) a large area reserved for Notes and observations, recording dialogs with others (e.g. research associates, supervisors, key gatekeepers), summaries and issues that arise from literature (e.g., articles or reports that you read); (3) a smaller area, Ideas/ Reflections/Actions, alongside the Notes region, for recording your self-reflections, thoughts, ideas and any planned actions that emerge from your thinking or dialogs with others; and (4) a Maps & Sketches region at the bottom for drawing maps, diagrams, mindmaps, conceptual frameworks, room layouts, and so on. The distinction between the Notes and the Ideas/Reflections/Actions regions is not too dissimilar from what Ballenger (2004, pp. 146–154) termed the ‘double-entry journal’. Notes are for factual ‘objective’ narratives (e.g., addressing the what happened-, what was said-, to whom-, where-, when-types of questions) and Ideas/ Reflections/Actions are for your personal reflections on that narrative and what it means (the why-, what does it mean-, so what- and how does it make me feel-types of questions).

66

3 How Should I Record My Research Journey?

Once you have your page template formatted, you can print it out and photocopy as many pages as you require. You may even find it useful to photocopy the template onto different coloured paper to differentiate between types of entries, e.g. entries made during the research planning stages, entries made during your reading of the literature, entries made while collecting data, and entries made during the data analysis stages. You could also use different coloured pens or highlighters to draw visual attention to key insights, ideas or planned actions. In addition, having your research journal as a loose-leaf notebook means that you can incorporate external pages and documents (e.g. your project timeline, draft or pilot versions of letters and instruments, draft proposal and presentation notes, and chapter feedback notes from your supervisor) where it makes sense to do so. Of course, you will need to make continuous decisions about what is important to include and exclude from your journal, so that it does not become cluttered with tangential or non-essential information and insertions. Figure 3.1b provides a concrete illustration of what a research journal entry might look like for summarising a specific research article (part of literature review activity, see Chap. 13). The Notes section could record the bibliographic information for an article that has just been read (formatted in APA style), followed by narrative details about the article. The authors’ guiding assumptions could be recorded as well as details about a computer simulation software program, called Fire Chief, its intended goals and where it was developed. The Maps & Sketches section shows an evolving mindmap characterising the research goals and emerging questions/issues stimulated by reading the article. The Ideas/Reflections/Actions section record reactions to the article, signals interest in its relevance to intended research into dynamic decision making. Importantly, in this section, concerns are raised about the age of the software and whether it might still be useful today, 20 years down the track (underlined text). The reminder in underlined red ink prompts a check into both the availability and the viability of the software. Additional notations flag being convinced that this laboratory-based software package presents a dynamic decision task that is broadly representative of the tasks confronted by decision makers in a real-life firefighting context but observes that this occurs without participant exposure to the risks that would attend the task in a real-life context. The journal entry illustration shown in Fig. 3.1c pertains to an investigation guided by the positivist pattern of assumptions involving the use of quantitative questionnaires by a PhD student. The top section heading indicates that the entry focuses on questionnaire design issues and some theoretical relationships between constructs. The Notes section entry summarises a discussion held with the student’s supervisor where a decision was made about a key construct. The entry highlights not only the decision made but also why, and tries to clarify some theoretical relationships they expect, given the construct decision. The entry also indicates that discussion also touched on the structure of the questionnaire, notes a further

3.4 Illustrations of Research Journal Entries

67

decision relating to the possible scaling of questionnaire items, and sets out reasons for that decision. The Ideas/Reflections/Actions section captures the student’s sentiments about the meeting itself and about the progress of the project. Here, the student also flags a need for further literature research (a planned action—underlined) to try to clarify a currently unresolved theoretical relationship. Finally, on the left side of the ‘Maps & Sketches’ section, the student has diagrammed the anticipated and as yet unknown theoretical relationships between the OC and OCB constructs. In the centre of the section, the student has sketched, in broad terms, the overall structure of the questionnaire along with an indication of the design of the nine-point response scale that would be attached to the OC and OCB survey items. This type of entry is likely to occur rather early in the research planning process. Three years later, when the student is writing up the Methods chapter for her thesis or a methods section for a conference paper, this entry would provide clear reminders as to why the Meyer and Allen conceptualisation of organisational commitment and associated scaling choices were made, rather than relying on the student’s capacity to remember the nuances and details of decisions made three years before. Figure 3.1d illustrates a rather different use for a research journal entry, namely as a device for recording observations and notes related to a specific qualitative interview. Here, the investigation is following an interpretivist/constructivist pattern of guiding assumptions in the context of a major restructuring change event in a company named ‘Worldwide’. The researcher uses a journal entry to: (1) note that the interview was with a participant named ‘Jean’ (a pseudonym); (2) indicate when and where the interview occurred and how the interview dynamics unfolded; (3) record the researcher’s own reflections and reactions to those unfolding dynamics (including setting out fears about possibly ‘losing my researcher perspective’, a problem known as ‘going native’); and (4) provide a sketch of the interview room layout, showing where the researcher, Jean, and the digital recorder were positioned. The diagram also notes that Jean chose to sit where she did, rather than the researcher telling her where to sit. We should note that the research journal entries illustrated in Fig. 3.1a–c, reflect a narrative style of entry. The researcher basically tells a mini-story about what happened. This is not the only entry style that could work. Some researchers may find that a bullet-point entry system works better, or maybe even a short-hand system. Others may be well-versed in mind mapping techniques and would choose to expand or emphasise the Maps & Sketches section relative to the Notes and Ideas/Reflections/Actions sections. Figure 3.1e shows how the research article journal entry page (shown in handwritten form in Fig. 3.1b) might appear using OneNote software on a Microsoft Surface Pro 3 tablet computer. Both typed and handwritten textual notes are accommodated and a mindmap image, redrawn using the Inspiration (www. inspiration.com) mindmapping software package, has been imported into the

68

3 How Should I Record My Research Journey?

journal page. One advantage of storing your journal in electronic form using software like OneNote is that the journal entries become fully searchable (note the Search box in the upper right side of the screenshot in Fig. 3.1e).

3.4.1

Using the Contents of Your Journal

Your research journal is an expanding and ever-evolving record of your research journey. Used correctly and with discipline, it can map out your entire journey, including all its bumps, fits and starts. Probably the best mindset to adopt with respect to using what is contained in your research journal is to consider it as a distinct source of qualitative data. These ‘data’ may prove useful not only for helping you flesh out and enrich your research story, but also to qualify, amplify or modify your learning from other data sources (see Lamb, 2013, for a personal account of how keeping a research journal assisted in a number of aspects of his research). Research journal entries that focus on the literature you have read and theoretical frameworks, methodological approaches, constructs, ideas and gaps you have identified can inform not only your research planning and configuration but also your writing of Literature Review sections or chapters in various research outcomes. They can help you to clarify why you looked at certain material and how you have responded to that material as well as tracing how the literature has then influenced the shape and conduct of your research. Research journal entries that focus on pattern(s) of guiding assumptions, frame choices, MU configuration, data source sampling, data gathering strategies, strategies for gaining access and ethical clearance, and analytical strategies can inform your writing of a Methods chapter or section in a research outcome and may help you to openly defend the choices you have made to the reader. This will add to the convincingness of your research. Additionally, they may help you to appropriately qualify or contextualise your findings in a Results chapter or section. Research journal entries that focus on any aspects of the research context itself as well as the role(s) that you and your participants played in that context can help you to contextualise your entire research. This would certainly help you to develop any conceptual framework for your study, defend any methodological tailoring you had to do, and qualify, amplify or further characterise your results and their implications. Thus, research contextualisation entries may assist you in every phase of your research planning and writing endeavours. If your methodology involves detailed interactions with research participants and you have used your research journal to document details of those interactions, then those journal entries can help you tell a much clearer and more convincing story about what you have learned from each interaction. This will be of most value in the writing up of your results and their implications. If your research is based on an

3.4 Illustrations of Research Journal Entries

69

interpretivist/constructivist pattern of guiding assumptions, such entries can be of great assistance during the data analysis stages of your study. Specifically, reflecting on your thoughts during the data gathering processes (including your emerging theoretical ideas and interpretations) becomes a key strategy for (1) ‘bracketing your preconceptions’ at the point of data gathering (i.e. keeping your thoughts and perspectives separate from your observations, conversations and interpretations), and then (2) allowing your recorded views to be explicitly considered when making sense of the data after they have been gathered. In a more general sense, maintaining a research journal is an exercise in self-reflection. This process is termed reflexivity or ‘reflection on [your] own data-making role’ (Richards, 2009, p. 49) and is an analytical strategy advocated by interpretive researchers, including those employing a ‘grounded theory’ approach (see Carcary, 2009; Charmaz, 2014, Chap. 2; Flick, 2002, Chap. 14; Nadin & Cassell, 2006; Ortlipp, 2008). This is where the power of your journal really lies. You decide what is important to record and you record how you react to or think forward from what you have recorded. Thus, your journal evolves dynamically. For example, you might identify and record areas of weakness in your skill set and record plans to undertake activities to strengthen those areas (e.g., enhancing your competence in a specific type of data analysis or in conducting in-depth interviews). Using your journal, you may become aware of and record your thinking and logic about the guiding assumptions you might want to adopt. You might record your thoughts on things you have observed or heard and reflect on how you may or may not have influenced them. Throughout your journey, you can record the decisions and trade-off choices you have made and reflect on their implications. You can also record and reflect on limitations of your research and their impact on what you can say has been learned (or not learned) and how future research might address such limitations. You may also use the journal to reflect upon your own development as a researcher. Such thinking and self-reflection will be invaluable when writing up a specific research outcome, most importantly for postgraduate researchers, a thesis, dissertation or research portfolio.

3.5

Key Recommendations

Your research journal can record the entire story of your research journey, including all of the ups, downs, and complexities. • Remember that your journal is a personal record of your journey, so make it fit your needs. All we have done here is illustrate some possibilities. The better you can tailor it to meet your own needs and styles of thinking and recording, the more likely you will be to keep your journal up-to-date. Your journal is no place to censor yourself; be open and honest and get things out of your head and onto the page (or screen).

70

3 How Should I Record My Research Journey?

• If you are creative in the management of your journal, this may stimulate creative thinking later when it comes time to interpret your data and write up your results. For example, if you like using mindmaps, drawings, flowcharts or pictures, you may find that using them as a way of making entries in your journal may stimulate new ways of thinking about how you might display certain aspects of your research story to a reader. You may like to think in metaphors to characterise certain parts of your journey. For example, an entry might be “the interview I had today was like pulling teeth—I had to yank the information out of the person and it was clearly a painful process for them.” Doing this may provide you with creative ways of dealing with the emotional ups and downs of the research journey. Following on from the previous example, “I hated being the dentist in that scenario; it was not pleasant for me to sit there and do that to this person”. Metaphors may also help you to make new connections between ideas or generate solutions to baffling problems, e.g., “maybe I need to review and improve my technique or approach to commencing interviews and building rapport or maybe this is telling me that my questions are too sensitive”. Metaphors can be very powerful in this way. • Use your journal to record all your research decisions and processes and your reasoning behind them. Remember it can serve as a critical aid to memory when it comes time to write things up. • You can use your journal to record contextual details about your research, the context in which it is being conducted, annotations about participants, preliminary interpretations and so on. When it comes to writing chapters later, these details can help add colour and texture to your story as well as help you to make your story more convincing. • Your research journal can also serve as a data recording medium, especially useful if you are collecting qualitative data. • In addition to research-specifics, you can also record your personal thoughts and feelings along your research journey, reactions to criticisms, content and outcomes from discussions with your supervisor(s) and stakeholders. Such information aids the reflection process. • If you make your journal expandable (when it is handwritten) or if you manage it in electronic form (always backing it up, of course), remember that other documents can always be included as part of the journaling process.

References Ballenger, B. (2004). The curious researcher: A guide to writing research papers. New York: Pearson Longman. Carcary, M. (2009). The research audit trail: Enhancing the trustworthiness in qualitative inquiry. Electronic Journal of Business Research Methods, 7(1), 11–24. Charmaz, K. (2014). Constructing grounded theory (2nd ed.). London: Sage Publications.

References

71

Engin, M. (2011). Research diary: A tool for scaffolding. International Journal of Qualitative Methods, 10(3), 296–306. Flick, U. (2002). An introduction to qualitative research. London: Sage Publications. Glaze, J. (2002). PhD study and the use of a reflective diary: A dialogue with self. Reflective Practice: International and Multidisciplinary Perspectives, 3(2), 153–166. Lamb, D. (2013). Promoting the case for using a research journal to document and reflect on the research experience. Electronic Journal of Business Research Methods, 11(2), 84–92. Nadin, S., & Cassell, C. (2006). The use of a research diary as a tool for reflexive practice: Some reflections from management research. Qualitative Research in Accounting & Management, 3, 208–217. Ortlipp, M. (2008). Keeping and using reflective journals in the qualitative research process. The Qualitative Report, 13(4), 695–705. Richards, L. (2009). Handling qualitative data: A practical guide (2nd ed.). London: Sage Publications. Saunders, M., Lewis, P., & Thornhill, A. (2012). Research methods for business students (6th ed.). Harlow, England: Pearson. Stevens, K., & Asmar, C. (1999). Doing postgraduate research in Australia. Melbourne: Victoria Melbourne University Press. Walsh, E., & Cho, I. (2012). Using Evernote as an electronic lab notebook in a translational science laboratory. Journal of Laboratory Automation, 18(3), 229–234.

Chapter 4

How Should I Manage My Relationship with My Supervisor(s)?

Connecting postgraduate students with potential supervisors is not as straightforward a process as you might think. However, it is critical that you are paired with one or more supervisors that you can develop an effective working relationship with. While you may not always be able to control who your supervisors are, you can certainly do things to manage your relationship with them more effectively so that your chances of successfully completing your postgraduate research journey remain high. In some instances, you may need to deal with only one supervisor, but it is far more common for you to have to deal with two or more supervisors, depending upon the country and university you are studying at and the particular degree program you are or will be enrolled in. Whatever pathway you travel in connecting with your supervisors, it is important that you record, in your research journal, relevant details of your thinking, people you have considered/approached, feedback you have been given and what you have learned along the way. This can be a powerful reflective device to have at hand should problems, opportunities or the need for adaptation emerge during your journey.

4.1

Learn What the Supervisory Requirements Are for Your Postgraduate Program

Globally, there is a range of different supervisory models associated with master’s level and doctoral-level research and the differences seem to be largely associated with national educational cultures (see, for example, Kehm, 2006; Nerad & Heggelund, 2008). For example, research master’s degrees and PhD degrees in the US and Canada tend to require supervisory committees or panels (typically chaired by the primary or principal supervisor), comprised of three or more academics. In the United Kingdom, Australia and New Zealand, and Europe, supervision in such degrees largely resides with one or two academics (a primary or principal supervisor © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_4

73

74

4 How Should I Manage My Relationship with My Supervisor(s)?

and, in many cases, a co-supervisor). Where professional doctorates are concerned, the supervisory requirements may look somewhat different. Professional doctorates (e.g., an EdD, DBA, Doctor of Nursing) tend to involve research intended to have an impact or potential to impact on policies and practices within a specific community, profession or industry, which means there is a wider base of stakeholders to be considered (Kot & Hendel, 2012; Maxwell, 2003; McWilliam et al., 2002; Neumann, 2005). Some programs, such as the PhD.I (or PhD(Innovation)) at the University of New England in Australia, may focus on the contextualised development and evaluation of an innovation. Supervision for such degrees will therefore often involve not only one or more academic supervisors, but also at least one member of the relevant industry or profession. It is in your best interests to explore, early on, what the supervisory requirements are for your specific degree. Where more than one supervisor or a committee/panel is required, issues associated with the social and professional relationships between those supervisors become relevant to consider. You should also understand that, in some cases, the choice of supervisor may not reside in your hands but in the hands of the institution or the potential supervisor(s). But, no matter how you and your supervisors end up becoming connected for your research project, it is always within your purview to understand as much about each supervisor as you can. Such knowledge can be invaluable in helping you to more effectively manage your relationship with them.

4.2 4.2.1

Before You Meet—Finding/Attracting Supervisor(s) How Do I Go About Finding Supervisor(s)?—Seeking Commitment First

If you aspire to undertake a postgraduate research degree program, you may have at least some idea of what area you want to do your research in. This can help you narrow your search for an appropriate degree as well as a faculty, school or department in which to undertake that degree and once those things are pinned down, you can then look for potential supervisors among the academics that are on staff. As discussed in Chap. 1, nowadays this search can easily be done via the Internet, e.g., by scanning university, faculty, department and staff websites for background details and possibly staff CVs and by searching Google Scholar for the types of papers published by staff (some universities now provide online tools to assist with this search; for example, see http://www.une.edu.au/research/hdr/expertsupervisor for the University of New England). You may also have word-of-mouth recommendations about good potential supervisors from other postgraduates or staff members. Things to look for in a potential supervisor at this very preliminary stage include: • a publication track record in the area in which you want supervision or in related areas, including the pattern(s) of guiding assumptions under which they tend to

4.2 Before You Meet—Finding/Attracting Supervisor(s)

• • • • •

75

conduct their research (it would be a bonus to have read some of their papers, before you contact the person); openness to, or experience with, a multidisciplinary or pluralist focus in research —if it is of interest to you or possibly relevant to your intended research project; previous postgraduate supervisory experience; other supervisors they have worked with if a supervisory committee/panel is required; methodological expertise; and/or evidence of publishing with their postgraduate students (if that is of interest to you).

As we have previously suggested, when you identify a potential supervisor, it is a good idea to contact him or her directly prior to submitting your application for enrolment. In this contact, you could: • indicate your interest in him or her possibly becoming involved in your supervision and how you found out about them; • indicate that you have looked at their background and at some of their published work and have found it to be directly relevant to your interests; • provide some background on yourself, including your strengths, knowledge and preferences with respect to patterns of guiding assumptions and any previous research experience you may have; and • provide at least a rough sketch of the type of research topic and project you would like to undertake. If you make this contact by phone, it makes the contact more personal, but limits the depth of information you can provide and discover. If you make contact via email, the contact is more impersonal, but you can include attachments such as your CV and perhaps an example paper or two that you have written. The best strategy is to phone first then follow-up with an email contact to provide additional information. Remember that you are trying to stimulate a commitment to supervision so that, when you apply, your application can be directed straight to this person, making the whole process much more efficient. It will also show initiative, on your part, that you took the time and effort to research and contact them and that, in itself, may enhance their commitment to a supervisory role.

4.2.2

How Do I Go About Attracting Supervisor(s)?— Applying First

You may have applied to undertake a postgraduate research degree program; however, you may not have had any idea who might be available, willing or assigned to supervise you (this may occur, for example, if you have applied to several universities at once). This sequence of events means that the relationship

76

4 How Should I Manage My Relationship with My Supervisor(s)?

between you and your supervisor(s) may start even before you meet. Potential supervisor(s) as well as other academic personnel may vet your initial application to the degree program and the quality and character of your application may be what stimulates either commitment to supervise you (or not) or assignment to you. At this very early stage, then, the first impression you will make will be via your application. Your first impression on a potential supervisor or staff member charged with assigning supervisors to students can be enhanced by careful attention to content and detail in your application. Remember that your goal is to invite commitment on the part of potential supervisor(s) or to invite assignment of appropriate supervisor(s). • Clearly articulate the topic area that you wish to do your research in. While a full-blown proposal may not be required at this early stage, you should provide a relatively detailed summary of the area(s) you are interested in and the type of research investigation you might wish to undertake. Try to avoid being overly generic or ingratiating as this is generally unhelpful and often off-putting to supervisors (e.g., avoid statements like ‘I want to do research in accounting’, ‘I want to do a project on leadership or change’ or ‘I want to do research in your area of expertise’). • Clearly summarise your background and any research-related skills and achievements that you already possess and indicate those skills you may need to develop further, linking them to your proposed topic area. • Provide evidence that you can write in an appropriate academic style. This can be placed in an appendix to the application and can be provided in the form of a research paper you have written for other purposes or a previous publication or report you have authored. This can be especially helpful if you are an international student where English is not your first language. • Where possible, research, via the Internet, the possible supervisors available in the school or department you are applying to work in and specifically identify who you see as the most relevant possibilities in your application. A phone call to a potential supervisor indicating that you have applied and may be interested in them as a potential supervisor can smooth the way through the application process as well as gain you some clarification about any additional information they would like to see in your application.

4.3 4.3.1

Beginning the Relationship(s)—‘Backgrounding’ and Clarifying Expectations ‘Backgrounding’

An important early part of the supervisor-student relationship is learning about each other. You should learn about your supervisor(s) areas of interests and expertise.

4.3 Beginning the Relationship(s)—‘Backgrounding’ and; Clarifying Expectations

77

Try to get a feel for their areas of methodological and data analysis expertise, guiding assumption and discipline commitments and, where relevant, industry/ profession/community connections. Also, try to convey the same information to them about yourself and where you are coming from. Read some of the papers they have published to get a sense of how they write, what they research and how, and their preferred journal outlets. It may also be useful to find out how many postgraduate research projects they have previously supervised in what areas and read some of the theses/dissertations/portfolios they have supervised. This can help you get a feel for the extent of their previous supervisory experience (i.e., their track record as a supervisor) as well as for the nature of research outcomes their students produce. You might even talk to some of the students they have supervised in the past. You should also get to know your supervisor’s preferences and styles and gauge how your preferences and styles match up (Lewis & Habeshaw, 1997). For example, Vilkinas (2005) identified eight different roles that supervisors may play during the PhD and argued that matching roles to student needs is a critical issue. We can simplify things a bit here by describing three prominent supervisory styles. • Directive style: summed up as “I am the boss/expert/guru—you are the apprentice—you do what I say”—characteristic of many traditional academics. The directive supervisory style tends to be rational, technically-focused, formal and often prefers to maintain a reasonably large interpersonal distance between supervisor and postgraduate students. This style is aligned with the ‘technical rationality’ model of PhD supervision (Wisker, 2005) where supervisory emphasis is placed on efficiency, adherence to processes and techniques and a focus on achieving outcomes. Supervisors who reflect a strongly directive style will also tend to have a strong disciplinary allegiance, which makes it a very suitable style for programmatic research (such as in the natural and psychological sciences, for example). However, this style can potentially create problems in a multidisciplinary, pluralist or supervisory team environment and is ill-suited for postgraduate students who prefer to have a strong say in how their research journey unfolds. The balance of power between a directive supervisor and a postgraduate student strongly favours the supervisor. • Collaborative style: summed up as “we are colleagues and we’ll work together”. The collaborative supervisory style tends to be inclusive, informal, focused on discussion and team-based problem-solving and often prefers to maintain much closer interpersonal connection with postgraduates. This style is aligned with the ‘negotiated order’ model of PhD supervision (Wisker, 2005) where emphasis is placed on effectiveness, openness to change and postgraduate input at all stages of the research journey and a focus on undertaking the journey together. Supervisors who reflect a strongly collaborative style will also tend to have more fluid disciplinary allegiances which is well-suited to multidisciplinary, pluralist or supervisory team environments. This style can be more adaptable and flexible in times of uncertainty but would be less well-suited to postgraduate research students who need a high degree of guidance through their journey or

78

4 How Should I Manage My Relationship with My Supervisor(s)?

who want a high degree of control over their own journey. The power dynamic between a collaborative supervisor and a postgraduate student is approximately equalised, giving postgraduates a much stronger voice, relative to the directive supervisor. • Facilitative style: summed up as “you take the lead and I’ll help you”. The facilitative supervisory style tends to be inclusive to the extent that the student is expected to take the lead/initiative on most matters, acts as gatekeeper to resources, research contexts and expertise and serves as a touchstone for advice and assistance on research-related matters (including meeting institutional policy requirements). The balance of power between a facilitative supervisor and a postgraduate student strongly favours the postgraduate student and works best where the supervisor has high confidence in the abilities and independence of that student. This means that a facilitative supervisor would not be a good fit for a postgraduate student who needs strong guidance but would be an excellent fit for a postgraduate student with previous research experience or one who wants a high degree of control over their own journey. Note that with each supervisory style, there is an inherent power dynamic to be recognised. This is what makes finding a good fit so important; you want to be sure you can fit comfortably within that dynamic. Thus, early on, you want to see if potential supervisor’s approach to supervision will meet your needs as a candidate. Additionally, you will want to be on the lookout for signs and signals that reveal key aspects of the supervisor’s behaviour and expectations. It is possible to distinguish fairly clearly between characteristics of good versus poor supervisors. However, you should realise that supervisors exhibiting different styles will show different blends and degrees of these characteristics as well as differing capacities to be flexible in their style. Nowadays, supervisory style flexibility, i.e., ability to adaptively switch styles when circumstances demand, has become increasingly important. As universities open their doors to a wider range of domestic and international students, at various levels of preparedness and with different learning and social interaction styles (which may have important cultural roots that should not be ignored), so supervisors have had to become more adept at meeting a more diverse range of student needs. Some supervisors will be able to do this better than others. Equally, as Wisker (2005) has argued, increasing institutional emphasis on higher/faster postgraduate completion rates may be putting additional pressure on supervisors to change their style, usually toward the more directive end of the continuum. What are the characteristics of good and poor supervisors? Here are some characteristics of good supervisors that you should look for (see also Cryer, 2006, Chap. 6; Fraser & Mathews, 1999; Wisker, 2005, Chap. 2). • They have strengths, not only in relevant content and disciplinary domains, but also in appropriate methodological and data analysis areas.

4.3 Beginning the Relationship(s)—‘Backgrounding’ and; Clarifying Expectations

79

• They are open-minded with respect to other perspectives and other patterns of guiding assumptions including your own. • They are willing to and capable of working together rather than competitively with you. • Where appropriate, they are willing and able to work with other supervisors as a team, especially important in a multidisciplinary or pluralist context, without losing sight of your needs as the researching postgraduate student. • They are willing to communicate openly and frequently with you and sensitively balance criticism and praise. • They are willing to acknowledge their own weaknesses, blind spots and skill deficits and to openly suggest where you might go or resources you might tap into for help when needs in those areas arise. • They actively seek and listen to your views on constraints, issues and directions with respect to your research. • They treat you respectfully and professionally at all times. • When they criticise your work, it is focused on your arguments, approaches and actions rather than on you as a person. • They actively display the attitude that the research is your project, not theirs. • They demonstrate flexibility and willingness to change directions if something, even something based on their own advice, is not working out. Here are some characteristics of poor supervisors that you should watch out for. • They may have strengths in relevant content and disciplinary domains, but not in appropriate methodological and data analysis areas, or vice versa. • They tend to be closed-minded to other perspectives and non-preferred patterns of guiding assumptions including your own and may expect you to conform to their perspective if you want to remain with them. • They have a tendency to work competitively rather than together with you; often to the point of reinforcing the view that they are better than you are. • They are unwilling to communicate openly with you and much of their communication may involve criticism. • They are unwilling to acknowledge their own weaknesses, blind spots and skill deficits and reluctant to suggest where you might go for help when needs in those areas arise. They may even take the view that you are betraying the ‘relationship’ if you seek help from other people. • They avoid seeking your views on constraints, issues and directions with respect to your research. • When they criticise your work, it may be personal and even hurtful. • They do not respect you or treat you professionally, perhaps even engaging in objectionable or harassing behaviours. • They actively display the attitude that the research is their project, not yours. • They are demonstrably inflexible and reluctant to change directions if something, especially if based on their own advice, is not working out.

80

4 How Should I Manage My Relationship with My Supervisor(s)?

If you have continual unmet needs or conflicts in your current supervisory relationship, then switching supervisors or working to get another supervisor on board who can potentially meet those needs can be a useful strategy. However, you should realise that you may not always be in control of who your supervisor(s) are, which means that some incompatibilities may be inevitable and will have to be worked through or accommodated in some way. Supervisory teams/committees/panels Many universities now require more than one supervisor for a postgraduate research project. Normally, one supervisor is designated as the principal or primary supervisor and the others as associate or co-supervisors (some associate- or co-supervisors may potentially come from outside the university). This builds some redundancy into the supervisory relationship and provides continuity of supervision when the principal supervisor goes on sabbatical or leaves the university. A supervisory team, committee or panel with complementary strengths and voices among the members may often be beneficial but may also introduce some complexities and dynamics that might have to be managed. If you are undertaking a professional doctorate, it is likely that your supervisory team is required to include at least one member from the relevant industry/ profession/community in which you are embedding your research. This adds even more complexity to the mix as such members will likely not be academics, but ‘practitioners’, and will likely have views on what research means and what counts as research impact that may diverge quite markedly from traditional academic research perspectives (Carr, Lhussier, & Chandler, 2010, p. 279, described this as the “unique challenges of balancing academic requirements with praxis”). In such circumstances, some degree of supervisory team cohesiveness must be maintained if the team, as a unit, is to remain committed to your project. You can play your part in this process by learning how to dialog effectively and meaningfully with each member as well as with the team at large; the rest of the team, of course will need to play their parts equally effectively. Carr et al. (2010) argue that this can be achieved through thinking more systemically and inclusively about the issues surrounding your candidature. What are some characteristics of a good supervisory team that you should look for? • There are complementary skills sets between the supervisors (e.g., covering differing discipline areas or patterns of guiding assumptions that your research brings together, especially useful for triangulated, pluralist or multidisciplinary research; offering expertise, differing methodological approaches and approaches to data collection and analysis; bringing industry/professional/community experience/access and academic/theoretical rigour together). • The team demonstrates open-mindedness to each other’s perspectives and other patterns of guiding assumptions as well as to yours. • Members of the team are willing to and capable of working together rather than competitively, including being willing to selectively take a leadership role in

4.3 Beginning the Relationship(s)—‘Backgrounding’ and; Clarifying Expectations

• • • •

81

supervision at certain points during your research process when circumstances or a need for their specific expertise demand. The team is willing to communicate openly and frequently with each other as well as with you. The interpersonal dynamics among the team members do not dominate the relationship; you are in the equation as well. The team members demonstrate equal or nearly equal commitment to and involvement in your project. In a professional doctorate context, the team members effectively relate to each other in ways that do not automatically privilege their own world views.

What are some characteristics of a poor supervisory team that you should look watch out for? • The team exhibits unresolved status, power, values, guiding assumption or discipline differences between supervisors which create constant tensions and conflicting pressures on you as the candidate that may adversely affect your research processes as well as your progression through the degree. In these circumstances, you would not call such a group of supervisors a team. • The supervisors work as competitors rather than as a team, meaning that each is trying to ‘one-up’ the others in providing direction and advice about your research project; perhaps even denigrating the advice given by other supervisor(s). • The team members seldom engage in open or frequent communication with each other and/or with you. • The interpersonal dynamics among the team members dominate the relationship; you may not even be in the equation at all. • The team members demonstrate unequal commitment to and involvement in your project, to a point where it may become very difficult for you to get one or more of them involved. • In a professional doctorate context, the team members cannot effectively relate to each other; instead they interact in ways that automatically privilege their own world views while at the same time diminishing the value of alternative world views.

4.3.2

How Do I Go About Clarifying My Expectations?

The most essential thing to get in place as early as possible in your postgraduate journey is an understanding of what is expected of you (see, for example, Cryer, 2006, Chap. 4; Phelps, Fisher, & Ellis, 2007, pp. 45–49; Stevens & Asmar, 1999, Chap. 3; Wilks, 2006; Wisker, 2005, Chap. 4) and to gauge this against your own expectations about the journey. Mis-matches in expectations, if not addressed, can create downstream motivational impacts and productivity problems (Holbrook

82

4 How Should I Manage My Relationship with My Supervisor(s)?

et al., 2014; Wisker, 2005, Chap. 11). Thus, you should learn about and possibly even negotiate expectations with your supervisor(s) as soon as you commence candidature. In fact, supervisors may be required by their university to formally negotiate an agreed list of expectations with you (see, for example, a document from the University of New England in Armidale, New South Wales, which can be downloaded by clicking the Supervision Agreement Between Students and Supervisors link at http://www.une.edu.au/research/hdr/hdrformsandpolicies). You should also be aware that supervisory styles may strongly influence the expectations that a supervisor holds for you. Epstein, Boden, and Kenway (2005) discuss a number of these issues and expectations from the supervisor’s perspective, particularly the novice supervisor. They make it clear that there is no single correct pathway for supervision and that supervisors may have just as many doubts and uncertainties about the supervisory relationship as you do. Understanding this may help you to better see the supervisor-student relationship from their perspective and perhaps provide insights into why they are asking you to do particular things. Here are some important expectations for you and your supervisor(s) to reach agreement on (see also Cryer, 2006; Krone, 2006; Stevens & Asmar, 1999; Wisker, 2005). • Expectations about milestones. These expectations concern specific tasks and timelines for their completion during your candidature. Some of these tasks and deadlines may be prescribed by university policy whereas others may depend upon your supervisor’s preferences as well as on the evolving nature of your research project. For example, when does your supervisor want to see your proposal, your proposal presentation, ethics approval application, draft questionnaires, instruments or other data collection protocols, draft chapters? Some supervisors want to see a completed (or at least well-developed) literature review very early on in the candidature. You may also be required to make a presentation to a candidature confirmation panel of some description. • Expectations of each other (you and your supervisor(s)). These expectations are often influenced by the personal preferences of both you and your supervisor. Supervisory styles and your own pattern of learning needs and preferences will typically inform expectations such as how often you should meet or communicate with your supervisor, the nature of such communications, the degree of ‘structure’ desired or imposed on your candidature, the degree to which the supervisor-student relationship remains strictly formal and professional or evolves to have a more informal social dimension as well, and the degree to which each of you expects the other to be available on an as-needs basis, where a formal appointment is not made. • Expectations about the nature of your research. Supervisors will differ in their strength of commitment to and expertise within a particular pattern of guiding assumptions or approach to conducting research and this may strongly influence how they want you to shape your project. Some supervisors will be strong in specific content or discipline areas, others may be strong in methodological areas and some may be both. Equally, you will find (if you haven’t already) that

4.3 Beginning the Relationship(s)—‘Backgrounding’ and; Clarifying Expectations

83

you have a natural affinity or preference for research conducted within a specific pattern of guiding assumptions or approach. These expectations, if they do not agree, at least to some extent, with your supervisor(s), can be a source of conflict and dissatisfaction in the supervisor-student relationship. Conversely, if they agree too strongly, it can lead to a fairly narrow approach to research, even when a broader or more pluralist approach is called for. Basically, a sensitive and negotiated balance is required. Many supervisors are open to alternative world views and to the possibilities of triangulated, pluralist or multidisciplinary research, as are many students. A confluence of such perspectives in the supervisor-student relationship can lead to some very creative and innovative (but perhaps more complex) research investigations. • Expectations about writing and chapter turn-around. These expectations are usually a matter of personal preference and a clash of expectations can lead to some conflict and dissatisfaction if they are not openly discussed and clarified. For example, will your supervisor prefer to see drafts of partial chapters, whole chapters or perhaps just a draft of the entire thesis or dissertation? How quickly should you expect your supervisor to provide you with comments on your drafts? How quickly will your supervisor expect you to produce revisions based on those comments? Does your supervisor expect absolute conformity with their comments and proposed revisions or is there room for you to negotiate or argue for a different direction? While addressing these questions becomes extremely critical as you near the end stages of your candidature, it is important that you and your supervisor clarify these expectations at the start of your relationship. • Expectations about meetings. These are mechanistic but nonetheless important expectations to be clear about. One important question is what are the preferred location(s) and time(s) for face-to-face meetings? Some students, for example, may prefer to meet their supervisor on neutral ground rather than in their office. Some supervisors will only meet students in their office. Another important question is should all supervisor(s) attend every meeting, or can meetings be arranged with individual supervisors, depending upon the issue to be discussed? What is the preferred structure, if any, for meetings? Some supervisors expect students to start the conversation with an update report on where they are, whereas others may start the conversation by raising specific questions or ideas. What kind of preparations do you need to make, or materials do you need to bring, to specific meetings? Here, the answer depends on what is to be discussed at the meeting and on whether you need supporting materials (e.g., a published article you have just located or a copy of a questionnaire you have just drafted) to help you make specific arguments or seek specific advice. • Expectations about communication. Face-to-face meetings with your supervisor (s) are just one of many modes of possible communication. Some supervisors will prefer frequent face-to-face meetings; some will prefer very few, if any. Of course, you would have your own preferences to consider here. If you are undertaking your research degree through distance education, then email, phone or Skype/Facetime communications have become the norm (although many universities will typically require some face-to-face time with your supervisor

84

4 How Should I Manage My Relationship with My Supervisor(s)?

each year). Nowadays, chapter drafts, for example, can be submitted to your supervisor via email attachment or in a DropBox directory and they may return their comments electronically (via tracked changes in Word, for example) to you. If you have an off-campus co-supervisor, then Skyping or video-conferencing may be an option, especially for proposal presentations. It is important to be very clear on how frequently your supervisor expects communication of various types as well as how frequently you would like such communications to occur. • Expectations about the support you expect and can expect from them. Many universities now have policies that set out the minimum facilities and support that postgraduate students can expect through their candidature (for example, check out the Minimum Facilities for UNE Postgraduate Research Students link at http://www.une.edu.au/research/hdr/hdrformsandpolicies for the University of New England). However, on many important issues, such policies will be silent leaving you and your supervisor(s) to sort out relevant expectations. For example, can or will your supervisor provide contacts for gaining access to institutions/organisations for your research? When you submit written work to them, will they provide constructive criticism/comments on your writing style, expression, grammar and spelling or will they leave that for you to worry about? Can they facilitate access to funding for attending conferences/workshops? Will they work with you during the data analysis phases of your research or will you be expected to handle those phases on your own or at least with minimal input from them? Supervisors will differ in the extent to which they have the willingness and/or the capacity to address these types of questions and it is important that you discuss these issues with them when they arise instead of leaving them undiscussed until it is too late. • Expectations about publishing and ownership of ideas. Expectations about publishing from your research during your candidature as well as after you complete your degree are critical to pin down. Does your supervisor expect you to publish aspects of your research as you go? Do you expect to publish aspects of your research as you go? If the answer to either or both questions is yes, then how will authorship be worked out? Some supervisors (especially those who prefer a Directive style) will have a policy that they will always go as first author while you are a student. Other supervisors will always go as second or later author. Still others may not expect authorship unless you specifically invite them to join the effort as an author. An issue closely associated to that of publishing is understanding what rules will apply to any intellectual property, in such things as a new training program, change intervention or measurement scale, you may have developed during your research (some universities have policies governing this aspect of postgraduate research). Some professional doctorate programs, like the PhD.I at the University of New England, have as a central expectation, the development of and research on an innovation and intellectual property issues between the institution, the student’s workplace employer or profession and the student him- or herself must be ironed out early on. Supervisory styles, your own career aspirations as well as the degree of input into and influence on

4.3 Beginning the Relationship(s)—‘Backgrounding’ and; Clarifying Expectations

85

the research that both you and the supervisor(s) have had, will likely influence these expectations. There will be further discussion of these issues in the chapter on ethics and intellectual property.

4.4 4.4.1

Maintaining Supervisory Relationship(s) How Do I Keep the Ship Afloat and Relationship(s) Healthy?

A healthy relationship with your supervisor(s) is arguably one of the most important predictors of success of your postgraduate research. If the supervisory relationship falters or, worse, becomes toxic, then the focus on completing your research becomes secondary to just coping with the relationship, thereby creating a strong source of competing distractions and stress. It is thus in your own best interests to try to manage the relationship toward health in whatever ways you can. This is not to deny that the supervisor(s) play an equally essential role in this process, but our focus here is on your side of the relationship. In this light, there are several strategies you could adopt to help ensure that you develop and maintain a healthy relationship with your supervisor(s). • Style Awareness. One important strategy for maintaining a healthy relationship with your supervisor(s) is to acknowledge and work with expectations in light of different supervisory styles. As mentioned earlier, different supervisory styles imply very different power dynamics between you, as the student, and the supervisor. – With a supervisor exhibiting a directive style, the power in the relationship rests clearly in their hands. You are the apprentice and are not ready to stand on your own and make your own decisions; you must observe and learn from the supervisor. What he/she says generally goes and the learning/ communication direction is generally one-way—from supervisor to student and may extend from what topic will be investigated to the finer details of choice of pattern of guiding assumptions, how to gather and present the data and how to write up the results. Upward communication with a supervisor exhibiting this supervisory style will generally be like that from a subordinate to his/her superior in any other organisation: non-assertive rather than assertive; impersonal rather than personal and indirect rather than direct. This style of supervision often suits students who prefer to be led rather than lead, who may be unsure about their status or legitimacy with respect to research or who, because of their cultural background or prior experiences, prefer the supervisor take on this role. Debate about issues would generally be discouraged by a directive supervisor.

86

4 How Should I Manage My Relationship with My Supervisor(s)?

– With a supervisor exhibiting a collaborative style, the power in the relationship is generally shared between supervisor and student; it may not be exactly equal (in many cases, it may constitute a mentoring relationship), but the balance is much more toward you, as the student, than is the case for a directive supervisor. The learning direction here would be two-way: from supervisor to you and from you to supervisor. Communication in this relationship is more lateral in nature, as if between equals or colleagues: direct, personal and assertive where there are no rules about who should initiate contact and conversations. You would have as much to offer in this relationship as the supervisor. This style of supervision often suits students who are independent, have assertive communication skills, have a high achievement motivation, and who would thrive in a close collaborative relationship. Vigorous debate would be one style of conversation that is valued with a collaborative supervisor. – With a supervisor exhibiting a facilitative style, the power in the relationship generally favours you as the student; the supervisor takes a back seat and gives, even expects you to take, the lead. They are there to support and guide but not direct. The underlying assumption operating here is that the student is an independent thinker and capable of and willing to take the lead, as long as choices and decisions can be appropriately defended, rather than debated. Communication in this relationship is also more lateral in nature, as if between equals or colleagues: direct, personal and assertive, but often with the unspoken proviso that you are expected to initiate most of the contact and conversations. You would have as much if not more to offer in this relationship than the supervisor. The facilitative supervisor’s primary role is to be a constant resource for you. This supervisory style may strike some students, especially those who are looking for a stronger interpersonal connection, as rather laissez faire and aloof and may create a feeling of interpersonal distance in the relationship. – No one supervisory style is preferable or more likely to lead to a successful outcome—it truly is a matter of which style best suits your needs and capabilities. In a supervisory team, you may see a mix of styles, which may provide some positive synergies, but also add some complexities. For example, a supervisory team where supervisor A has a collaborative style and supervisor B has a facilitative style may work quite well if A is your principal or primary supervisor, working most closely with you throughout your project and B’s role in the team is to support and advise you in certain phases (e.g., expertise in specific content areas, methodology, data analysis). On the other hand, if B were your principal or primary supervisor and A was a co-supervisor, then an unusual dynamic might evolve where you form a closer working relationship with A, thereby largely usurping or negating B’s supervisory role, leading to potential friction. – It is also important to note that many of the better supervisors are able to ‘flex’ or modify their supervisory style to match emerging student needs more closely. Supervisors who do ‘flex’ or modify their styles would do so

4.4 Maintaining Supervisory Relationship(s)

87

on the basis of their perceptions of you and your work and progress to date. You can therefore facilitate effective style flexing in your supervisor by providing them with clear feedback about your needs and how well they are or are not being met by their current style. For example, we have observed, on several occasions, that a supervisor who naturally adopts a facilitative or collaborative style at the start of a supervisory relationship may flex their style to directive if student progress is slow, or the student seems very confused about how to proceed or is having difficulties coping with the degree of independence being assumed (perhaps because the student comes from a non-English speaking background, see Cryer & Okorocha, 1999). In some instances, those students have actually asked their supervisors to become more directive and to set deadlines and milestones to help them achieve a clear task focus and direction. This is a clear instance where a student’s awareness of his or her own needs can be translated into concrete actions and expectations to meet those needs. In the absence of this type of feedback from the student, the supervisor, especially if he/she is not perceiving the problem, may well persist in their original style to the ultimate detriment of the supervisory relationship (i.e., the student does not get the concrete expectations and tasks that they need, and the supervisor may become frustrated and disconnected). The lesson here: make your needs known so that your supervisor has a chance to alter their style of interaction with you in order to better meet those needs. – You should also be aware that in some universities, a supervisory novice (e.g., a recent PhD graduate on staff) may become part of your supervisory team as a co-supervisor to be mentored by the principal supervisor as a way of learning the ropes. Such a person could be a valuable resource for you since they have just recently completed the journey you are commencing, and their memories and experiences will be fresh. However, to avoid creating awkward power dynamics, you should not look to a novice supervisor for advice in preference to your principal supervisor if he/she has a directive or facilitative style. • Communication. Another extremely important strategy for maintaining a healthy relationship with your supervisor(s) is to keep in frequent communication with them. Obviously, the frequency and type of communication expected will differ for different supervisory styles, but it is your responsibility to keep your supervisor(s) apprised of your status and progress and advised of any difficulties you are encountering. Some students only communicate with their supervisor(s) infrequently (or not at all!); a situation that will likely not only frustrate supervisors but also place the student at much greater risk of getting off-track and missing key milestones, perhaps disastrously so, and not progressing as effectively as possible. • Inclusiveness. An important strategy, if you have a supervisory team, is to always keep all supervisors informed regarding your progress, decisions you have made along the way, problems you have encountered and solutions you

88

4 How Should I Manage My Relationship with My Supervisor(s)?

have devised. By doing this, you are ensuring that no supervisors remain in the dark about any aspect of your project. If you develop a closer relationship with one supervisor than to the others (a very common evolution), then you will need to take care that this doesn’t lead to communicating with and seeking advice from only that supervisor. • Timeliness. It is important to meet agreed deadlines or milestones (e.g., submission of your proposal, progress reports and chapter drafts) set by your supervisor(s) or, where appropriate, by university policies. This will help you to keep your momentum going and will give your supervisor(s) concrete timeframes within which they can expect to have to review and provide feedback on work you have done. If unforeseen circumstances do arise where you cannot meet a deadline, discuss them with your supervisor and renegotiate the deadlines and milestones. Timeliness extends to attending appointments and meetings as well. Chronic lateness to meetings is disrespectful and does not contribute to a favourable view of your capacity to complete your project on time. • Responsiveness. Most supervisors work hard to provide you with feedback on aspects of your project. This hard work can be easily repaid by acknowledging and being responsive to their feedback. Some feedback may be critical of what you have done, but you need to listen to it in the spirit with which it is offered— after all, they have the experience and, in many cases, the relevant knowledge. Of course, supervisors are not always right and there may be opportunities to discuss the feedback and negotiate ways forward, particularly with more collaborative or facilitative supervisors. Being assertive in these circumstances can provide satisfying results and may enhance a supervisor’s view of your capabilities. For some supervisors, you may need to be assertive in asking for their feedback, if they are busy or not closely connected with your project. Feedback is essential for your ultimate success, so don’t be shy in asking for it if it is not readily forthcoming. Once provided, timely responsiveness on your part will indirectly reward the supervisor for their efforts, perhaps making it more likely that they will offer feedback without your asking in the future. • Flexibility. No postgraduate research project is carried off, start to finish, without some problems, obstacles and constraints arising. For example, organisations you want to include in your sample may decline to participate in your study, the response rate to a questionnaire you have administered may be too small to permit the type of statistical analysis you had planned, or a study may have just been published that addresses exactly your research question. In such circumstances, it is important for both you and your supervisor(s) to maintain a learning attitude and be willing to change direction, tactics and approaches where needed (Howarth & Cornforth, 2005). Here is where a good supervisory relationship can pay great dividends in the process of solving emergent and unexpected problems.

4.4 Maintaining Supervisory Relationship(s)

4.4.2

89

Handling Emerging Problems

Not everything necessarily runs smoothly in supervisor-student relationships. Sometimes the ‘wheels come off’ or at least threaten to come off. Some kinds of problems (e.g., philosophical differences, different views on appropriate methods, feedback that is not constructive), even though they look serious, may be resolvable simply by discussing things openly with your supervisor(s). These types of problems may often result from a previous pattern of non-communication or from style mis-matches. So, one solution here is to improve communication, both in terms of openness and frequency. Another is to openly discuss the mis-match in styles that is creating differences in expectations; sometimes a supervisor may be unaware of the mis-match and may be able to adapt in a productive way once you have raised the problem with them. However, more serious problems do arise, not all of which may be research-related, and you need to have some strategies in place for handling them. Rest assured that there are concrete things that you can do when these more serious problems arise. Some of these strategies have been at least partially discussed in Boden, Kenway, and Epstein (2005, Chap. 5).

4.4.3

What Can/Should I Do if My Supervisor and I Can’t Agree About Issues Involved with My Research or My Supervisor Gives Me Incorrect Advice?

Disagreements between students and their supervisors can arise because of differences in strongly-held paradigm assumptions and beliefs about starting points and pathways for the research. Such differences may not surface at the start of the relationship but may emerge as your research project evolves and you begin to develop a different perspective from that of your supervisor on how the research should unfold. With some supervisors (especially those favouring a collaborative or facilitative style), these differences can often be resolved through open communication and debate. What you must ensure in these circumstances is that you are well-prepared to make your case, backed up with sound logic and empirical evidence. In other cases (especially where a supervisor adheres fairly strongly to a directive style), the disagreement may be intractable and not open to resolution by debate. Here, you would be well-advised to bring someone else into the conversation to help provide a buffering influence based on expertise (e.g., another academic colleague), advocacy (e.g., a representative from the postgraduate association if your university has one) or power associated with a formal role they occupy (e.g., a Postgraduate Research Coordinator or Head of School/Department). If you are part of a peer support network (a good strategy in any event), then that group might be able to suggest avenues as well. If this approach does not achieve the desired result, then you may have to resort to changing supervisors via a formal

90

4 How Should I Manage My Relationship with My Supervisor(s)?

university-approved policy/procedure; some differences may simply be irreconcilable. Remember that you always have the right to replace a supervisor in cases where the relationship has irrevocably broken down. Disagreements may also emerge around issues of intellectual property and publication of outcomes from your research project. This is more likely to occur if you and your supervisor have not clarified such expectations at the start of your relationship. Check your university’s policy on intellectual property and/or commercialisation (if it has such a policy) to clarify what your rights and obligations are and what your supervisor’s rights and obligations are. If this is the sort of disagreement you become involved in, then the answer may have a legal aspect, and this can be discussed with a legal officer (or equivalent) as a third party. As a student, you have the right to not have your legitimate intellectual property usurped by your supervisor. Equally, you must fairly acknowledge any contribution made by your supervisor to any intellectual property associated with your project. In any case, what could be involved here is not only claims of authorship for any published outcomes from your project, but also any commercialised materials developed in the context of your project. The best strategy for handling these types of disagreements is to seek expert advice immediately—don’t let things fester.

4.4.4

What Can/Should I Do if I Have a Supervisory Team Where the Supervisors Don’t Get Along or Have Contrary Views/Opinions/Paradigm Preferences?

This can and does happen, often in circumstances where you are undertaking a pluralist, multidisciplinary or interdisciplinary research project or with a supervisor whose research or previous training has strongly influenced their supervisory style. This type of conflict centres largely on academic differences of opinion, which means they are probably openly discussable. In these circumstances, your strategy may depend upon whether the conflict seems to originate primarily with your principal supervisor or one of your co-supervisors. The best strategy to initiate is to organise a meeting with the entire team and openly discuss the issues and perhaps the contradictory advice you are receiving as a consequence of their very different and strongly-held positions. This strategy has a better chance of success if you and your principal supervisor are in accord and the problem resides primarily with one of your co-supervisors. If the problem is affecting your relationship with your principal supervisor, then the discussion may have to evolve to a place where different supervisory arrangements are made. This particular conversation would be a delicate one to negotiate because of the potential power dynamics and emotional issues involved. However, if the conversation is kept at a professional level, focusing on logical academic and research-centred arguments rather than focusing on allocating blame or attributing hidden motives to people, a successful outcome has a chance of being achieved.

4.4 Maintaining Supervisory Relationship(s)

91

In all of this, the conversation needs to focus on your needs as the postgraduate student they are supervising. They need to be aware of the confusion that the conflicts are creating for you and that you want to find a way forward that is satisfactory to everyone. Sometimes a supervisor will simply be so entrenched in their position that their continued involvement in your project could be damaging. In this case, the best strategy is to follow your university’s procedures for replacing a supervisor and advise the person as to why you are pursuing this pathway. Again, remember that you always have the right to replace a supervisor in cases where the relationship has irrevocably broken down.

4.4.5

What Can/Should I Do if My Supervisor, in My View, Is Making Unrealistic or Unacceptable Requests or Demands?

You should openly discuss your concerns about their requests, being clear and assertive as to why you think the requests are unrealistic or unreasonable. Equally, you must listen to assertions and arguments made by your supervisor about why they have made their requests or demands. Once these views on both sides have been aired, it may become clear that the issue is simply one of misunderstood or mis-communicated expectations and the problem is then easily resolved from that point forward. What constitutes reasonable versus unreasonable demands is often a subjective judgment and this needs to be recognised. You also need to recognise that you may be making unreasonable requests or demands on your supervisor, from his or her perspective. In either case, leaving this matter unresolved will simply eat away at your relationship to the ultimate detriment of your research project. The best strategy for dealing with this issue, if open discussion of views and concerns as suggested above does not work, is to bring a third party (a very experienced and well-respected supervisor of other students in the same general discipline area would be a good choice) into the conversation to share their views. This can provide a way of more objectively gauging how reasonable or otherwise the demands are on both sides.

4.4.6

What Can/Should I Do if Interpersonal Conflicts Develop with a Supervisor?

The student-supervisor relationship is often a very close interpersonal relationship. It should always remain professional. While this is a responsibility of both parties, primarily responsibility for maintaining a professional relationship rests with your supervisor because of the inherent power differential that exists between student and supervisor. However, sometimes the professional attitude in

92

4 How Should I Manage My Relationship with My Supervisor(s)?

the relationship breaks down and one can see verbal, emotional and/or physical harassment or abuse occurring. Sometimes verbal and emotional abuse may be face-to-face; at other times it might be embedded in email messages or in the content of supervisory feedback on work you produce. However it occurs, it is wrong. Sometimes the relationship crosses acceptable boundaries, becoming too close, too intimate or too intimidating. In response, you may find yourself actively avoiding your supervisor or conforming to their demands, however inappropriate, out of fear or complaining to others (e.g., an advocate of your postgraduate association or a colleague in your peer network, but not to your supervisor) about the behaviours you are seeing and emotions you are feeling. This state of affairs is very serious and must be dealt with as soon as possible. However, make sure you have evidence (e.g., documentation such as your research journal, previous complaints by others) to back up your claims because you will be asked for it. Note that sometimes it is the student that crosses the boundaries of acceptable professional behaviour (e.g., becoming too attached or romantically involved with your supervisor or too aggressive in pursuing your point of view). As part of cultivating a professional attitude and work ethic, you need to be sufficiently self-aware of when you are becoming at risk of slipping into this abyss and take active steps to neutralise the problem. One strategy to handle this type of situation would be to discuss the issue with the Head of School or Department and let that person deal with the matter in their formal managerial role. Alternatively, you can follow your university’s grievance mediation procedures (such policies may actually prescribe an order of escalation that must be followed). Remember that you also have the right to replace a supervisor in cases where the boundaries of professionalism have been crossed in the relationship. In more extreme cases, replacement of the supervisor may be just one of several possible required actions.

4.5

Key Recommendations

Given that an effective supervisory relationship is critical to the successful completion of your postgraduate research journey, it is a relationship that is well worth working on and persevering with. To summarise, here are some key items of advice: • In some circumstances, you will be seeking and finding a supervisor, while in other circumstances where you are applying to a program, you may be assigned a supervisor. If you are in the first category of seeking commitment, the responsibility will be on you to make contact to engage commitment for their supervision. To achieve this, you will need to show initiative in making the call. If you find it difficult, then script out what you intend to say prior to making contact.

4.5 Key Recommendations

93

• When attempting to attract a supervisor through your application, the emphasis will be placed on your written material and demonstrating your ability to write in an appropriate academic style, so take good care with the documentation. • Ask the supervisor and other postgraduate students about the supervisor’s preferred style of engagement. Often there may be subtle changes in the style of engagement as you move from being an early stage PhD candidate to a more experienced researcher in the latter years of your study. • Record all relevant aspects of the processes for connecting you with your supervisor(s) in your research journal—what you did, who you talk to and about what, what you learned and anything else that might be pertinent. • It is now much more common to be assigned a supervisory team, as opposed to a single supervisor. Wherever possible, ensure that the team members have complementary skills and varying strengths to be of most effective support to you. • Right up front, when you establish the relationship with your supervisor, seek clarity on what your supervisor’s expectations are and on the myriad of areas that could impact on your relationship. That is, reach agreement on not just expectations about meetings, communications and chapter turnarounds, but also expectations regarding publications and ownership of ideas. • It is your responsibility to keep the communication channels open and information flowing so that your supervisor remains fully aware of your progress. Don’t hesitate to flag any difficulties you are experiencing. Your supervisor is there to assist you, but you do need to be fully and consistently engaged throughout your journey. • Don’t be too shy to ask for feedback if it is not readily forthcoming. Once again, it is about taking responsibility for your own learning outcomes and, if your supervisor needs a little prompting, that is what is required. Conversely, there is an expectation that you will be responsive to the feedback that has been provided. If the supervisor feels that their recommendations are being repeatedly ignored, they may quickly lose interest. • If you are having difficulties with your supervisor, ensure, initially, that you are working from an informed basis, that is, you have done your homework and can knowledgeably debate the issue at hand. If this still does not resolve the circumstances, you are advised to bring someone else into the discussion, preferably someone in a senior position in the school or department. This may also work if there are differences within your supervisory team, however, a meeting of all parties can often reconcile differing perspectives. • When dealing with your supervisor, and to avoid any interpersonal conflicts, it is important that both you and your supervisor maintain a professional demeanour through all interactions. As we have indicated, if you find the relationship is becoming too close, intimate or intimidating, then seek help as soon as possible from a counsellor or student advocate. The school or department will have suggested policies and procedures available to you. Follow those procedures and present your grievance with appropriate supporting documentation.

Fig. 4.1 Mindmap of important considerations with respect to managing supervisory relationships

94 4 How Should I Manage My Relationship with My Supervisor(s)?

4.5 Key Recommendations

95

• Remember that the key to a successful supervisory relationship is open, honest and frequent communication underpinned by mutual respect. While this is essential for every supervisory relationship, in a team context, it becomes even more imperative to ensure that all viewpoints can be shared and explored safely and respectfully, especially where member diversity crosses academicpractitioner boundaries. • Finally, use the mindmap, provided in Fig. 4.1, as a handy reminder of the issues and considerations raised in this chapter. The mindmap shows the interconnectedness of supervisory styles and expectations and obligations, highlights the fact that some expectations may be negotiable, some may not, and supervisory styles may influence the likely success of such negotiations. The mindmap also flags a range of potential tough problems that can emerge if supervisory style mis-matches or conflicting perceptions of expectations and obligations emerge during the research journey. Underneath each of the “Tougher things to deal with”, in italics, is a suggested strategy for coping.

References Boden, R., Kenway, J., & Epstein, D. (2005). Getting started on research. London: Sage Publications. Carr, S. M., Lhussier, M., & Chandler, C. (2010). The supervision of professional doctorates: Experiences of the processes and ways forward. Nurse Education Today, 30(4), 279–284. Cryer, P. (2006). The research student’s guide to success (3rd ed.). Berkshire: Open University Press. Cryer, P., & Okorocha, E. (1999). Avoiding potential pitfalls in the supervision of NESB students. In Y. Ryan & O. Zuber-Skerritt (Eds.), Supervising postgraduates from non-English speaking backgrounds (pp. 110–118). Buckingham: The Society of Research into Higher Education and Open University Press. Epstein, D., Boden, R., & Kenway, J. (2005). Teaching and supervision. London: Sage Publications. Fraser, R., & Mathews, A. (1999). An evaluation of the desirable characteristics of a supervisor. In K. Martin, N. Stanley & N. Davison (Eds.), Teaching in the Disciplines/Learning in Context, Proceedings of the 8th Annual Teaching Learning Forum, University of Western Australia, Perth (pp. 129–137). Holbrook, A., Shaw, K., Scevak, J., Bourke, S., Cantwell, R., & Budd, J. (2014). PhD candidate expectations: Exploring mismatch with experience. International Journal of Doctoral Studies, 9, 329–346. Howarth, J., & Cornforth, I. (2005). A fluid model for supervision. In P. Green (Ed.), Supervising postgraduate research: Contexts and processes, theories and practices (pp. 154–162). Melbourne: RMIT University Press. Kehm, B. M. (2006). Doctoral education in Europe and North America: A comparative analysis. Wenner Gren International Series, 83, 67–78. Kot, F. C., & Hendel, D. D. (2012). Emergence and growth of professional doctorates in the United States, United Kingdom, Canada and Australia: A comparative analysis. Studies in Higher Education, 37(3), 345–364.

96

4 How Should I Manage My Relationship with My Supervisor(s)?

Krone, M. (2006). Managing the relationship with your supervisor. In C. Denholm & T. Evans (Eds.), Keys to successful doctoral study in Australia & New Zealand (pp. 23–31). Camberwell: ACER Press. Lewis, V., & Habeshaw, S. (1997). 53 interesting ways to supervise student projects, dissertations and theses. Bristol: Technical and Educational Services Ltd. Maxwell, T. (2003). From first to second generation professional doctorate. Studies in Higher Education, 28(3), 279–291. McWilliam, E., Taylor, P. G., Thomson, P., Green, B., Maxwell, T., Wildy, H., et al. (2002). Research training in doctoral programs - What can be learned from professional doctorates?. Canberra, Australia: Commonwealth Department of Education, Science & Training (DEST). Nerad, M., & Heggelund, M. (Eds.). (2008). Towards a global PhD? Forces and forms in doctoral education worldwide. Seattle: University of Washington Press. Neumann, R. (2005). Doctoral differences: Professional doctorates and PhDs compared. Journal of Higher Education Policy and Management, 27(2), 173–188. Phelps, R., Fisher, K., & Ellis, A. (2007). Organizing and managing your research: A practical guide for postgraduates. London: Sage Publications. Stevens, K., & Asmar, C. (1999). Doing postgraduate research in Australia. Melbourne: Melbourne University Press. Vilkinas, P. (2005). The supervisor’s role as manager of the PhD journey. In P. Green (Ed.), Supervising postgraduate research: Contexts and processes, theories and practices (pp. 163– 177). Melbourne: RMIT University Press. Wilks, S. (2006). The process of supervisor selection. In C. Denholm & T. Evans (Eds.), Keys to successful doctoral study in Australia & New Zealand (pp. 15–22). Camberwell: ACER Press. Wisker, G. (2005). The good supervisor: Supervising postgraduate and undergraduate research for doctoral theses and dissertations. Basingstoke: Palgrave Macmillan.

Chapter 5

How Should I Manage the Research Project?

5.1

Planning the Research

It has been pointed out that while postgraduate research students typically enter a program highly motivated and looking forward to their research, they often have little understanding of the demands, finding themselves in a program that bears little resemblance to previous degree programs they have successfully completed (Grover, 2007). Many candidates do not understand that during their course of postgraduate research studies, there are two parallel tasks: conducting their research and writing an examinable research outcome while the other is to plan and efficiently manage a research project (Katz, 2009). There have been many descriptions given to the postgraduate research process. Postgraduate research, particularly at the doctoral level, has aptly been described as being maze-like, with many twists and turns. Another way of thinking about your research is to equate it with entering a large, multi-floored mansion with the ultimate destination being the roof. With stamina you can always get to the roof but there are many alternative passages and stairwells you can take. There are also many beautiful rooms (interesting reading) along the way that you may wish to spend time luxuriating in. However, the longer you spend in those rooms, the longer it will take to get to the roof. Ultimately, you may go down some corridors which will take you in the wrong direction or you may have to re-trace your steps. It is a big project and getting lost occasionally is to be expected. So, whether it be a maze or a large house, if, at the start when you entered, someone gave you a map and time frames for each step of the way you would be able to measure your progress and with the exercise being completed a lot quicker and with a lot less stress.

© Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_5

97

98

5.1.1

5 How Should I Manage the Research Project?

How Can I Manage My Study to Be as Efficient as Possible?

A research project, such as undertaking a doctorate, has been commonly identified as a temporary undertaking, with a defined beginning and end, designed to produce a unique result despite being constrained by limited time and funding (Katz, 2016). The nature of these projects therefore requires project management skills for meeting prescribed constraints of quality, time, scope, and budget and other resources. However, students do not often recognise the need for such critical skills. Managing your research project is essentially all about planning and mapping your research journey and it will require time and energy to do it effectively. Effective planning will give you a scaffold from which to work as well as a way to monitor your progress (Gosling & Noordam, 2010). Planning is one of the elements of project management. The downside of not planning is pretty obvious. A research project is a major undertaking and, if you don’t plan systematically, you will end up trying to do everything at the last minute, and it will show. Thus, you need to understand how your work will be structured in relation to the kind of project you expect to do, then divide up the time and other resources you have available and specifically plan out how your work will fit into the timetable (Thomas, 2017). We strongly suggest, therefore, that you take a project planning approach to managing your research journey as a means of reducing uncertainty and risk. Despite variations in research topic, questions, guiding assumptions, methodology and the inevitable hiccups, your postgraduate research will progress along a well-trodden path. Knowledge of this path and an appropriate plan of action in anticipation of each stage are essential to avoid significant downtime and overshooting your expected completion date. Even though you will not be able to anticipate all the potential things that could happen during your journey, you can still plan most aspects of it. Planning can provide you with a sense of control over events and while you shouldn’t buy too deeply into that sense of control, planning can help you to sort out the nature and sequencing of actions to implement, how and when to best deploy your resources and identify key milestones you want to achieve during the journey as well as key deadlines you may need to meet.

5.2 5.2.1

Research Planning Tools What Are the Benefits of Planning Your Journey as a Project?

Postgraduate study operates within a relatively unstructured environment where most deadlines are self-imposed. In fact, the research task has been equated with students confronting “a featureless plain” (Hockey, 1994, p. 181) and it is up to the student to define the boundaries and meaningful way-points. Apparently only a

5.2 Research Planning Tools

99

small percentage of postgraduate students manage to complete their examinable research outcome on time (Backlund, 2017; Bruin & Hertz, 2010). Consequently, valuable time spent early in the research, mapping out what is required and by when, will help you manage the processes. When working your project into manageable phases, with a timeline for each phase, make sure to also set scheduled time off for yourself, because you can’t work all the time and it isn’t healthy to do so. The benefits of planning out your research journey, which will span not months but years, are that it will: • • • • • •

keep you focused on what is important; ensure that you undertake activities in a defensible and logical order; avoid unnecessary delays when you are waiting for something to happen; help to ensure that you maintain momentum during the course of your journey; keep you within regulation time-lines for submission; and save you time.

The time-frames associated with undertaking postgraduate studies can be extremely deceptive. At the beginning, it feels as if you have an immense amount of time to work your way at a leisurely pace through the relevant literature and establish your research questions before embarking on data gathering. In reality, time runs out very quickly. Students who have successfully completed will have structured their journey as far as possible, anticipated and accomplished key tasks and milestones, remained cognisant of when self-imposed deadlines were due and kept to those deadlines. Project management can be a rather sophisticated process, but it need not be for the purposes of planning your research journey. The concepts of project management are fairly generic and transferrable. Therefore, they can be applied with only slight variation to any particularly academic discipline (Katz, 2009). To help you with the mechanics of the planning process, there are a number of specific project planning and management tools that can be applied to the research process. The following sites provide some useful tools: 1. http://www.nextscientist.com/manage-a-large-research-project/; 2. https://www.enago.com/academy/dont-let-science-fail-research-projectmanagement/; 3. https://www.vitae.ac.uk/doing-research/leadership-development-for-principalinvestigators-pis/leading-a-research-project/managing-a-research-project/ project-management-tools-for-researchers; 4. http://www.blogs.hss.ed.ac.uk/pubs-and-publications/2016/12/16/every-appneed-phd/; 5. http://www.ippt.pan.pl/attachments/article/234/Successfully_navigating_ doctoral_study.pdf. Regrettably there are only a small number of textbooks devoted specifically to project managing one’s postgraduate research (e.g., Bruin & Hertz, 2010; Finn,

100

5 How Should I Manage the Research Project?

2005; Katz, 2009). One useful site for sourcing a wide range of more generic project planning and management techniques is Mind Tools, (http://www. mindtools.com/), which covers such areas as estimating time accurately, scheduling simple projects, Gantt charts, critical path analysis (for projects with sequentially dependent stages), PERT (Project Evaluation and Review Technique), and the planning cycle, which is a planning process for middle-sized projects. You can map your journey by hand or using software support such as Microsoft Excel spreadsheets or Google Sheets spreadsheets, Microsoft Project software or an online software tool such as Smartsheet (https://www.smartsheet.com/product-tour). Trawl through these various methodologies to find one that is most appropriate to you. Alternatively, look at a variety of planning templates in Word or even consider developing your own flow diagram. As a concrete example, one of the more popular ways of chronologically mapping your research journey as a project is a Gantt-type chart. Some key components of the sort of Gantt-type chart we will illustrate here (which we refer to as the ‘Research Journey Timeline’ or ‘RJT’ chart) include: • Spreadsheet-type grid—a row-and-column structure for organising the chart; • Timeline—a time line broken down into days, weeks or months onto which tasks/activities, milestones and deadlines can be mapped. • Tasks/Activities to carry out—you can be as general or specific as you need to be, but you define all tasks to be accomplished. • Milestones to achieve—these are specific outcomes, which you will produce, to be achieved at specific times. • Deadlines to meet—dates by which a specific activity or set of activities needs to be completed. • Resources needed to accomplish a task/activity, produce a milestone outcome or meet a deadline. Figure 5.1 illustrates a fairly basic RJT chart for a postgraduate research project employing qualitative interviews, constructed using Microsoft Excel. The research project is initially planned to be completed within 24-month period. Various tasks/ activities associated with the postgraduate project are listed in a roughly linear sequence down the rows and resources needed to complete each task/activity are listed in the adjacent column. Simple arrows approximate the period of time the task/activity is projected to occupy. Note this particular chart is more impressionistic than precise in that specific dates are not shown, although they could certainly be overlaid. Symbols are used to signal key milestones in the journey (defined below the chart) and a vertical red line signals a deadline by which a specific milestone should be achieved (which, for a postgraduate student, would typically be institutionally imposed, such as confirmation or ethics approval request submission date). When constructing an RJT chart, you need to understand and map any institutional or other external requirements and expectations along the journey. If you are doing a professional doctorate, you may also need to factor in course work

Data sources for trialling (not for main study) Professional transcriber; funds to pay transcriber Data sources; Samsung tablet; digital recorder; backup USB storage Professional transcriber; funds to pay transcriber Data sources; Samsung tablet; digital recorder; backup USB storage Professional transcriber; funds to pay transcriber MAXQDA 12 software Supervisor Deadline Draft chapter

Gatekeepers controlling access to data sources & interview space I need

Supervisor; quiet space Microsoft PowerPoint; PowerPoint facilities Supervisor; University Human Ethics Committee; online ethics approval application form

Supervisor; key studies

Resources Needed Loose leaf notebook w/ ~100 pages Supervisor Google Scholar; university library Supervisor; key studies; Inspiration software for mindmapping

1

2

Fig. 5.1 llustrative RJT chart for a postgraduate research project

Analyse interview & document data Write, revise & submit PhD thesis Milestone to achieve Draft proposal

Revise transcripts as needed

Conduct participant feedback interviews on transcripts

Transcribe interviews & check for accuracy

Prepare interview transcription guide & consult w/ transcriber Conduct interviews & collect desired documents

Conduct trials of data gathering approaches

Identify/negotiate access w/ key gatekepers

Submit ethics approval application

Configure & scope the research; identify desired data sources (w/ Plan B alternative) Write research proposal Present research proposal to department staff & students

Formulate research goals & questions

Task/Activity Commence & maintain research journal Discussions with supervisor Critically review key literature

3

4

5

6

Meth

7

8

Lit

9

10

11

Month 12 13

14

15

16

17

18

19

20

21

22

23

24

5.2 Research Planning Tools 101

102

5 How Should I Manage the Research Project?

activities as well as schedule significant time blocks and perhaps concrete milestones for stakeholder engagement activities. You can also set personal milestones to achieve, which can play an important motivational role. For a postgraduate student, this can include regular meetings with supervisor(s) (shown in Fig. 5.1). Note that the mapping of resources needed to accomplish tasks/activities include the necessary technology and software. In constructing your chart, don’t forget to think about analysis requirements and competencies early on, so you can make clear data management and software support choices and can rectify any skill deficits you may have (e.g., by planning to attend a key workshop or meeting with an expert or a mentor). You can make the RJT chart as general or precise as you wish as long as it works for you; it should form an important and prominent entry in your research journal. However, it is critical to understand that the RJT chart does not set things in stone; you will usually find that it needs to be significantly updated, altered or otherwise adapted to changing and emerging circumstances. For whatever tool you use to work, however, you will need to have a good understanding of the key components of the project that needs to be managed, that is, to drill down to get the essential steps within the process. Some writers refer to a “critical path” that is common in most postgraduate research despite the variations in topics. The closer you stick to the critical path, the less likely you are to travel down unnecessary side roads, and the more likely you are to complete your research on time. The usual planning structure goes something like this: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

consider the broad topic area; review the literature; develop research questions/hypotheses; contextualise and configure your research activities; develop, present and confirm your research proposal; sample data sources; gather the data; manage the data; analyse the data; write up examinable research outcome; and submit research outcome.

While this type of structure is simple and clearly laid out, usually in diagrammatical form, it is, however, a little too broad and doesn’t really give credence to the complexity of activities within postgraduate research. It also does not clearly acknowledge the role that research frames, guiding assumptions, research configurations (to be discussed further in Chaps. 9–12) play, which in many cases, may create iterative loops between structural elements 6 through 9 in the above list (encompassed in the ‘Data Triangle’ in Fig. P.1). In addition, while there is a need to prepare in advance for future steps, there is often also an iterative process within a step, which may require you to cycle back a few times on a step before moving on. It may take, for example, many consultations with your supervisor(s) before the research questions are finalised and, undoubtedly, there will be many versions of

5.2 Research Planning Tools

103

your conceptual framework before it is nailed down. A further point is that some steps occur not just once but several times through the research process. As an example, reviewing the literature doesn’t just occur at the conceptual stages of the research but also at frequent intervals right through the project. In fact, you may still be inserting references just before the final thesis document is printed. If you plan to implement a grounded theory investigation using a sequential/saturation sampling scheme (see Chap. 19, Sect. 19.3.9) or an evolutionary research configuration (see Chap. 12, Sect. 12.2.8), your data gathering timeline will, of necessity, need to remain somewhat open-ended because you cannot anticipate precisely all of what you will need to do up front; in these cases, you might need to do some educated guessing as to what time frame to allow. To accommodate these complexities, you will need a planning structure that is robust enough to provide sufficient detail to be useful and flexible enough to allow for various iterations within a step, timeline extensions and, where necessary, the revisiting of some steps throughout the project. As the research process has a finite time limit, it also becomes imperative that your project is planned in sufficient detail to allow you to allocate deadlines, but with some degree of flexibility should certain stages of the project, for example, data gathering, go over time. Ideally, planning of your postgraduate study should start before enrolment and include any specific requirements that have been built into your program such as course work, a research proposal, confirmation of candidature, milestone reporting and even possible publishing requirements. Araujo (1995, p. 191) has observed that postgraduate research is not often considered as a genuine time experience located in a linear structure. We have attempted to address this. In Table 5.1 The Doctoral Research Planning Guide, we have gone further than the general, simplified description of the steps in the research process and have provided a more specific and detailed plan of what could be perceived, in a generalised form, as some of the key tasks in the postgraduate research process. Please recognise that the planning guide is provided only as a generic framework as each postgraduate research project needs to be individually planned and the framework suitably tailored. This tailoring is best done with your supervisor(s) who, given their prior supervisory experience can guide you on what needs to be achieved and how long it might take, you will note that we have not tried to pull out differing paradigm or methodological approaches but have attempted to provide you with elements that can be drawn upon for the construction of your own project plan. The intention is to indicate a more direct route for your research and to break this huge task into manageable chunks. As one PhD student commented “I really agree with breaking things down. I think I made many references to pineapples when I was writing my thesis – you can’t eat a pineapple just as it comes. Your mouth isn’t that big, well, yours might be but mine isn’t! So, you have to cut into it and make the pieces bite-size for optimal chewing … so you cut it into chunks, just like the pieces of a pineapple – it really is too big to physically comprehend as one thing but as a series of small, easily manageable tasks it is quite possible to cope with.”

Writing & presentations

Data gathering & analysis Step varies with different research approaches Note: The postgraduate journey reflected here is a basic one, subject to negotiation with your supervisor(s)

Identify support networks for your study from family and peers

Develop a tentative topic and tentative broad research question(s)

Identify key-words relevant to your topic for use in literature searches

Undertake a broad bibliographic search on topic areas and read broadly in your topic(s)

Investigate institutional and program requirements

1.4

1.5

1.6

1.7

1.8

(continued)

Reading & literature development

LEGEND

Commence research journal writing to record your thoughts, plans and choices

STAGE SEVEN: Final 6 months

1.3

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

Reflection & academic development

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

Identify your personal extrinsic and intrinsic motivations

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

1.2

STAGE THREE: Year 2 PT; Sem 2 FT

Personal & administrative

STAGE TWO: Year 1 PT; Sem 1 FT

Set the goal(s) for your postgraduate research project

STAGE ONE: 6 months prior to Enrolment

1.1

Table 5.1 The doctoral research planning guide

104 5 How Should I Manage the Research Project?

Secure resources/grants and submit applications

Investigate potential supervisors

Establish your work space and work parameters

Read books on undertaking postgraduate study

Set up Endnote or other bibliographic recording system/become familiar with the expected referencing system for research outcomes at your institution

Set up hard copy filling system and/or electronic database

Fine-tune your initial thoughts on research topic/questions/hypotheses

Finalise enrolment

1.9

1.10

1.11

1.12

1.13

1.14

1.15

1.16

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 105

Identify institutional assistance structures and postgraduate research resources

Establish new priorities for your time and in relation to your study

Take a library tour

Confirm supervisory relationships

Become fully acquainted with the program regulations

Continue writing in your research journal

Establish meeting times and negotiate expectations with supervisor(s) and other key stakeholders and gatekeepers

Develop a detailed project plan, with critical milestones indicated

1.17

1.18

1.19

2.1

2.2

2.3

2.4

2.5

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT STAGE SEVEN: Final 6 months

(continued)

106 5 How Should I Manage the Research Project?

Amend plan and timetable following feedback from supervisor(s)

Reflect on time management and work/life balance; establish strategies

Schedule and undertake relevant course work/seminars/workshops

Establish relationships with other postgraduate students

Review recently completed theses/ dissertations/portfolios in your area

Undertake a more focused review of literature

Undertake extensive note-taking, recording in your research journal

Identify relevant theories and perspectives

2.6

2.7

2.8

2.9

2.10

2.11

2.12

2.13

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 107

Meet supervisor(s) on a regular basis (weekly, fortnightly, monthly; meetings could be faceto-face, via email or virtual)

Firm up research topic and title

Write up research project rationale

Consider the most appropriate patterns of guiding assumptions to adopt for your research

Continue to connect with relevant academic and non-academic stakeholders, especially important if doing a professional doctorate

Consider and develop the contextualisation and positioning of your research and settle upon your research frame

Refine your research question(s)/hypotheses in line with your guiding assumptions, choice of research frame and contextualisation/ positioning strategies

Write up a list of potentially relevant concepts and constructs, where appropriate

2.14

2.15

2.16

2.17

2.18

2.19

2.20

2.21

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

108 5 How Should I Manage the Research Project?

Write first draft of conceptual framework, where appropriate

Consider who/what your potential data sources might be

Decide upon your research configuration and reflect on alternative methodologies and data gathering strategies that you could use

Consider possible analytical methods and evaluate your capabilities – seek training, if required

Develop, write and submit your research proposal to your supervisor(s)

Commence the research ethics approval process appropriate to your institution

Prepare and present a seminar on your research proposal (may be part of a formal institutional confirmation process)

Revise proposal using feedback from your supervisor(s) and others (submission of revision to administration may be required)

2.22

2.23

2.24

2.25

2.26

2.27

2.28

2.29

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 109

Develop a tentative Table of Contents for your thesis/dissertation/portfolio

Set up large document format/ develop skills in large document formatting and management

Develop draft of Introduction chapter containing rationale, contextualisation and positioning of the study

Submit draft Introduction chapter to supervisor(s)

Revise Introduction chapter following feedback from supervisor(s)

Revise Introduction chapter into a potential journal paper on current key issues in the topic area (optional/negotiable with supervisors)

Continue review of literature, theses and papers in other but potentially relevant disciplines; don’t neglect relevant grey literature

Focus on theoretical and practice-oriented foundations relevant to your research

2.30

2.31

2.32

2.33

2.34

2.35

3.1

3.2

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

110 5 How Should I Manage the Research Project?

Draft Literature Review chapter identifying key gaps in the literature

Submit Literature Review chapter to supervisor(s)

Revise the Literature Review chapter following feedback from supervisor(s)

Recraft the material on the relevant literature, theory and key gaps into a potential journal paper (optional/negotiable with supervisors)

If appropriate, further refine research questions following links to theory and/or practice

Reflect on potential theoretical and practical implications as well as the strategic implications of your research

Prepare your ‘elevator pitch’

List information needs and identify potential sources

3.3

3.4

3.5

3.6

3.7

3.8

3.9

3.10

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 111

Consider data source issues and methods of approaching data sources including how you might need to access them via relevant gatekeepers

Make initial contacts with potential data sources

Refine the conceptual framework/conceptual map

Write up the conceptual framework/ conceptual map and research questions in summary form

Submit conceptual framework and research questions material to supervisor(s)

Revise conceptual framework and research questions following feedback

Conduct research on alternative methodologies that could be used and summarise this information

Consult with supervisor(s) and others regarding appropriate methodologies for your study

3.11

3.12

3.13

3.14

3.15

3.16

3.17

3.18

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

112 5 How Should I Manage the Research Project?

Undertake, where necessary, additional training on data analysis and management, including training for using relevant software support systems

Decide and firm up the research strategy and methodology (you may need to complete this step before finalising your ethics approval application in Step 2.27)

Consider the analytical tools related to your chosen methodologies

If required, prepare an interim progress report for your university/institution

Write up draft chapter sections on conceptual frameworks and specific research questions or hypotheses

Revise and update plan and timelines

Investigate possible conference attendance/ check submission dates/ investigate funding

Seek supervisory input on sampling approach(es)

3.19

3.20

3.21

3.22

3.23

4.1

4.2

4.3

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 113

Develop a sampling plan (you may also need to complete this step before finalising your ethics approval application in Step 2.27)

Confirm sampling methodology

Operationalise concepts and constructs, where appropriate

Develop data measurement and gathering techniques, where relevant

Consider paradigm appropriate research quality criteria and meta criteria issues as you plan

Further consideration of analytical tools/seek additional training, if required

Initiate, approach and negotiate access to data sources (may need to work closely with your supervisor(s) here)

Secure commitment from all data sources

4.4

4.5

4.6

4.7

4.8

4.9

4.10

4.11

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

114 5 How Should I Manage the Research Project?

Undertake pilot testing or trialling of your data gathering strategies

Undertake pilot/trialling data analysis and reflection

Consider reliability and validity or other data quality issues with data generated

Consider possible limitations of the research which could be addressed at this point

Start writing Methodology chapter

Undertake full data gathering

Data entry/transcript development

Data cleaning, both qualitative and quantitative

4.12

4.13

4.14

4.15

4.16

4.17

4.18

4.19

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment STAGE TWO: Year 1 PT; Sem 1 FT STAGE THREE: Year 2 PT; Sem 2 FT STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 115

Data checking

Preliminary data analysis/initial superficial analysis and tentative observations

Seek supervisory feedback on analyses as you undertake them

Reflect on initial data findings

Further data analysis

Interpretation of data findings

Supervisory discussion

Reflection and discussion of data findings with others

4.20

4.21

4.22

4.23

4.24

4.25

4.26

4.27

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

116 5 How Should I Manage the Research Project?

Update Methodology chapter

Prepare possible conference paper based on preliminary findings (optional/negotiable with supervisor(s))

Prepare possible paper on research design, alternatives, and issues relevant to your study (optional/negotiable with supervisor(s))

Undertake possible further data gathering for clarification or correction

Undertake possible further data analysis

Write up fieldwork data gathering, where relevant

Reflection/interpretation/theory building

Analyse data against your questions/return to your questions to see how they compare

4.28

4.29

4.30

5.1

5.2

5.3

5.4

5.5

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 117

Reflect on whether there are any new questions that your data can answer

Juxtapose your findings with existing literature and practices

Refer back to your conceptual framework while reflecting on your results

Discuss results with supervisor(s) and others

Undertake further literature investigation based on results

Reflect and refine analyses

Write up Results and Discussion or equivalent chapter(s) and submit to supervisor(s)

Complete the Methodology chapter and submit to your supervisor(s)

5.6

5.7

5.8

5.9

5.10

5.11

5.12

5.13

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT STAGE SEVEN: Final 6 months

(continued)

118 5 How Should I Manage the Research Project?

Review and finalise Methodology chapter following feedback from supervisor(s)

Present conference paper, where relevant

Write up brief summary of findings to send to participants. Thank them for their involvement

Formulate tentative theoretical or strategic implications

Submit interim progress report to your university, where required

Prepare a possible paper on one dimension of the study (optional/negotiable with supervisor(s))

Revise Results and Discussion chapter(s) following feedback from supervisor(s)

Revise and update your research project plan and timelines

5.14

5.15

5.16

5.17

5.18

5.19

5.20

6.1

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 119

Check regulations regarding submission

Develop a tentative theory and/or practice change schema based on your findings, where relevant to do so

Consider the implications of your research for theory, methodology and practice

Undertake any further work required for developing theoretical, strategic or practical implications

Draft chapter sections discussing the conclusions and theoretical, strategic or practical implications of your research

Submit chapter sections discussing the conclusions and theoretical, strategic and practical implications to your supervisor(s)

Reflect on the original contribution dimensions and frame contextualisation/positioning strategies of the research in conjunction with with relevant research quality criteria and meta-criteria

Reflect on critical limitations of the study and include these in the Methodology or Conclusions and Implications chapter, as appropriate

6.2

6.3

6.4

6.5

6.6

6.7

6.8

6.9

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment STAGE TWO: Year 1 PT; Sem 1 FT STAGE THREE: Year 2 PT; Sem 2 FT STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

120 5 How Should I Manage the Research Project?

Revise conclusions, theoretical, strategic and practical implications following feedback from supervisor(s)

Consider what possible future research could be undertaken

Finalise the Conclusions and Implication chapter

Submit Conclusion and Implications chapter to your supervisor(s)

Update the Literature Review chapter

Revise Conclusions and Implications chapter following feedback from supervisor

Review Introduction chapter to ensure continuity of the research

Polish up the conceptual framework

6.10

6.11

6.12

6.13

6.14

6.15

6.16

6.17

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 121

Consider condensing tables and graphs and how to provide more accurate displays of what you have learned

Review the documentation again regarding submission timelines and processes

Write the Abstract

Write Acknowledgements

Undertake document checking and polishing

Undertake proof-reading and layout checks

Undertake bibliographic checking, matching and tidy up

Complete the first full draft of the entire research outcome (thesis/dissertation/portfolio)

6.18

6.19

6.20

6.21

6.22

6.23

6.24

6.25

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

122 5 How Should I Manage the Research Project?

Submit the first full draft to supervisor(s)

Preparation/finalisation of paper(s) relevant to the findings of your research for submission to relevant journal(s) (optional/negotiable with supervisor(s))

Obtain feedback from supervisor(s) on the full draft

Undertake revision of material following feedback from supervisor(s)

Update relevant literature

Undertake holistic review/checking of the entire research outcome (thesis/dissertation/portfolio)

Review of the final draft by a professional proof-reader and make necessary changes

Submit final research outcome for examination, with the approval of your supervisor(s)

6.26

6.27

7.1

7.2

7.3

7.4

7.5

7.6

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment

STAGE TWO: Year 1 PT; Sem 1 FT

STAGE THREE: Year 2 PT; Sem 2 FT

STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT

STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT

STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

STAGE SEVEN: Final 6 months

(continued)

5.2 Research Planning Tools 123

If required, prepare for a Viva or oral defence

Prepare possible theoretical implications paper for publication (optional/negotiable with supervisor(s))

Prepare a possible spin-off paper on material you have generated in a related topic (optional/negotiable with supervisor(s))

Review and incorporate feedback from Examiners/examining panel/Viva audience

Have revisions approved as per the degree regulations

Submit bound and/or electronic pdf copies (per your institutional requirements)

Party

7.7

7.8

7.9

7.10

7.11

7.12

7.13

Table 5.1 (continued) STAGE ONE: 6 months prior to Enrolment STAGE TWO: Year 1 PT; Sem 1 FT STAGE THREE: Year 2 PT; Sem 2 FT STAGE FOUR: Year 3 PT; Y 2 Sem 1 FT STAGE FIVE: Year 4 PT; Y 2 Sem 2 FT STAGE SIX: Year 5 PT; Y 3 Sem 1 FT

!!!!!!!

STAGE SEVEN: Final 6 months

124 5 How Should I Manage the Research Project?

5.2 Research Planning Tools

125

Breaking the whole process into manageable tasks is also advocated by Phillips and Pugh (2015) who suggest “there is a form to a PhD that structures the overall amount of work to be undertaken. This form generates a series of stages that have to be gone through. These stages, in turn, will point to a series of tasks that you will have to do. Going from ‘form’ to ‘stages’ to ‘tasks’ in planning what needs to be done becomes more and more specific to the individual research project” (Phillips & Pugh, 2015, p. 127). In Table 5.1 The Doctoral Research Planning Guide, we have broken the big ‘pineapple’ of a postgraduate research project into stages, for both full-time and part-time students, and have indicated the likely tasks that will be undertaken in each stage. These tasks, or groups of tasks, can then be linked to concrete milestones or deadlines. We do, however, acknowledge that the stages we describe in The Doctoral Research Planning Guide, and for most of this chapter, work best for a fairly linear investigation guided by the positivist pattern of guiding assumptions. However, it will require adaptation for an interpretivist/constructivist or other non-positivist investigation where the stages get more blended together and are more iteratively cross-linked. For example, in a grounded theory approach, a conceptual framework may emerge from the data rather than having been formulated prior to gathering data. Also, data gathering and analysis occur nearly simultaneously in grounded theory research with early analyses suggesting further pathways, questions and data sources to pursue in later data gathering efforts. If you are undertaking a pluralist investigation, it is likely you will have to make adaptations to the general framework as well. To accommodate such variations, we have highlighted the areas in the planning guide where divergences might occur depending upon your pattern of guiding assumptions. Rather than using a project management tool, you may prefer a sequential flow diagram with feedback loops, demonstrating where you need to interact with your supervisor to provide material and receive feedback, and which sections may be iterative; the research journey visualisation in Fig. P.1 might prove useful in this process. In addition, a flow diagram could provide an opportunity for demonstrating where subsequent reflection and review may be required, for example, going back in the final stages of your project and re-writing your introductory chapter in order to provider a tighter presentation of your research. A flow diagram would be able to provide you with a link from the latter part of the thesis back to when Chapter One was first drafted. Some postgraduates prefer to use mindmapping as a way of visualising the various stages of their research and the interconnections between them (note that mindmaps are inherently nonlinear so that they will not work as well for relating those stages to a concrete timeline). One outcome from using the sequential flow diagram or mindmapping approach is that you may find the resulting diagram somewhat complicated. However, this may appeal to your way of thinking. Choose whichever planning approach, whether it is an RJT diagram, a critical path diagram, a mindmap, spiral diagram or a sequential flow diagram, sits most

126

5 How Should I Manage the Research Project?

comfortably with you and will provide a valuable advisory tool for you through the duration of your entire postgraduate journey. We do caution that each particular research project is unique and deviation from these exemplar plans will occur as circumstances (including unforeseen problems) arise, so you will need to create your own plan and be prepared to update it continually. With these caveats in place, let’s take Table 5.1 The Doctoral Research Planning Guide, and elaborate on it further. We have taken into consideration time-frames in the form of stages for both part-time and full-time students. Stage 1 is the first six months prior to enrolment. Each subsequent stage (Stage 2 to Stage 6) equates to six months’ full-time study, or one-year part-time study. The final stage, Stage 7, is the six months associated with final submission and examination of your thesis, dissertation or portfolio. We have listed the myriad of tasks that might fall in each one of the stages and have further coded them as to whether the task is: personal and administrative, involving reflection and academic developmen,t reading and literature development, writing and presentations, or data gathering and analysis. In the development of your personal plan you will need to look closely at the regulations for your institution in order to determine the minimum and maximum time restrictions for your study. Most institutions will stipulate the minimum period of enrolment before submission of a thesis/dissertation/portfolio, as well as indicate the maximum period of enrolment. These regulations will typically be relatively inflexible and need to be closely monitored. It is highly unlikely that you will be submitting prior to the minimum period of enrolment, as most students find they are running out of time rather than under time. Some institutions will allow an extension of time but, once again, it usually requires a formal request. The trick in planning is to work backwards from your intended completion date, providing dates for the commencement and completion of each stage. Having put in the dates for each stage, you will now look at the specific tasks (including possible milestone requirements), contained within each stage. Unfortunately, in postgraduate research, there is very little you can delegate to others (except for possibly transcribing and typing up of interview recordings or quantitative data entry), and it is probably inappropriate for you to do so. As you are the only one doing this project you can only do one thing at a time. Without a team of night elves, it is really one step after the other, so we have taken a sequential approach. As previously mentioned, some of these steps are more iterative and you may need to oscillate between several areas for some time until you achieve an appropriate outcome, e.g., finalising your research questions, constructing your measurement instruments or completing data gathering and analysis. You may also revisit an earlier stage later on in the research to tie things up and ensure there is a natural flow and consistency to your narrative. As there are a number of tasks within the project plan, we will take only a brief run through on what to expect for each of the stages and highlight some areas. All the key requirements of the project plan will be picked up in more detail in subsequent chapters in this book.

5.3 Stage One (Six Months Prior to Enrolment)

5.3

127

Stage One (Six Months Prior to Enrolment)

You will note that we have started the planning process some six months before you actually enrol and suggest that a lot of preliminary work can be undertaken during this first stage.

5.3.1

What Can I Do Before I Actually Enrol to Adequately Prepare Myself?

We briefly touched on setting goals in Chap. 1 but, when planning your project, you may wish to initially re-visit, refine and re-state your intentions around doing a postgraduate research degree and to firm up the goal with a specific time-frame. Don’t be embarrassed to visualise your success as it is a common motivational tool. However, there is a difference between dreaming about having something in the future, and visualising actually having it in the future. “Visualising implies a more structured and disciplined view of what you are trying to accomplish. By visualising, you look at your goal from many different viewpoints” (Gleeson, 2000, p. 113). This may be visualising yourself submitting your examinable research outcome to the university, walking across the stage to collect your degree, phoning your parents/partner after completing your oral defence (if one is required), making your first booking at a restaurant as Dr … whatever pushes your buttons, gets an emotive reaction and will spur you on is worth visualising at opportune moments. In addition to setting your goals, you will also be developing a tentative topic and broad research question and undertaking some preliminary literature searches. Commence your journal and start making notes. Keep a list of key words relevant to your research and record them in your research journal. Continue adding to them as your vocabulary on the subject area widens. Read broadly within your intended topic area. Prior to enrolment, if you have not done so already, you should be undertaking institutional and program searches in order to narrow down the university at which you will enrol. There are pragmatic dimensions that also need to be considered such as securing resources, grants and submitting applications as well as considerations of living accommodation, living costs, available schools and other community amenities. Once you have selected an institution, you then undertake the process of investigating potential supervisors and further exploring potential topics. Become fully acquainted with your chosen institution, its support structures and resources. If you are unfamiliar with the university you have enrolled in, explore the facilities and find out about the services available to postgraduate students. Top of this list should be a library tour and any training that you require to effectively manage remote on-line access to your institution in relation to coursework, database searches and support. Take a look at completed theses/dissertations/portfolios in your area, typically available through the library, to get an idea of what topics have

128

5 How Should I Manage the Research Project?

been studied before and the approaches taken. Pay particular attention to research outcomes that have been previously supervised by academic staff in your intended department so that you gain an indication of potential sources of expertise. You should also acquire a copy of any program regulations and become fully acquainted with them as part of this preliminary phase. If you have not done so already, set up your work space, filing system, computer and software requirements, particularly the bibliographic recording system you have chosen (EndNote being the most common). In some cases, institutions or schools will have a preferred referencing system they want you to use and it will pay to devote some time to learning the ins-and-outs of that system. Start making notes on your reading now, enter references into EndNote and start practising with any software you are not familiar with. In this preliminary first phase, you will also be confirming the commitment and support of others who will potentially be affected by your extended study.

5.4 5.4.1

Stage Two (Year 1, Part-Time/Year 1, Semester 1, Full-Time) What Should I Expect in My First Year?

As most advisory texts on postgraduate study indicate, the first year of your study is critical and the need to show significant progress will not only be a university requirement but will motivate you and lay the foundations for the rest of your studies. In examining the problems that first year social science students tend to encounter, Hockey (1994) observed that the first year is the most critical period and it is within this time that students encounter “their point of maximum novelty and, in turn, possible difficulty” (Hockey, 1994, p. 177).

5.4.2

What Should I Do If I Think that I Am Studying in the Wrong Place?

If, in the first few months of your candidature, you find that the institution is just not for you, that is, you are not comfortable there or the supervision is woefully lacking in expertise in your subject domain or in meeting your needs, you need to consider a possible move. If it is purely homesickness or settling-in difficulties, try to weather the storm. If, however, you feel that the difficulties you are experiencing, such as a poor library or inadequate supervision, are going to significantly impact on your postgraduate study, most definitely make the change to another institution.

5.4 Stage Two (Year 1, Part-Time/Year 1, Semester 1, Full-Time)

129

It is better that you do it now, in the early stages of your doctoral work, rather than mid-stream or later. Changes at a later stage can severely impact on the time taken to complete your postgraduate research.

5.4.3

What Is Provisional Registration/Probationary Candidature?

In some universities, provisional registration or probationary candidature may be imposed on you. Provisional registration, as the phrase suggests, entitles you not to full enrolment but provisional enrolment. As a consequence, there is a proviso on whether you will be able to continue, and that proviso is often termed ‘confirmation of candidature’. Provisional registration or probationary candidature is usually reviewed at the end of twelve months and/or following submission of a research proposal, which would include a provisional thesis/dissertation/portfolio title, a discusssion of research questions and intended methodological approach, an outline of the intended research timeline and outcome structure and a statement of the resources you will require to complete your research. There are a variety of reasons why provisional registration or probationary candidature might be imposed. In the past, for example, some students have come from predominantly coursework Masters, such as MBAs, and there is some hesitation regarding their ability to independently perform at a postgraduate, especially doctoral, level of research. Don’t let it weigh too heavily on you; just keep working on your research plan and demonstrate your ability to perform. In fact, you can use a confirmation hurdle as an additional tool in your motivational kit: get to that hurdle with strong strides and get over it; beyond lies the rest of your postgraduate study!

5.4.4

What Other Activities Should I Expect in Stage Two?

Stage 2 is all about creating effective working relationships, generating a topic and research questions and establishing good work practices. This is when you will actively engage with your research journal writing (if you have not done so already; recall Chap. 3). As mentioned in Chap. 2, good communication skills and networking play important roles in one’s postgraduate research journey. Some of the most important working relationships will be with your supervisor(s). It has been acknowledged that a good relationship with one’s supervisor(s) is directly linked to student satisfaction and disappointment iand s a critical for a candidate’s well-being and success. Conversely, a poor relationship will ruin a good doctoral project regardless of any other factors (Katz, 2016). By now your university will have confirmed who your supervisor or supervisors will be. You may have more than one supervisor or even a supervisory panel and, especially in the case of a professional doctorate, one or more supervisors may be

130

5 How Should I Manage the Research Project?

non-academic professionals. At this stage, you need to work on establishing the relationship(s) and expectations as to how your supervisors wish to work with you and, equally importantly, if you have more than one supervisor, how they will work with you and each other. If your supervisors comprise a mixture of academics and non-academics, you will need to negotiate expectations with all of them in the room (actually or virtually) and this will take effort and willingness to compromise with respect to all parties. Conflicting advice needs to be resolved if you are to make progress. At the start when there is a lot to discuss, schedule fortnightly or more frequent meetings on your project plan with your supervisor(s), and ensure the meetings are both in your diary and in theirs. Changes will occur, but at least they are in the diary and will act as an impetus for you to progress your work and be ready for the meetings and relevant discussions. Keep the dialogue open and frequent as you develop rapport with your supervisor(s). Seek out their advice; supervisors like to feel that they are adding value to your experience so don’t hesitate to say when you have found a meeting, or information they have provided, particularly useful. They too sometimes need to be encouraged and acknowledging their input into your learning will certainly strengthen the relationship. Chapter 4 provided further advice on how to get the best out of supervisor/student relationships. At this time, you are also advised to get to know other students, as other postgraduate students can provide an immense amount of peer support. It is worthwhile making the connections as well as strengthening relationships with administrative staff in the department, including those who are external to the department but are important to your time as a postgraduate student. For example, the staff in the admissions or research services office, the library and support services can prove invaluable allies in smoothing out bumps in the road. In the first six months to a year you should also be undertaking any relevant course work, seminars, workshops or training that may be required to generate necessary skills or specific competencies. For example, knowledge of statistics or skills with specific computer programs such as SPSS, MAXQDA or NVivo, could save a considerable amount of valuable time later. Start actively writing your research journal and get into the habit of regularly recording your thoughts in it as you continue to read extensively in the literature surrounding your topic and explore related domains. It is also advisable to look again at theses/dissertations/portfolios in relevant discipline areas to get an indication of what is involved, what such research outcomes look like, the types of methodologies used, and the standards and presentation requirements of your university. As it is now a more common practice to co-supervise postgraduate students outside of one’s home institution, not all the research outcomes that your supervisor has supervised will be in the library and you may wish to ask him or her if you can borrow those you cannot access from the library. This way you will get greater insight into the patterns of guiding assumptions, research frames, contextualisations and positionings and data gathering/data analytic approaches with which your supervisor is most familiar or most closely associated.

5.4 Stage Two (Year 1, Part-Time/Year 1, Semester 1, Full-Time)

5.4.5

131

How Do You Generate a Research Topic?

Undoubtedly, one of the critical areas that students struggle with during the first part of their postgraduate program is coming to grips with a manageable topic. A very good piece of advice is that your research does not have to debate “the meaning of life” but it is far better to find a gap in the knowledge that can be explored in the time that you have (Keshavan, 2012). Chapter 11 will look at developing your research problem in more detail but, by way of introduction, research topics are usually generated from a variety of sources, for instance: • A social issue or problem—you may, from your reading of the popular press, become acutely aware of a social issue or problem which you believe requires further investigation. • A gap in the literature—you may have identified an area which has not been explored and from which your research topic and question can be generated. • Personal interest—a research topic about which you currently know little, but in which you have a genuine personal interest. • The current research field of your supervisor—if a supervisor is actively engaged in numerous research projects, there is the possibility for a postgraduate student to carve out an aspect of one of those research projects for their research project. • An extension of your prior research—it is not unusual for students to feel that there are still unanswered questions following their Masters research and to pick up the thread of it again for a doctorate. In reality, these are not mutually exclusive and choosing a topic could, in fact, be a combination of some or all of the above. There will usually be some subsequent refinement of the topic undertaken with your supervisor(s).

5.4.6

How Do I Make Sense Out of the Wash of Ideas that I Have for My Research?

Generating topics is easy but actually refining one of them into your own postgraduate research topic is tricky. Up until now you will have tried not to back yourself into a corner by keeping your options open and by not narrowing your focus too early into one particular area. You would have explored a number of alternative topics, discussed them with others and looked at the relevant literature. But, as part of re-evaluating potential topics and ultimately getting on with your research project, you will need to narrow your topic down to something manageable that will become the subject of your research. In order to narrow a broad subject into something manageable, you should browse through the literature and internet, reading widely on the general topic. Then, in light of your contextualisation, positioning and the resources you have available, word your general topic as a

132

5 How Should I Manage the Research Project?

question, answer your general topic question, create a list of more narrow topics and sub-questions, and keep an open mind as you strive to drill down to a topic that will produce an appropriately-sized research project covering a manageable topic with well-scoped questions. Bounce these ideas off your supervisor(s) and other graduate students and be guided by their responses.

5.4.7

How Do I Develop Research Questions?

The questions you generate will, of course, be informed by the existing literature, discussions with your supervisor and others, informal discussions with other colleagues, or possibly from preliminary research discussions with stakeholders and likely participants, and from pilot work that you may have undertaken. When creating research questions, initially, generate as many as possible, don’t censor yourself, just write them down and keep adding to them. Then go back later and tighten them even further. Delete those questions that overlap, that you will not be using, and possibly condense others. Now, try to group the questions into a logical theme or order and do some further refinement. Some students continue the process until they come up with one broad research question and about three to five sub-questions or hypotheses (depending upon your adopted pattern of guiding assumptions). We will have more to say about patterns of guiding assumptions in Chap. 9 and the process of conceptualising and stating research questions in Chap. 11. When finalising your research questions, you do not want to cover old ground, that is, questions that have been asked or researched before. You will want to generate questions that are fresh, interesting and intriguing enough to retain your interest over many years. Remember, the more questions you have the more analysis you will require and the more extensive and detailed your findings and discussion will be. Too many questions can quickly blow out the length of a postgraduate research project; you need to strive to set out a feasible set of research questions to focus on. While we are presenting the development of research questions as a linear process, the reality is that there are many points of intense activity, possibly including a number of iterations, reading, observations, data gathering, reflection, and re-consideration, until an outcome is achieved. So, even at the conception point of your research, from the recognition of the over-arching problem to development of your research questions, there will be a number of recurring steps as questions are posed, rejected, modified and refined. While questions are derived at the start of your project, let’s not forget that the same iterative process may also occur throughout your research process and, particularly, in the final stages of your research when appropriate data have been generated. At this point, you may find that you have answers to questions that you did not initially pose but would now like to do so.

5.4 Stage Two (Year 1, Part-Time/Year 1, Semester 1, Full-Time)

133

Naturally, the pattern of guiding assumptions you have chosen will significantly influence the development of research questions, with positivist studies more likely to state the research questions up front with much greater specificity (usually in the form of specific hypotheses), and post priori questions being kept to a minimum. However, that is not to say that they do not occur. With interpretivist/constructivist research, the questions are likely to be more iterative and much more open-ended. With pluralist research, you may have both specific and general research questions. By the end of Stage Two, you should be starting to firm up your research topic and title, having written up your research project rationale (that is, why the topic is important and what the critical questions are). You will also have considered patterns of guiding assumptions and epistemological alternatives, the type of study you are doing, and the frame, contextualisation and positioning of your research. Some students find it quite rewarding to develop a tentative Table of Contents to get a feel for what their final research outcome might look like. Through your reading, you will be getting a good grasp of the relevant concepts and constructs related to your research area and may be attempting to draft an initial conceptual framework. Keep in mind the alternative methods that you might use for data gathering and analysis, and what the potential sources of your data might be. To accomplish all of this, you will need to establish time management systems and to reflect on how you will achieve work/life balance if you are going to maintain momentum but not burn out. Fortunately, at this stage, the motivation is usually quite high and will be propelling you along. Nevertheless, take a moment to reflect on what study locations and times are working for you and, more importantly, where possible improvements could be made. Don’t be too harsh on yourself and expect that you will be working all the time, as you do need to have some distractions such as exercise or a bit of a social and/or family life if you are to maintain some balance. Without this you will get pretty sick and tired of the continuous routine of working. This is analogous to eating a diet of the same food; it can get boring. Similarly, if you are working well by being in the library, that is fine. Your study location may be good for the moment but don’t be surprised if, at later stages, you change your preferred working environment when you find things are slowing. A change of location and study times can help put variety into the process if needed. Work out what works for you but, once again, do try to have a dedicated work space where you can locate all your materials in one place. To create a sense of accomplishment and to prove to yourself that it is not as scary as you think, start writing a draft introductory chapter containing the rationale for your study. The good thing about the Introduction is that you will be going back to revise it as you complete your research outcome, so this is a great chapter just to get going on. You will have a good opportunity later to make changes, so it doesn’t have to be perfect. Submit the draft chapter to your supervisor(s) for critique as this way they can start to see where your thinking is in relation to your topic and to advise you on your writing style. For the more ambitious, you may wish to consider turning your introductory chapter into a journal paper with the main focus of identifying the key issues and research questions in your topic area (this might be something you negotiate with your supervisor(s)).

134

5.5

5 How Should I Manage the Research Project?

Stage Three (Year 2, Part-Time/Year 1, Semester 2, Full-Time)

By now you will have read extensively in the main discipline relevant to your research topic. However, it is also advisable to branch out and review the literature in other potentially relevant disciplines. This will generate new insights, terminology and may introduce additional dimensions to your study. For example, for a PhD student in business ethics, a tangential look at the criminology literature proved to be very useful. Once this foray into unknown waters has been completed, you will be in an excellent position to start preparing your literature review chapter. Chapter 13 discusses creating literature reviews in more detail. However, by way of brief comment, your literature review will not only provide an appropriate structure for presenting the relevant theoretical foundations and related research (if there is a lot of material, some students split this into two chapters), but is an opportunity for you to demonstrate your ability to critique related material, identify gaps in the literature and synthesise ideas across different disciplines. In Stage Three, you will be coming to grips with what your study is about, refining the research question(s) you wish to address, identifying possible sources of the data that you need to answer those questions, and planning how you might gain access to those entities, individuals, documents and so on. As there is usually a significant lead time required to gain access to data sources, some initial contact with gatekeepers at this stage may speed up the process for later, particularly where internal organisational approvals may be required. One PhD student wished to send their questionnaire out with the membership magazine of a prominent professional association. The newsletter was only issued biennially and approval was required, initially from the CEO of the professional association and the board so, consequently, starting early certainly avoided delays later in the study. When you are negotiating data access virtually everywhere you go you will be asked, “So, what is your research about?” Don’t bother counting how many times you have been asked as you will quickly lose count. What is vital is that you have an acceptable response to ensure that whether it is a potential data source or a person you meet at a party, you come across as someone who knows what they are doing. For this, you will need to prepare “the elevator pitch”. As most people only want a one-minute précis of what you are doing, imagine that you are in an elevator and you have to explain your research to them in the time it takes to get from the ground floor to, say, the 10th floor. You want to sound relaxed, not rushed, don’t want to use big words, just state the problem or research question you are investigating and why it is important, that is, what the implications of your study might be. It is worth practising this as you will definitely be using it frequently. Some universities also conduct an annual 3-Minute Thesis Competition. This is a competition where students are challenged to explain their entire thesis on a stage in 3 min. If your university runs these competitions, try one out as it can definitely hone your skills of explaining your research.

5.5 Stage Three (Year 2, Part-Time/Year 1, Semester 2, Full-Time)

135

For a more academic audience where you may be giving a brief presentation, the development and use of a visual depiction of your conceptual framework can be helpful at this stage for positivist studies. For research guided by non-positivist patterns of assumptions, the conceptual framework may be developed much later nearer the end of the study. However, you will still need to have some initial descriptors of what your study is about when presenting at postgraduate seminars or workshops that you will be participating in around this time.

5.5.1

What Does a Conceptual Framework Look like?

Chapter 12 includes detailed discussion on how to build a conceptual framework or model, so the following discussion is merely introductory. Conceptual frameworks are often referred to as “visual models”. They describe the theoretical ideas, concepts, constructs and variables and their potential or anticipated interrelationships that have captured your imagination, and they are the basis upon which your research configuration is formulated. Essentially, conceptual models reduce theoretical thought into key components or dimensions. Leshem and Trafford (2007) have indicated that conceptual frameworks provide focus and that the difficulty in developing conceptual frameworks comes from research methodology texts lacking a common language regarding the nature of conceptual frameworks (Leshem and Trafford, 2007, p. 94). At this stage, you will also be starting to move from what you will study to how you will study, with consideration given to researching alternative methodologies, undertaking additional training on data gathering techniques and data management. In doing so, you will be firming up your research strategy, particularly your methodology and the analytical tools that relate to your chosen methodologies. Here, you will be putting the flesh on the bones of the ‘Data Triangle’ in your research journey. If you have not done so already, you will be finalising your research proposal and, possibly, your research ethics approval application (this may require you to submit draft versions of your proposed data gathering and sampling strategies).

5.5.2

What Is an Interim Report?

In Stage Three, it is not uncommon for universities to require interim reports from supervisors and also from doctoral students, which are usually sent to the research office/registrar/graduate school/head of postgraduate programs. These reports (and possibly a more formal interview with you) will indicate what point you have reached in your study and identify any issues that could be impeding progress. The interim report provides the student and supervisor with the opportunity to indicate whether they perceive any problems, both with the research itself and with the

136

5 How Should I Manage the Research Project?

supervisory relationship. If you are experiencing difficulties with your supervisor, it is advisable to deal with this initially in a face-to-face meeting, rather than the supervisor reading of your concerns for the first time in a document that could surface within the department. Interim reports are commonly due at the end of the academic year and don’t forget that you will need to re-enrol each academic year until you submit your examinable research outcome. Another activity commonly used in universities is the milestone review. This is where the university convenes a small panel chaired by a senior academic who reviews your work to date and asks questions about your research. Once again, the intention is to identify any impeding issues and to offer support and guidance. A milestone review could be repeated annually during your program. Alternatively, postgraduate students may be invited to or required to present an update on their research in a yearly seminar session.

5.6

Stage Four (Year 3, Part-Time/Year 2, Semester 1, Full-Time)

Stage Four involves moving from what you will do to how you will do it. This stage is characterised by the specifics of actually doing your study and where. The activities in this stage centre on developing your sampling plan, negotiating access to data sources, refining your measurement or other data gathering strategies, operationalising your concepts and constructs, where appropriate, and considering how the data are to be analysed. Considerations of these activities will naturally depend on the nature of your study, its frame, relevant contextualisations, positioning of yourself as researcher as well as of participants and other data sources and configuration.

5.6.1

What Are the Main Influences on Data Gathering Strategies?

When it comes down to specific data gathering questions, some postgraduates (as well as some supervisors) have a stronger leaning toward quantitative data collection methodologies, while others are more disposed to qualitative approaches. Pluralist approaches (including what has been termed mixed methods in the literature) have become more relevant as a pathway toward a more convincing research story but are acknowledged to typically extend the time taken at this stage of the study. The research frame and positioning that a researcher has adopted will significantly influence the choice of research configuration, the methodology and data gathering strategies adopted, the questions asked, and the methods of analysis. In

5.6 Stage Four (Year 3, Part-Time/Year 2, Semester 1, Full-Time)

137

short, data gathering strategies should be consistent with the range of choices you have made while conceiving your research project.

5.6.2

What Is Meant by Operationalising Concepts?

In quantitative research guided by the positivist pattern of assumptions, whatever the concepts being studied are, they will, at some stage, need to be measured and, as a consequence, you would need to be giving serious consideration to how you will ask the questions related to each construct as a precursor to commencement of data gathering. For example, you may be using the term ‘family income’. What specifically are you referring to? Is it monthly or yearly income, before or after tax, income of both partners, what about additional family input? Are you talking about salary or additional income from investments? Such terminology needs to be fine-tuned. Note the way other researchers have previously operationalised and defined their concepts in the literature. You may want to maintain consistency in order to replicate their study for the purpose of establishing generalisability between studies. Alternatively, you may feel there is a problem in the way prior researchers have operationalised the concept which you wish to address. Therefore, you will need to build an alternative approach to measurement. However operationalising proceeds for you, the key thing to remember is that it is a process undertaken before any data are gathered and once operational definitions have been fixed, they cannot be changed for the duration of the study. Chapter 18 focuses more closely on the issues surrounding measurement; issues that primarily emerge in the context of positivist research. In Chap. 14, we will discuss measurement as a data-shaping gathering strategy in its own right. The process of operationalising has no real counterpart in qualitative research guided by interpretive/constructivist or other non-positivist assumptions. Instead, what you work toward is displaying a clear and unambiguous story about how the data were gathered and what you are interpreting them to mean, always with the focus on the research participants’ perspectives. Constructs and meanings then emerge and evolve from continual interrogation of the data as they are gathered and analysed. Thus, constructs and meanings typically emerge after data gathering rather than being pre-conceived prior to commencement of data gathering.

5.6.3

How Do I Ensure Data Quality?

Whatever means you use to gather data to address your research questions, they need to meet paradigm-appropriate criteria for demonstrating data quality. In research guided by the positivist pattern of assumptions, the relevant data quality

138

5 How Should I Manage the Research Project?

criteria are construct validity and reliability. These criteria would raise questions such as: • Are your operational definitions precise and do they accurately reflect what you are supposed to be measuring? • Would somebody else operationally define the research concepts in the same way? • Is the intent in the design of your measures to explain, understand and/or predict? • Will your measures provide consistent and replicable indications of whatever construct they are measuring, relatively uncontaminated by various sources of error? In research guided by interpretive/constructivist or other non-positivist assumptions, the relevant criteria are transparency, authenticity and sufficiency. These criteria raise questions such as: • Would independent researchers discover the same constructs and interpretations in the same or similar situations as you have? • Would an independent researcher, given a set of previously defined constructs/ interpretations, match them to the data in the same way that you have? • Are you actually observing/hearing/feeling what you think you are observing/ hearing/feeling, from the point of view of the participants? • Have you sampled enough people/instances/occasions/perspectives/documents to permit the presentation of a complete and sufficiently convincing account?

5.6.4

What About Pilot Testing or Trialling?

Pilot testing or trialling your data gathering techniques is highly recommended, at this stage, in order to iron out some issues that could prove later to be problematic for your study. Some students dutifully undertake pretesting of their data gathering methodology but forget that, in some cases, analysis of pilot data is also required to avoid future problems. For example, in quantitative research guided by the positivist pattern of assumptions, the data gathering method (how the question is asked) could generate nominal or ordinal data which may then restrict the type of statistical analysis that can be done at a later stage. Awareness of this will enable you to go back and possibly recast the question in such a way as to generate the right type of data for effective analysis. By pilot testing, possible limitations of your research can often be identified and addressed before the full study commences. In research guided by interpretive/constructivist or other non-positivist assumptions, data gathering procedures, such as interviews or participant observation, should be trialled in contexts very similar to those in which you plan to conduct your research in order to evaluate the feasibility and adequacy of the

5.6 Stage Four (Year 3, Part-Time/Year 2, Semester 1, Full-Time)

139

procedures you plan to implement. Trialling provides a good training ground for developing your interviewing and observing techniques in context as well as helping you to detect potential issues that may need to be addressed before you commence collecting the ‘real’ data. Once the pilot testing has been completed and changes made, the exciting part of getting your study into full swing commences. Don’t wait until all the data are in before you start the transcribing or data entry process; get onto it as soon as possible. This way you can undertake concurrent tasks and maintain momentum. You will also start to get a feel for the data and be generating some initial observations. Data management can be pretty tedious, so be prepared for that and be conscious that procrastination can often overtake you at this point. Worse still, once you have inputted all the data they will need cleaning, that is, there will be errors in the transcription or miscoding, all of which needs to be cleaned up before you can accurately analyse the data. The analysis will be an iterative process as preliminary findings are discussed with your supervisor(s) and further work undertaken but, by now, you will be starting to get answers to some of those broad research questions you proposed at the beginning of your study. Keep asking the why and what questions? Why did this result come about? What might be causing it? What might be an explanation? We will have much more to say about these issues in Chaps. 20 and 21. With some preliminary data analysis at hand, even if it is from your pilot testing or trialling work, you may wish to start looking at attending conferences in your field. The timing of conference attendance and presentation requires you to think well ahead. For example, an extremely popular Australasian conference for postgraduate, particularly doctoral, students to attend is the Australian and New Zealand Academy of Management (ANZAM) conference which is held every year in early December. However, to have a paper accepted it must be submitted some six months ahead, usually in June, in order to accommodate the peer review and programme preparation processes. As you can see, it is necessary to put these dates into your planning schedule. A conference presentation of your work will provide an excellent opportunity to network and get feedback on your preliminary findings from academics outside your university. You may also be able to identify potential future examiners. So, it is worth the effort of attending. If you are considering attending a conference, investigate potential funding from your department or school (although don’t hold your breath as funding is usually quite tight).

5.7

Stage Five (Year 4, Part-Time/Year 2, Semester 2, Full-Time)

As you enter Stage Five, you will start to realise how much work is actually involved and will need to remain optimistic as you complete your data gathering, analysis, interpretation, and start to write up the results of your study. All too often

140

5 How Should I Manage the Research Project?

you are compelled by the thought, “I wish I had asked that” and are tempted to go back and undertake further data gathering. Alternatively, your findings may have brought up issues that require further exploration. Or, by the very nature of your study, for instance, in action research or grounded theory research, you may be undertaking some further data gathering and analysis. For some students, this would have been part of their research configuration, for others it is an additional activity that is deemed necessary. However, be careful as data gathering can go on forever if you allow it. As one PhD student commented, “While I became a traveller, I did not allow myself to be led just anywhere. I was attracted to particular things and I probed, gently without using excavation machinery, but my purposeful curiosity about my participants’ experiences and ideas left me lacking in innocence with regard to the notion of mining. I allowed participants to wander, but when I suspected that there were no useful souvenirs, I politely directed them back to the bus” (Meldrum, 2008, p. 59). Be conscious of the time commitment that additional data gathering will require and weigh up the benefits versus costs in terms of time. By now you have been living with the project for what seems like far too long and, potentially, are becoming sick and tired of it. To attempt to overcome this you can, at this stage, do some mixing of tasks to generate some variety. Continue with your additional data gathering and analysis but also begin writing your Methodology chapter. Prior to commencing significant writing, you will find it useful to review Chap. 22 and keep referring back to it for guidance throughout your writing process. Another task is to reflect on the data that have been generated to date; what are the preliminary findings, what relevance do they have to existing theory, to new theory, to your conceptual framework, to your preliminary research questions or any new questions? Discuss the results with your supervisor(s) and others and contemplate their insights. Go back into the library and undertake further literature investigations based on your preliminary results. Finalise your drafts of the Methodology and Results and Discussion chapters. Write up a brief summary of the findings and send it to all the participants thanking them for their participation in your study. As mentioned previously, it is not unusual for additional research questions to be generated at this stage. The reality is that the material uncovered for a postgraduate research project is not always strictly sequential. It is not uncommon in the analysis stage to find the answer to a question which you have not previously posed. While this is not ideal, it is not unusual to have a post priori question developed later and inserted in your project. This will require going back and ensuring that the relevant theory and literature pertaining to that question have been covered, and that the question has been included earlier in your research writings. It is also not uncommon for you to delete a question that you no longer believe is pertinent to your study, given the way your study has evolved. Once again, this will require revision of prior material to ensure consistency.

5.8 Stage Six (Year 5, Part-Time/Year 3, Semester 1, Full-Time)

5.8

141

Stage Six (Year 5, Part-Time/Year 3, Semester 1, Full-Time)

At the start of each semester/term you should reflect on and refresh your project plan, where needed. Revising and updating your plan will ensure that you are anticipating the tasks ahead and putting in achievable deadlines. As a rule, expect everything to take longer than it should. As another general guide, at least one meeting a month minimum should now be scheduled with your supervisor(s). In this stage, it most certainly will be a case of a dim light visible at the end of a long tunnel that will get brighter as each week progresses. If you get the opportunity, this is an excellent point to arrange a chunk of time that you can devote entirely to your postgraduate research. As this last stage is characterised by a significant amount of writing, and in order to keep the momentum going, having the ability to work on your thesis/dissertation/portfolio for continuous periods of time will be extremely valuable. You will likely notice that your concentration levels are very high, and a three-hour writing session will just fly by. Also, at this stage, you will be alternating writing with reflection as you develop conclusions based on your findings and considering the implications of your findings for existing theory, new theory, methodology, key stakeholders and practice. Additional work may be required to support or substantiate your theoretical, strategic or practical implications, but make sure this work does not blow out into another project. Although it probably doesn’t seem like it, you are now on the home stretch as you write up the chapter(s) on the conclusions, theoretical, strategic and practical implications of your research. In anticipation of writing your final chapter, you will also be considering what the original contributions of your study are, the critical limitations of your research, and the possible future directions that research could take, following on from what you have done and learned. If not already done, you will be updating your literature review to ensure that you capture recent publications relating to your topic. Although it may seem as if you are cycling backwards through the project, it is also at this stage that you will in all probability rewrite the introductory chapter to better reflect what has become the ultimate focus of your research. The Introduction and Conclusion and Implications chapters are frequently read in that order by examiners, so it is key that, in addition to updating the rationale for your study, there is also clear evidence of continuity throughout the study, particularly given that your research has been conducted over several years and may have evolved significantly from its original direction. A further check and polishing up of your conceptual framework and any data displays should also be done at this stage. The final pieces to be written are the Abstract and the Acknowledgements. Most students agonise over the Abstract given that there is so much to say in such a limited space. As a first draft, try taking the critical points from each of the chapters, wordsmith them and see how that looks. Give the Abstract to a few people to read and garner their honest feedback. You have been living with this project for such a

142

5 How Should I Manage the Research Project?

long time that it is very hard to see it with fresh eyes. It is important that an Abstract is not too obscure to be understood, that is, ensure it could be read by a perfect stranger in your field. In a sense, the Abstract is a somewhat more rounded out written version of your elevator pitch. In regard to Acknowledgements, while it is suggested that you write them now, you may want to hold off from including them in the first full draft. Views may differ, but a more appropriate time to include them is in the final submission document, thus creating a fresh compliment to your supervisor(s), key stakeholders, participants and those who have assisted with and/ or supported you through your journey. The practicalities of your examinable research outcome will kick in as you review the Program Regulations regarding submission timelines and processes. The step that postgraduates under-estimate, virtually without exception, is how long it takes to do the document checking process before the first full draft goes to the supervisor(s) so make sure you allocate ample time in your journey timeline for this process.

5.8.1

What Is Involved with Checking Your Thesis/ Dissertation/Portfolio?

Document checking should entail, at a minimum: • condensing/revising tables, graphs and diagrams to provide a clearest, most accurate and convincing pictorial, tabular and/or textual presentation of your results; • ensuring that your document adheres to the preparation/presentation requirements and recommendations of your department/institution; • undertaking proof-reading, page numbering and layout checks; • undertaking general document checking and polishing (e.g., checking for typos, spelling errors, clarity of grammatical expression; • ensuring that essential items, and only those items, are included the appendices; • checking for appropriate transitional links between chapters; • undertaking bibliographic checking for accuracy, formatting, completeness and consistency between the references in the body of the document and those in the list of references (this includes checking the accuracy of all quotes and their attribution to source); and • If you are a postgraduate student from a non-English speaking background, this stage is where it would be important to ask a native speaker of English review your entire draft document. Once all this checking is complete the first full draft of your research outcome can be submitted to the supervisor(s).

5.9 Stage Seven (Final Six Months)

5.9 5.9.1

143

Stage Seven (Final Six Months) What Do I Need to Do to Finish This Darn Thing?

As Race (1999) has indicated, “a dissertation is never finished; it’s just abandoned at the least damaging point” (Race, 1999, p. 121). Undoubtedly, your supervisor(s) will recommend changes and it is hoped these will be relatively minor, although in our experience there will typically be one or two areas that will need some focused reworking. Postgraduates who ignore their supervisors’ advice do so at their own peril, as the problems identified will also tend to be those that examiners will likely pick up on. In addition to revision of the material following feedback from your supervisor(s), most postgraduates will continue to update the literature although usually only to include selective and highly relevant recent publications. In Stage Seven, you will typically tweak and modify your research outcome in order to improve its clarity and readability and you will finalise your introductory chapter. However, you will reach a point of diminishing returns, when further reading and review will, in fact, not substantively improve things and, if you persist, you may even inject ambiguity. If you know that you are an innately picky individual, you may have to pull back at this point. Perfectionism is good, but it may inhibit you from actually submitting the research outcome for examination. There comes a point where you will need to stop and hand it over to a professional proof-reader for the final review. Usually a month prior to the date of the submission of your research outcome (or whatever is stated in your Program Regulations) you will be required to notify the university in writing of your intention to submit. In some institutions, this is also the opportunity where you may be asked to supplement your notification of intention to submit with an indication of names of potential examiners, and of individuals who may be unsuitable to examine as a result of potential conflicts of interest. In countries such as the US and Canada, the members of your supervising committee will typically be examiners, perhaps along with a nominated external examiner. Alternatively, you may need to provide your supervisor(s) with a list or possible pool of people you think might or might not be good examiners and review this list with them. They will then arrange for examiners and you will remain unaware of exactly who will be examining your thesis (this is common in Australian universities, for example). Note that, in some universities, the student may have no influence or knowledge of who might examine their thesis, dissertation or portfolio, which is unfortunate, but one needs to live within the regulations of the university. Whatever your institutional requirements are, you should know that you will need to allocate some time and effort to this task. In addition to the usual examiner review of your research outcome, some postgraduates will need to prepare for an oral defence known as a viva. Chapter 24 discusses how theses/dissertations/portfolios are examined and judged. While you are waiting to incorporate the examiners’ feedback (which typically takes about two to three months), start planning your future after you complete your degree. If this

144

5 How Should I Manage the Research Project?

means looking for a job, now is the time to start looking at job advertisements, getting your CV organised and preparing to party when the final changes have been made as a consequence of feedback from examiners!

5.9.2

As a Part-Time Student, If I Were to Take Time Off from My Employer for Concentrated Study, When Would Be the Best Time(s)?

If you are going to take any blocks of time off from work to focus on your study, the best times are right at the beginning, when you are exploring the literature and determining the scope of your topic and research questions, and in the final stages of the writing-up of your research outcome when such time can be more effectively used. This can be a particularly intensive time and a high level of concentration is required in order to achieve an integrated research outcome that tells a coherent and convincing story. Some students also benefit from the occasional day or two at points where they feel they are getting behind. Missing targets can be very demotivating and taking a few days to get back on track can help with ensuring that the total plan does not start to unravel. Whatever you do, try not take a new job before finishing your postgraduate journey. Supervisory experiences are littered with examples of students who have neared completion and then have taken on a new role or position at another institution with disastrous consequences for their study. Separation from the usual work environment and study practices, coupled with the demands of a new job and the trials of resettlement can, anecdotally, put students back one to two years.

5.9.3

What if I Want to Suspend My Studies?

Sometimes life events get in the way of your study. If you have determined that it is better to put things on hold in order to give your full attention to the issue at hand, you will be looking to suspend your studies. The length of time for which a student’s registration may be suspended will be commonly in multiples of whole calendar months. It will usually be for a period of 6 months but commonly not longer than a year. If it is longer than a year, you run the risk of your enrolment being terminated. You will need to check your Program Regulations to see what the time limitations and notification requirements are, and to whom you must apply for approval. If you are studying on scholarship, make sure you understand the requirements and implications associated with suspension of candidature. Don’t let your need for a suspension of candidature come as a big surprise to your supervisor (s); meet with them first to discuss your intentions before putting in any formal requests for suspension (typically they will need to sign off on your request). Also,

5.9 Stage Seven (Final Six Months)

145

keep in mind that the longer that you are away from the research, the longer the lead-in time required to get back up to speed when you do ultimately return.

5.9.4

What Happens if I Want to Terminate?

Regrettably, terminations of enrolment in a postgraduate research program can and do happen. The most common reasons for terminating a postgraduate research program are: • • • • • • • • •

you are no longer interested or motivated to continue; failure to re-enrol for the academic year; failure to make payment of tuition fees; you did not complete coursework requirements to the standard required; you have not made satisfactory progress under the program regulations, usually as a result of the submission of an unsatisfactory periodic report by your supervisor(s); you failed more than one milestone review (e.g., your candidature is not confirmed); you are experiencing personal difficulties that are impeding your ability to complete the program; you are moving abroad and are unable to continue with close supervision or with your current supervisor(s); or your examinable research outcome has not been submitted or resubmitted in the time stipulated.

If you are leaving a postgraduate program and were unhappy with the way you were treated by the department, school or an individual in relation to your studies, you may wish to ask for an exit interview with the Head of Department or the Dean of Postgraduate Students or equivalent. This way constructive feedback can be received and put into the program process for future postgraduate students.

5.10

Conclusion

Undertaking research that will culminate in a thesis/dissertation/portfolio is a unique type of research which has been captured in various metaphors such as a journey, an apprenticeship, a transition from lay person to reasoning enquirer. The concept of a journey, which is the metaphor we adopted for our book, is also helpful in a wider sense. Here is a quote from a PhD student that speaks to the research journey.

146

5 How Should I Manage the Research Project?

The research journey begins at a point that might be called Curiosity, and it ends in a far away location named Thesis. On the way, many places are visited and many people are met. Some places are visited so often that the traveller slowly comes to feel comfortably at home. With some others, only the key landmarks become known, and others again are passed through slowly and without stopping. (Meldrum, 2008, p. 48)

What often characterises descriptions of the research journey is that the process is somewhat complex but has a sequential element to it that unfolds, with one step forward and possibly two steps back but, nevertheless, is always heading in a set direction. The importance of planning and adapting the plan has been recognised as a key dimension of postgraduate study. Postgraduate research degrees are awarded to students who have demonstrated the general ability to conceptualise, design and implement a project for the generation of new knowledge, applications or understanding at the forefront of the discipline. So the ability to plan but also to adjust the project conceptualisation in the light of unforeseen problems is essential. When considering your management of your research journey, there is, essentially, a three-step strategy. The first is the “hover”, where you get an understanding of the totality of the project. It involves envisaging the research project in its entirety. The next step is “slicing” the project up into chunks or stages, and the third step is “dicing” these activities into even smaller pieces or tasks. These tasks could be further refined on a weekly basis with the creation of a “to do” list of activities that need to be undertaken that week. Essentially, you need to have the ability to plan both in the long-term and in the short-term. That is, to: • develop an overall structure for your research; • define the stages and the key steps and understand the resources you will need to call upon to complete each; • set realistic timelines/milestones/targets; • operate a yearly, monthly, weekly and daily planning system; • set up regular meetings, face-to-face or virtual, with your supervisor; • develop a structure for your final research outcome; and • revise the plan and targets as your research progresses. We noted earlier that project management need not be complicated for the purposes of planning your research journey. Whether you use a project planning tool, a mind map, a good old Excel spreadsheet or even a schedule in MS Word, make sure that you have an electronic version as you will probably need to modify your timelines and other details at a later date. Have one copy of your research plan pasted into your research journal and the other on the wall next to your work desk. All key deadline dates should be entered into your calendar and it is appropriate to work backwards in the time frame. In other words, work out when it must be finished, and plan backwards from that point. To help develop your timeline, we have provided a tentative plan and you would be advised to look at it and overlay your own research approach. Discuss your proposed plan with your supervisor(s) in order to more accurately ascertain the key dimensions of your research and the probable timelines and resources required. As

5.10

Conclusion

147

mentioned, we have taken a fairly generic approach to the research process and treated both qualitative and quantitative research guided by different patterns of guiding assumptions with similar timelines, so some further refinement might be needed in relation to your research. Engaging in some form of planning will enable you to prioritise the process and not be overwhelmed. It will introduce some certainty and help minimise risk. You will need to take full responsibility for your project. Where you have identified that you do not have the necessary skills in order to progress, it is up to you to take responsibility for developing those skills. The only person you can count on in order to reach the end goal is yourself. When you have a bump in the research journey, it is up to you to work out strategies to overcome the difficulties. If you find that the research is not proceeding according to the timeframe, it is up to you to revise your plan and re-schedule your work. Your supervisors can support you in your journey, but they cannot take that journey for you. A plan is only any good if it is being actioned, that is, if it is being implemented and the deadlines are being met. A sense of accomplishment is quite empowering, whereas, if you are falling short of the deadlines, it can be particularly de-motivating and stressful. For planning to be meaningful, the deadlines must be realistic. Some postgraduates don’t realise that while there is a sequential element to undertaking a big research project and writing it up, the inner activities are actually quite untidy and iterative. There is sometimes a lot of back-tracking, revising of questions, refitting or reshaping of data, asking questions at the analysis stage that weren’t formed in the beginning, adapting to changes in data sources, etc. This requires a lot of adaptation along the way, hence the need for flexibility. Things won’t always go the way you intended them to, and, as a consequence, you need to be open to variations in your plan. It is expected that you will periodically go back and refresh the project plan so that it remains realistic and achievable. Use achievement of milestones as way points to reward yourself for what you have accomplished.

5.11

Key Recommendations

Here are some key recommendations to keep in mind when developing a project plan to manage your project: • Check your regulations regarding the minimum and maximum completion times. Most students start off with a strong desire to finish well within their timeline, however, it is probably best to work on the realistic side and take the maximum time in order to give yourself as much opportunity as possible to do a good job. • The length of time associated with undertaking postgraduate research can be extremely misleading, giving students a false sense of security regarding the time available. However, experience has indicated that a leisurely approach is

148

•

• • •

•

• • •

• •

5 How Should I Manage the Research Project?

problematic and that you will need to keep moving if you are to accomplish key steps within realistic timeframes. If you enrolled in part-time study or are doing your research degree at a distance (i.e., you are not resident on campus), you cannot be lackadaisical about keeping up your momentum and maintaining contact with supervisors. When establishing your timeline, speak to others who are doing, or have completed, their research journey. Also, set up your timeline in conjunction with your supervisor(s) who have experience with the process and can also provide input as to their availability at certain key periods. Be aware of when deadlines are looming and keep to them. In doing so, you will not let the project plan slip and will avoid running over time. Keep to your schedule but maintain the latitude to modify it when needed. As early as possible, set up your work space, establish your calendar, a filing system, a meeting schedule with your supervisor(s), relationships with other graduate students, and start reading. For part-time students, if you are going to take time away from your employer for more concentrated study, probably the best time would be at the start of your research journey when you are first reviewing a lot of material and literature and doing most of the conceptual work surrounding your research and at the end when you are doing the write-up. It is important to start writing as soon as practical within your research process. This will enable you to sort out your writing style and parameters (when and where you write best) as well as to start developing your ability to document the process as you are undertaking it (i.e., keeping a research journal). Submitting drafts of preliminary chapters will provide you with the opportunity to receive feedback from your supervisor. It is also quite satisfying to see the pieces build with the addition of new pages and chapters. Present at workshops/conferences in order to get feedback and further insight as well as to develop/extend your professional network. It is a good idea to have a couple of pre-prepared items, both written and verbal, which explain your postgraduate research. One will be the ‘one-minute’ version that you will deliver to interested parties in the elevator, or your grandmother, and a ‘three-minute version’ which you will be able to articulate in postgraduate workshops, seminars, when you are asked to introduce yourself and your topic, and when you encounter other academics as well as non-academics. Take full responsibility for your project. It’s yours, not your supervisors’. It will only come about through your concerted efforts. Try not to change jobs or start a new job until you have completed your postgraduate research journey; timing is critical! In most cases, a research journey is incompatible with a new job journey.

References

149

References Araujo, E. R. (1995). Understanding the PhD as a phase in time. Time and Society, 14(2/3), 191– 211. Backlund, F. (2017). A project perspective on doctoral studies—A student point of view. International Journal of Educational Management, 31(7), 908–921. Bruin, J., & Hertz, B. (2010). Project management for PhDs. The Hague, Netherlands: Eleven International Publishing. Enago Academy. (2018). How to manage your research project. Crimson Interactive Inc. Retrieved December 31, 2018, from https://www.enago.com/academy/dont-let-science-fail-researchproject-management/. Finn, J. A. (2005). Getting a PhD: An action plan to help manage your research, your supervisor and your project. London: Routledge. Gleeson, K. (2000). The personal efficiency programme: How to get organised to do more work in less time (2nd ed.). Etobocoke, Toronto, Canada: Wiley. Gosling, P., & Noordam, L. D. (2010). Mastering your PhD: Surviving and success in the doctoral years and beyond (2nd ed.). Heidelberg: Springer. Grover, V. (2007). Successfully navigating the stages of doctoral study. International Journal of Doctoral Studies, 2(1), 9–21. Hockey, J. (1994). New territory: Problems of adjusting to the first year of a social science PhD. Studies in Higher Education, 19(2), 177–190. Katz, R. (2009). Shorten the time to Doctorate. A guide to managing your PhD project. Bloomington, IL: AuthorHouse. Katz, R. (2016). Challenges in doctoral research project management: A comparative study. International Journal of Doctoral Studies, 11, 105–125. Keshavan, M. S. (2012). How to come up with a research idea. Asian Journal of Psychiatry, 5(1), 108–110. Lantsoght, E. (2013). The smart way to manage a large research project. Next Scientist. Retrieved October 5, 2018, from http://www.nextscientist.com/manage-a-large-research-project/. Leshem, S., & Trafford, V. (2007). Overlooking the conceptual framework. Innovations in Education & Teaching International, 44(1), 93–105. Meldrum, R. J. (2008). A curriculum for entrepreneurial creativity and resourcefulness in New Zealand. Unpublished PhD thesis, Deakin University, Victoria, Australia. MindTools. (1996–2018). MindTools: Essential skills for an excellent career. Mind Tools Ltd. Retrieved October 4, 2018, from https://www.mindtools.com/. Phillips, E., & Pugh, D. S. (2015). How to get a PhD: A handbook for students and their supervisors (6th ed.). Maidenhead, Berkshire, UK: Open University Press. Race, P. (1999). How to get a good degree: Making the most of your time at university. Philadelphia: Open University Press. Smartsheet. (2018). Retrieved November 16, 2018, from https://www.smartsheet.com/product. Thomas, D. (2016). Every app you need for your PhD. Retrieved October 5, 2018, from https:// www.blogs.hss.ed.ac.uk/pubs-and-publications/2016/12/16/every-app-need-phd/. Thomas, G. (2017). How to do your research project: A guide for students (3rd ed.). London: Sage Publications. Vitae. (2018). Project management tools for researchers. Retrieved October 5, 2018, from https:// www.vitae.ac.uk/doing-research/leadership-development-for-principal-investigators-pis/ leading-a-research-project/managing-a-research-project/project-management-tools-forresearchers.

Chapter 6

How Should I Manage My Time?

6.1

Managing Time

With increasing pressures being placed on institutions for timely degree completions and with funding often being directly linked to those completions, universities are now becoming very intent on ensuring that students finish their postgraduate (especially doctorate-level) study within a specified time frame. The likelihood of being able to extend your time is limited and this puts pressure on students to fulfil all the obligations of their postgraduate study within the time specified by the regulations inherent in their qualifications. For full-time students the time frame is, on average, three years for doctorates in Australia, New Zealand and the UK (often with no required coursework), while for part-time students it can be up to six years. In the US, a full-time doctorate takes about five years, including any required coursework. Depending upon the program, a research master’s degree may take 1–2 years. The time frame may also include elements of a taught program covering areas such as research methodology and proposal preparation. The pressure for completions, therefore, translates into the need for you, as a student, to manage your time effectively.

6.1.1

So, Where Does My Day Go?

There are only so many hours in the day and, of those, only so many can be devoted to your postgraduate research; what time you do have to work on your research needs to be productive? Being productive doesn’t mean full-on theoretical development all the time; it could just as easily be full-on filing, data entry or writing. The secret of managing your time is to work not harder but more efficiently (Lewis & Habeshaw, 1997, p. 12). By being productive and more efficient, you will be (a) working on the tasks that you should be working on that are going to yield the © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_6

151

152

6 How Should I Manage My Time?

best outcomes for your project at that point in the process, and (b) giving full and devoted attention to the task at hand. To achieve the first part, you will need to go a step further in project planning, prioritising and breaking down scheduled tasks. This is down to the level of what activities you will be doing this month or week as well as what tasks you will be doing in each work session. The second part of being productive and managing your time is ensuring that you are focusing on the designated tasks and avoiding the bountiful array of activities that can seduce you away from the work at hand. Essentially, it means avoiding procrastination and the lure of unrelated activities. For effective time management, regrettably, there isn’t one golden rule which, if followed, will instantly make you more productive. Rather, there is a series of dimensions that can both enhance and frustrate your time management and that impact on your productivity. They relate primarily to scheduling your study sessions, establishing your working environment and organising materials. They are not nearly as exciting as talking about structural equation modelling or grounded theory, but it is these practical dimensions that can impact on your work performance, the meeting of deadlines, your level of enjoyment and the timely completion of your research journey. If all of these dimensions are managed well and in alignment, there is no doubt you will significantly enhance your time management and achieve more in the time allotted to your research. Before we look at these dimensions, let’s first clarify that the reality is that we don’t actually manage time—we manage ourselves within a time frame. To get an indication of how you actually use your time, do an audit of a few working days. Keep a diary of all your activities divided into 15-minute intervals. A couple of apps which you may find helpful here are ManicTime (https:// www.manictime.com/) and RescueTime (https://www.rescuetime.com/), both of which can help you track how you spend your time. I think you will be surprised and perhaps appalled at how much time you actually are unproductive. Ask yourself: • • • •

What are you doing well? What would you like to improve? What is wasting your time? What is saving you time? (Time management for PhD students: http://www.bbk. ac.uk/downloads/research/time-management.pptx/at_download/file)

So, let’s look at dimensions of time management and provide some practical advice in each area. The advice may sit comfortably with you, or not. It is your call as to which ideas you try out and adopt. But don’t lose sight of the dimension that you are trying to manage and, if you have a better strategy for managing that dimension, by all means use it.

6.2 Planning and Prioritising

6.2

153

Planning and Prioritising

The first key dimension to time management is to actually undertake some formal planning rather than just plod along from one task to another (recall Chap. 5). For a postgraduate student, Cryer (2006, p. 107) has highlighted the benefits of formal planning. These are: • easing anxiety by externalising your planning so that it is not constantly occupying your mind; • providing a focus for discussions with supervisors and others; • providing a sense of security that you are ‘on track’; • preventing you from spending too long on only vaguely relevant activities just because you like doing them; • allowing you to enjoy taking time off with a clear conscience; and • providing a basis for reflecting on what has worked and what hasn’t so you can plan more realistically in the future. A postgraduate research journey is just that—a journey which has identifiable stages. By breaking down the stages and project managing the process, you will be able to identify units of work, each with a beginning and an end, and to assign approximate time lines for their completion. By identifying discrete steps, such as: literature review, development of conceptual framework, data collection, data analysis and so on, it will most certainly help you break a big project into manageable pieces and you will find it motivating as you are able to tick the activities off as they are completed. A helpful text is Shorten the Time to Doctorate (Katz, 2009). Of course, for planning, you could just use your Google calendar with the added advantage that your calendar will be synced to all you other devices. By using a project plan, you will be able to see in advance what will be required and can then schedule your work to ensure no hold-ups. For example, negotiating access to a research site will need to occur some 3–4 months before you actually commence your data gathering. Being aware of impending tasks and adequately preparing for them will minimise the difficulties associated with incomplete or delayed information. Also, by realising there are some things that could hold you up, such as getting an interlibrary loan, access to a data source, scheduling interviews with key participants, or receiving draft chapters back from supervisor(s), you can incorporate them into your plan to ensure no down time and a smooth flow of work. Listing all the stages in a nice planning document is, however, not enough. You now need to take that planning tool and make it work. To do this, you need to operationalise the plan by breaking the stages down even further into tasks that can be scheduled into a weekly and sessional basis. At this point you are advised to move your project plan material onto a wall planner or electronic diary. On the yearly calendar put in key dates, for example, academic year start date, when your proposal is due, meetings with your supervisor(s), when you hope to have the first draft of introductory or other chapter submitted, any conferences you wish to attend

154

6 How Should I Manage My Time?

and, if so, the submission deadline for papers (put in the dates including travel days). It is also good to enter any dates when your supervisor is going to be away into the calendar. Incorporate the milestone steps from your project plan. Possibly using a different colour, put in the other dates for main life events through the year: birthdays, anniversaries, school holidays, and if you plan to be away for Christmas. Block out time when you will not be able to work on your research because of other commitments. Keep adding to your calendar as time progresses and new events arise. Any periodic revisions to your project plan need to be included in the calendar as new deadlines are determined. Now, each stage in your research plan will need to be broken down even further. It is all very well to state “make initial contact with potential data sources”, but there are a number of sub-tasks, such as internet searching, writing letters, making telephone calls and arranging meetings, that go into achieving that outcome. Of the myriad of tasks to be done, you will have to consider what tasks have priority this month, this week and today? And what about all the other things in your life that you need to accommodate? So, now focus on the next two months and break the tasks down even further. To the best of your ability put in the tasks, deadlines, events and commitments. This can be personal, work-related and notably tasks associated with your research. You will, of course, be adding to this on a regular basis. The weekly schedule will have specific meeting times, the times blocked out for classes, where appropriate, times blocked out for exercise and time for your study sessions. If you go to the gym three times per week, put it in and don’t forget to include travel time. If this is starting to sound overly pedantic, it is for a reason. You need to be extremely conscious of scheduling in your study if you are to be efficient with your time. We are now down to the day and specific hours in the day. Having already put in your weekly requirements, the daily log is to ensure that you maintain your schedule and to come up with your current To Do list. Useful electronic tools for creating To Do lists include Google Tasks (https://blog.hubspot. com/marketing/google-tasks) and the wonderfully titled apps Remember the Milk (https://www.rememberthemilk.com/) and Wunderlist (https://www.wunderlist. com/).

6.2.1

But What If I Am Not a ‘List’ Person?

Sorry, but given the enormity of the task and the imperative that you maintain momentum, if you are to complete on time and actually lead a balanced life, this type of detailed scheduling is necessary. It is impossible to keep all this information in your head and you need to have it in front you if it is going to act as a motivator. So, use your research journal to plan for tasks that are coming up, what needs to be done and to compile your daily To Do list. Carry the journal around with you at all times not only to record your thoughts but to update and refine your To Do list. Once you have recorded items either in writing or digitally, you can then relax,

6.2 Planning and Prioritising

155

knowing that you have things scheduled. Now you don’t have to concentrate on remembering things or stress about them. Before the start of each week, on Sundays, review what you need to do in the coming week. Do this in a quiet place for 20–30 min, determine what is important for the week and put the priorities in your calendar. Don’t fill up the entire day, leave some spaces free, but do put in the ‘big rocks’, those things that you definitely want to achieve. Having done your weekly planning, you are now prepared for your daily planning. At the beginning of each day (or at the end of the preceding work day), somewhere quiet, create your To Do list, it should take only 5–10 min. Yes, there will be spill-over tasks not completed from the previous day (or even the previous week), so try to be realistic with what you plan to do so as to avoid disappointment at the end of the day if you haven’t achieved everything on your list. Each day, check your weekly planning list to see what you need to be accomplish, transfer unfinished tasks onto your new list and then give an indication of priority for the most important jobs to be done. Some people like to colour code tasks using, for example, red highlight for upmost priority, orange highlight for important but not urgent, green highlight for minor tasks. Feel free to mix in some non-work-related or personal tasks on your list if they need to be done that day.

6.2.2

But I Need the Panic of an Immediate Deadline Before I Start Anything?

If you are a person who likes to leave things to the last minute because you secretly enjoy the rush of a deadline rather than planning things out, you should rethink this. A rush of adrenaline may motivate you and sustain you in the short term, but it is not something you can maintain over an extended period of time. Having a continual stream of stressful episodes over a number of years will most certainly be detrimental to your health and well-being and the build-up of stress can lead to counter-productive behaviours such as procrastination, deliberate seeking of distracting activities and less attention to detail.

6.3

Scheduling Research Sessions

Surprisingly, scheduling productive study sessions is often harder for full-time postgraduate students than it is for those studying part-time. Full-time students often find that with their entire commitment on one piece of work they have a large amount of time, but it is particularly unstructured. With a whole day to do something, things can be put off, procrastination is common, and the coffee shop or social media is a common retreat. Part-time students are usually run ragged with studying, trying to perform well in their full-time job and, possibly, also

156

6 How Should I Manage My Time?

undertaking business travel. For both full-time and part-time students there is also the added pressure, and ever-present juggling act, of trying to do postgraduate research while doing all the usual activities to ensure that you are fed, looking clean and tidy, paying the rent while also maintaining relationships with a partner, friends, family and, potentially the greatest consumers of time, children. For a full-time student, it is unlikely that you will work on your research fewer than 50 h per week and, for part-time students, it is not so much the hours (probably around 25 per week) but the frequency that is vital. Try to do something every day on your project. The longer you leave it, the harder it is to get back into it, so the frequency of research sessions is particularly important. When you have large chunks of time between sessions, the first hour or two is a slow climb back up the learning curve as you re-acquaint yourself with your material. However, if you schedule research sessions every day, re-acquaintance time is minimised to as little as ten minutes and, as such, the session is a lot more valuable given that you are productive much earlier in the session. More frequent sessions also provide continuity of thought and consistency in your writing style. If you have ever picked up your last point on a manuscript, re-read it and then asked yourself the question, “What was I trying to say here”, you will know what is meant about leaving it too long between sessions. It is also true that if you work daily on your project, when you are not at your desk, your subconscious will still be working on it. Writers have frequently found that when they resume their position at the computer the next day, the paragraph that they were working on previously flows better having been mulled over in the mind overnight. The power of the subconscious, unfortunately, only works when you are interacting regularly with your material. If you have not already done so, take time now to draft out on a weekly basis when your research sessions might be and try to get the sessions as close together as possible, ideally working every day on your research, even as a part-time student. A session of as little as half an hour can be worth pursuing.

6.3.1

How Do I Find Extra Time When I Have a Pretty Full Life as It Is?

Right from the beginning of your journey it is pivotal to recognise that sacrifices will have to be made. There are just not enough hours in the day to do everything you have been doing before as well as your research. The critical questions are— what are you prepared to give up, as well as what do you want to maintain, and what changes are you prepared to make in your life to accommodate the extra workload? This is about establishing a routine and balance that works for you and sticking to it. To schedule sessions that work best for you, be aware of your own body clock. Are you an owl, or a fowl? Some students work really well in the evening but go all wide-eyed and fuzzy-brained when an early morning session is suggested or

6.3 Scheduling Research Sessions

157

required. By now you will have been in the study game for a few years, so you should be pretty familiar with your own body rhythms and when you work at your best. However, in order to find the extra hours you will need for this project, don’t hesitate to experiment with some new time slots you would not normally consider. You don’t necessarily have to have the same time slots every day—they may vary, with a morning session one day and an afternoon or evening session on another, and full days on the weekends. Having identified when you are most productive, it is important to allocate the time on a regular basis. Create a level of guilt if you have not met this time commitment. Remember that it is probably better to have an hour each morning for five consecutive days, rather than one five-hour block (although it is true that some of us like Ray do tend to perform better in large block mode; again, be sensitive to your own rhythms). Just as we block in time for the research, we can also block in time for other significant commitments such as work, family or exercise. Get used to writing regularly while still maintaining your life.

6.3.2

What Additional Considerations Are There for Scheduling Research Sessions?

Consider the following potentially competing demands on your time and strategies for coping with them. • Other work commitments. What time do you have to be at work? Some part-time students have been able to successfully negotiate an extra hour from their employers on one or two days a week as support for their postgraduate studies. One or two hours may not seem like much but if that time is tagged on to a morning work session starting at 5:00 am, three days of that will give you 9 h, which is equivalent to another working day. • Commitments to friends. Saturday may be a no-go area for scheduling in a research session when you have promised your friends or flatmates you will spend time with them. Having said this, characteristically, many postgraduates indicate that during the course of their research their circle of regular friends with whom they spent time narrowed considerably. If they are true friends, don’t worry, they will be there when you resurface after your project and they will be able to adapt to your reduced presence. • Commitments to family. Your partner or family may be more willing to allow you a low-level of family involvement during the week and on Sunday if they know they are going to have your presence completely and unreservedly on say Saturday afternoons. So, in these circumstances, the bulk of your research sessions will be early mornings and evenings through the week and all day Sunday in order to have some time with the family. Leaving Saturday morning for chores.

158

6 How Should I Manage My Time?

• Commitments to children. Many postgraduate students who are also parents have come to the realisation that it is just impossible to undertake any serious intellectual work with the continued distraction of children around you. As a result, the only true research time is when the children are in bed. Dr Emma Kruse at the National University of Samoa indicated that her schedule was to eat an early dinner with the family and to go to bed at the same time as the children, around 7:30 pm, and then to get up at 3:30 am to work. That is pretty gruelling and probably required some good black-out curtains, but it worked for her for some time. Other students with children find evenings are a more productive time. • Can your schedule be sustained? Whatever schedule you devise, make sure your work plan is realistic. You will need to maintain it, so it is important that it is not excessively demanding and that you can still get an adequate level of sleep. The idea of getting up at 4:00 am every morning is, unless you are a super-person, probably not going to be manageable if you also continue to go to bed late— your new work schedule will just never be sustainable. You need to create a realistic research schedule that you can sustain over a period of time. • Block in some down time. Undertaking a postgraduate degree does require some lifestyle changes. However, it is your life and you do still need to enjoy yourself, otherwise the process will quickly lose its attraction. If you are not enjoying the process, then you are not going to make a good researcher, nor are you going to produce a high-quality research outcome. • Negotiate your time frame with partners and colleagues. Engage others in configuring your proposed schedule and periodically re-evaluate with key individuals whether it is working for them. One student I know of had a great schedule that involved specific committed times for her PhD. One of the slots was on Friday evening. After a year, his partner indicated that was one time in the week when she was looking forward to relaxing with him and asked for a re-scheduling of research sessions, so they could spend Friday evening together and Sunday afternoon was re-negotiated as being a research period. • Experiment. Your schedule may need fine-tuning as you settle into balancing the various commitments in your life. Don’t over-commit yourself. If you are trying to wake up at 5:30 am and work while you are exhausted, the session will just not be as beneficial to you as if you were working in a more refreshed state. Try out different research times from your usual study habits. • Stick with it. Remember, it takes five weeks to form a habit, so stick with it. If your schedule seems a little demanding, persevere with it for at least five weeks. • Record your hours worked. Keeping a log in your journal of the hours you are working on your research. The benefit of a log has been called the “merit badge effect”. This is when people have a squirrel-like hoarding instinct, where once they start collecting something, (whether it be Scout badges, bottle caps or hours working on their research) they want more. After you have been working and collecting hours for a few days, you start thinking “I did 10 h yesterday, I wonder if I can get 11 today?” (Peters, 1997, p. 130).

6.3 Scheduling Research Sessions

159

Most people at this stage of a postgraduate research journey have already experienced a number of years of undergraduate studies so are fairly aware of their ideal work patterns. However, things may change with age and with where you are in your research. As you get further into the research process, and particularly your writing stage, you may note that your research sessions comfortably lengthen. One student actually timed their concentration levels at the start of their research and measured them to be only 20 min. At the conclusion of their thesis writing, six years later, they were up to periods of 3 h. Be aware of your concentration span as it will be closely related to your level of productivity. Sitting there staring at a computer screen is not beneficial to you and is a waste of time. Also, don’t be lulled into following someone else’s schedule; everyone is different. The author, John Grisham for example, writes every day from 5:30 am until noon (Zerubavel, 1999, p. 21). That obviously works for him. If you don’t know when you are most productive, experiment with various sessions and record your level of work intensity.

6.4

Breaking Up a Work Session

Voltaire was asked (on his death bed) ‘If God gave you another 24 h, how would you spend it?’ He answered, ‘One day at a time’. What we do with the day and the work sessions within it is, clearly, a key part of time management. Rather than seeing the day or morning as a huge block of time to work your way through at a leisurely pace, it may be more appropriate to break the session up into smaller units of approximately two hours.

6.4.1

Why Break a Session into Smaller Units?

Breaking a session into smaller units is recommended for two reasons: work expansion and boredom. There is an axiom which states ‘work expands to fit the time allotted’. You may have already experienced this. If you have two weeks to prepare a report, then you take two weeks. If you have just five days then surprise, surprise, it actually takes you only five days. We have a natural tendency to fill in the time that is available to us but, by breaking the work session into two-hour chunks we are, in effect, creating mini-deadlines within the session in order to limit the possibility of our work expanding and using all the available time. Some tasks cannot be shortened but they sure can lengthen the time it takes to do them if we allow that to happen. The second reason for using smaller chunks of time in a research session is because of another human factor known as boredom. A mistake many students make is to schedule one task for the entire session and then wonder why they just cannot seem to sustain their concentration and find themselves checking emails on a

160

6 How Should I Manage My Time?

regular basis. They inadvertently switch to another task, usually one that wasn’t on their To Do list and is of a lower priority, spend a few hours doing that, then feel guilty because they have not been working on the task they should have been working on. When a task does not engage us, we get bored with it and our productivity drops considerably. Rather than stick with it, we are better off to flag a change, do something else and go back to it later. To draw an analogy, when we are bored with a task it is like driving a car in first gear rather than full speed—we are just not going that fast. Changing gears (or tasks) enables us to increase the speed, so don’t waste a valuable 8–9 h session going at half speed or, worse still, slowing down to the point where you get out of the car altogether. Switching tasks will help to maintain your interest and keep your productivity levels up. So, you may spend some time on reading literature, then switch to data entry, then switching to writing followed by some time spent planning next week. By switching tasks within a session, you avoid the ‘slowing down’ effect and will keep your energy levels up. You will be the best judge of how long you spend on each task before switching but roughly 45 min per task is something to aim for. For full-time postgraduate students it is, therefore, important that your day or research session is structured and that there is variety, that is, you are not spending eight hours on just one activity such as entering data. Instead, try the strategy of working in intervals you are most comfortable with. For a day-long session, it means properly working for a maximum of two hours on one task then changing to a different task. For example, start by reading and reviewing current articles, after two hours change to another task, perhaps writing, then do two hours of filing material. In this way, you are maintaining momentum throughout the day and not getting bored with one activity. If you are working well on a task (i.e., you are ‘in the zone’) and you have exceeded the two-hour time slot, by all means stick with it and give yourself another hour. At the end of that additional hour, re-evaluate to see whether you are still going at full speed or are slowing down. Become cognisant of your own productivity levels and whether you are flagging. Take a five-minute break (yes, only five minutes) and then come back to a different task. After the subsequent two-hour session, you may go back to the original task, but keep an eye on your concentration levels. The beauty of doing postgraduate research is that there is always a myriad of tasks that you can do. Try to align an activity with the way you are feeling. If you are a little tired in the latter part of your session, that’s when you may wish to do some of the more routine tasks like referencing. If you feel fresher at the start of the day, that’s when to focus on your writing and theorising.

6.5

Taking Breaks

It may seem strange that one of the dimensions of time management is taking breaks; however, stopping can be just as important as starting. When you transition from one task to the other, take a mini-break, get up and stretch. Learn some simple

6.5 Taking Breaks

161

relaxation exercises that you can do standing away from your desk, such as rotating your head, pulling your shoulders up and then dropping them, stretching your legs, doing lunges—stepping forward and bending both your front and then back leg. Rather than engaging in stretching, some students prefer a mental change of pace and do something quite different such as completing a Sudoku or crossword puzzle. Whatever you choose, research has shown that the mental processes used in play and creative acts actually change the chemistry of the brain and bring about more energy (McGee-Cooper & Trammell, 1994, p. 156). Engage in any activity that provides a sense of renewal. Taking a mini-break in the transitions between sessions will help you maintain momentum. If you know that you are rather prone to wandering, don’t go anywhere, just stay near your desk.

6.5.1

But I Like Wandering About, How Can I Deal with That?

Try to restrict yourself so that the only time you can wander is at lunch time. If you have to say pick up an interlibrary loan that has arrived, schedule it for your lunch time. For those students who are wanderers and like to go and talk to others, remember to respect other people’s time. Just because you are bored or easily distracted, doesn’t give you permission to erode someone else’s productive time.

6.5.2

But I Often Feel that I Don’t Deserve a Rest Break and Need to Keep Going!

The opposite of the wanderer is the martyr. They are characterised by the common lament that they do not deserve a break because they haven’t done enough. If this is you, it could, ultimately, be working to your detriment as you may increasingly, and without realising it, become more resentful about sitting at your desk. Acknowledge that you haven’t done enough, and that you are perhaps not up to where you should be. However, it is probably appropriate to get some fresh air and stretch your legs to enable you to be in better shape to tackle the rest of the current task you are working on or another task on the project. Another common psychological response is that of denial ‘I don’t need to take a break because this is far too important, so I am going to press on’. As a word of caution, avoid working when you are extremely tired because, from painful experience, this is when mistakes occur such as the inadvertent deletion of the material you have just written, which can be devastating. You be the judge of your productivity. If you are fully engrossed and happy, work on for a while but, ultimately, you will need to get up and refresh your body and brain.

162

6 How Should I Manage My Time?

If your energy levels are seriously dipping, this can be addressed using a number of strategies. Take a brisk walk and get some oxygen circulating in your blood. Eat or drink something (your glucose levels may have dropped) or do something playful or creative or just something easy and enjoyable, such as checking your email. Whatever strategy you decide on, make sure that you put a time limit on it of, say, 5–10 min. Anything more than that and you are getting into the arena of manufacturing distractions. When you do take these breaks and return to work, just make a mental note of how you are feeling. If you do notice that in the next session your concentration is better, that should reinforce the need to take a break even when you are not wanting to. Whatever the break entails, it is important to do something different from the routine of working. Once again, you may also need to vary what you do in the breaks. To ensure the flow of work once you return from a break, a good suggestion has been provided by Rowarth and Green (2006, p. 114) who recognise that getting back into your work after a break requires forward planning so, each time you stop, note the main task you want to do when you return from your break. This strategy can make a huge difference in terms of time and anxiety about where you have reached and is extremely important for part-time candidates. Similarly, for breaks in your research schedule that occur over night, towards the end of each day make a list of what you have to do tomorrow. This way you are neatly prepped for the next day and can relax because it has been scheduled. There are two schools of thought in regard to how to finish a session. The first is that you should leave something, one last piece that you have yet to do for the next session, i.e., “always park on a downhill slope” (Bolker, 1998, p. 96). Essentially, this means leaving yourself something easy and positive to do when you first pick up again at the next research session. This way you are motivated to pick up quickly and to finish it. The other school of thought is that it is more motivating and satisfying to complete a session with a sense of closure on a particular section, chapter or activity, thereby starting afresh on a new task next time. Take a moment to reflect on which you would find more beneficial, given your own personal working style. You may, in fact, wish to try both approaches to get a feel for which one works better for you.

6.6 6.6.1

Allocating Tasks Within a Session How Should I Organise the Tasks Within a Session?

Don’t just jump into a research session, take a moment to look over the task list for the current week as well as that for the next week (this is usually quite sobering as you realise how much you need to do and that you need to get a wriggle on). Now look at today’s ‘To Do’ list (hopefully prepared the night before). Make any adjustments. Having revised your list (usually adding a couple of items you forgot),

6.6 Allocating Tasks Within a Session

163

indicate which two or three activities are your top priority for the day. This five-minute re-group at the start of the day can be very valuable for keeping you focused and crossing things off a list can also be extremely satisfying. Now, allow yourself a warm-up. It would be foolish for an athlete to head off without an adequate warm-up, and it is not uncommon for writers to go through a start-up process (you may recognise this in your own actions). The warm-up usually has one or two parts to it. The first involves mindless little routines such as tidying your desk or sorting your papers, and the next stage is more muscle-stretching or re-reading the material that you were last working on. As Zerubavel (1999) has appropriately recognised, unlike robots, most people do not usually simply turn on their computer and start right away. As they begin a work session, some go through various elaborate routines such as sharpening their pencils, preparing a cup of coffee, checking their electronic mail and carefully rearranging things on their desk (Zerubavel, 1999, p. 18). These ‘opening ceremonies’ are appropriate but make sure they, too, have a time limit. Intriguingly, the urge for instant gratification usually prompts us to do small tasks or those that we enjoy first, but there is a difference between simply getting started and staying busy, and actively achieving outcomes. Busy work will give you a false sense of accomplishment that you have put in a full day’s work, whereas the reality is that you may not have achieved what is important and a priority. When sorting out what to focus your efforts on, two-time management principles should be mentioned: (1) the Pareto Principle, and (2) The Time Management Quadrant. The Pareto Principle, attributed to the Italian economist, Vilfredo Pareto, indicates that it is actually 20% of your efforts that produce 80% of your results. Consequently, we should focus on that 20%, that is, those activities which are important. Apparently, President Franklin D. Roosevelt had a heart condition that prevented him from working more than two hours a day. He was, nevertheless, an effective leader, probably because he concentrated only on those things that were important. Franklin Roosevelt was clearly effective because he only concentrated on the 20% that was vitally important to perform his Presidential duties. When identifying what is important, the Time Management Quadrant goes one step further. You may be familiar with the quadrants used for time management which look at things in terms of important and urgency. Tasks can be classified according to a 2  2 table (adapted from Race, 2007, p. 41): Urgent and important: 50% Urgent but not important: 15%

Not urgent but important: 30% Not urgent and not important: 5%

The percentages indicate approximately where most people usually and inappropriately allocate their time. • Urgent and important tasks are worth spending time on, however, the aim is to get fewer tasks in this category. • Not urgent but important tasks are worth spending time on and are often a way of avoiding future stresses.

164

6 How Should I Manage My Time?

• Urgent but not important tasks are those that jump out at you and beg to be done but, in reality, are taking you away from your priorities. • Not urgent and not important tasks are simply distractions or diversions that invite diversion from your priorities. There are always going to be some of these and it’s suggested that perhaps the best approach for handling them would be to only devote bits of left over spare time to those that you do feel like doing (adapted from Race, 2007, pp. 41–42). The reality is that often we put the urgent items such as reading emails or checking Facebook ahead of those things that are, in fact, important. Learn not to confuse urgency with importance. If something is urgent, there is usually a bit of stress attached to it. Important is what you should be doing, the things that you have identified in today’s list and, particularly, the items that you have indicated as priorities. If you are not working on them, you are, essentially, wasting time. Urgent items can be pretty compelling and one way to ensure that urgent items don’t crop up is to ensure that they appeared on your work list some time ago. Remember, it is the urgent stuff that makes you stressed. This is not a good space to be in; avoid it by pre-planning. One additional thing to realise is that procrastination is one sure way to create a pile up of urgent tasks—a problem that can quickly become debilitating! In order to give more variety to your work and avoid sudden rushes of deadlines, we suggest that you should intermix lower priority tasks with higher priority ones. That way, some of the long-term items are being covered in addition to short-term and more urgent tasks. You know you are humming when you are dealing with mainly non-urgent but important tasks. This is when you have a real feeling of control. You are less propelled by a crisis, a deadline, something that urgently needs to be done and can, therefore, relax and perform well knowing that the task is not urgent, but that it is important. Postgraduate students commonly seem to face the problem of dealing with pseudo-urgent issues. These include telephone interruptions, meetings, emails, visitors, interruptions from other people, requests for information. All of these seem to come couched with a sense of urgency but, in reality, the urgency is only psychological. We are the ones who elevate them to urgent status. To avoid this, we need to retrain ourselves (and sometimes others) as to what really is urgent and to keep focused on those priorities that have been identified for the day. Pseudo-urgent issues often have us rushing around doing fragmented tasks, however, when scheduling tasks in your sessions try to do one job at a time, and to complete it. To help maintain momentum, also try to devote the most productive part of your day (that is, when you have the most energy) to concentrating on the priorities that you have laid out. Leaving priorities to the low periods of your day is not a good idea. As you will not necessarily have the same research time slots every day, with a morning session one day and an afternoon or evening session on another, some sessions could be contrary to your ideal work mode. That is, where you are primarily a ‘morning person’ but you have to work evenings, or vice versa. So, when

6.6 Allocating Tasks Within a Session

165

scheduling tasks, you may also wish to vary the type of activity. For example, you may find it hard to be creative and write new material late in the evening, but seem to have no problem with reading, reviewing papers, correcting punctuation or just filing. Allocate tasks according to your own personal energy levels. Having said this, you should be trying to push a bit more when you are starting to flag at the end of the day. When you are allocating tasks, try to do the worst tasks first. Get started and repeat to yourself the mantra ‘Do it now’. Finally, a critical question to ask yourself is ‘will doing this activity progress my research?’ The answer to that should give you a good steer as to whether the activity is worthwhile.

6.6.2

My Problem Is Not Working; It Is Knowing When to Stop a Task!

While you work on the most important activities there will be a time when you will need to say ‘stop’ on a particular activity. Have you read and re-read the chapter you have been writing so many times that now you are just re-arranging the full stops? The whole point of being in a postgraduate research program is to learn from the experience, and one of the lessons is to know when to stop and to move on. The general advice is to aim for high standards but not perfection. You just don’t have the time for perfection!

6.6.3

So, When Do I Stop a Task?

Let your project plan and your supervisor(s) guide you here. Assigning time lines and estimating how long you should spend on a task can be tricky. The curious thing about undertaking postgraduate study is that when a student asks ‘how long should I take to develop my research proposal’, the answer is how long is a piece of string? It could take you a concentrated week, or it could take you months. It is, however, your decision as to how long it will take. You could end up with a proposal that has taken a month to write, or a proposal that has taken a week to write. Will the proposal that has taken a month to write be better? Possibly, but not necessarily proportionately better because of the extra time that was allocated to it. The point being made here is that, with a task, we can sometimes wallow in it and take forever. Additional time spent on a task will not necessarily improve its quality and, having spent a lot of extra time, we then end up taking some time away from another activity. You need to have momentum, so place time limits on the achievement of tasks, rather than just meandering. Without time limits, you could just dwell on a task for ages. A deadline is better than no deadline, and it helps you to get more done. Deadlines definitely provide pressure, yet, paradoxically, deadlines can also help

166

6 How Should I Manage My Time?

reduce stress. The last two comments are contradictory but, if you have a project planned out then you know that, ultimately, if you step through all the hoops, it will be achieved. A deadline, or rather achieving one, provides an inner sense of accomplishment. If you find a particular task that can go on for an indefinite amount of time, but you have also recognised that you could get bored with it, place a deadline on that activity for the day. For instance, you can enter data for two hours a day for the next two weeks, rather than just saying ‘I will enter the data until it is finished’. Having discrete time allocations over a defined period of time will keep you focused. Just working away at a task until it is finished presents the possibility of getting involved in further distractions as the task becomes routine, discouraging and not very satisfying, so as mentioned previously, force yourself to stay with the task for say 45 min to a maximum of two hours but then change to another task. Be specific and place a time limit on when you would like to have a particular task achieved. Yes, you may over-run, but it’s better than having an unstructured and open-ended time frame. Be realistic in your time estimates and build in some cushion time or wriggle room for the unexpected. For example, transcribing takes time, as does chasing up missing bibliographic references, and don’t forget Murphy’s Law—everything takes longer than you think.

6.6.4

But How Can I Avoid Getting Held Up, Not by Me, But by Waiting for Others?

If you are waiting for others to complete a task relevant to your research, such as chapter reviews from your supervisor, do not sit around twiddling your thumbs. While your supervisor is looking at your chapters, which could take a few weeks, get on with the other tasks indicated at the relevant stage of your research or in preparation for a future stage. Avoid putting yourself in a situation where you are dependent on others, such as your supervisor or a data source, to complete a day’s work. There should always be something you can work on while you are waiting for data access appointments or feedback from your supervisor. Have a list at hand of other related tasks that you always meant to get around to, but have never had time for, or that you mean to do at some stage in the future. Examples might be: • getting to grips with the more sophisticated features of relevant software; • tidying up your store of references from the literature; • checking on websites that have come to your attention, or have proved helpful in the past; • surfing the web, e.g., Google Scholar, for work related to your topic; • acquiring any important keyboard skills which you don’t already have, such as learning to touch-type using all your fingers; • producing neat and readable diagrams or tables that will help make your interpretations more convincing; and

6.6 Allocating Tasks Within a Session

167

• writing a draft of a chapter for your thesis/dissertation/portfolio (adapted and extended from Cryer, 2006, p. 115) There is also another strategy for coping with unpredictable delays, which is to build in 20% more time then you expect for everything (without getting lulled into a false sense of security) as this will allow you to take a more laid-back approach). Since you can never predict when these hold ups might occur, it is probably better to have some activities in reserve that you can get on with while waiting.

6.7

Establishing a Study Environment

An often-overlooked dimension of time management is the physical environment in which you work. A good work environment is critical to being productive as is the ability to continue working in odd locations in order to snatch valuable time. It appears that space plays an important role in how well we use our time and that if you find that you are not being as productive as you should be, then you need to consider if your regular work space might be part of the problem, e.g., are there too many distractions, is it too noisy, is it uncomfortable, and would a new regular working space help (Succeed in your PhD/MPhil—Tip #2 manage your time and keep to deadlines; https://www2.le.ac.uk/departments/doctoralcollege/training/ eresources/study-guides/starting/time-management, time management tip #7)?

6.7.1

How Do I Go About Creating an Ideal Work Environment?

Closely related to what you do is where you do it. Your ideal work location may, in fact, be one of two or three places. It could be a coffee shop, it could be the couch, or your bed (Rowarth & Green 2006). You should always have a primary spot where your documentation is kept, where you can leave work undisturbed and where you have the access and ability to return to it at any time. This might not always be possible but should be what you are aiming for. Apparently, the Bronte sisters had to clear their writing from the dining table each night, but that didn’t stop them from producing fine literature. When determining your primary work place, consider a secure location where your material can be left. It could either be at home or at the university (we do know of postgraduates who moved into a hotel in the final stage of their PhD, but that can get expensive). If you have rowdy flatmates or young children, you may prefer to work away from home, as the distractions and demands can quickly eat into your research time. Alternatively, you may find you are able to close the door and your home spot is the best.

168

6 How Should I Manage My Time?

For those students who want to work away from home at the university, your options may reside within the department, study space in the graduate school or, possibly, study locations in the library reserved for postgraduate students, which may be in a more secluded position than the general sitting area. Go to the student union and ask what you are officially entitled to regarding space allocation as a postgraduate student. They may or may not be able to answer you, but it should at least give you an indication of whether there is an existing policy or guidelines and where to get advice. So, check with the library, the administrative staff in your department or graduate school, as well as your supervisor, as to whether there are any spaces available for you. Most graduate schools have a study space which enables students to ‘hot desk’. You will not be allowed to leave your material there and other students, as well as the administrative staff, can get irritated if you are occupying a desk but are not physically there for any length of time. In addition to knowing whether there are any locations designated for higher degree research students, administrators and/or your supervisor may know of someone who has recently left which has yielded a spare office space. Don’t settle in too much and decorate it as it is likely that the office may only be available for a short time in anticipation of possibly a new staff member arriving or a visiting professor needing to be accommodated on a temporary basis. If you are camping out in an office, get an indication of how long you might occupy it and try to secure a guarantee of access for at least a semester.

6.7.2

What If I Would Prefer to Work from Home?

If you prefer to work from home, then this is probably the more ideal option rather than carting material around with you. You can put your home office in any location. Where there is no spare room, try the hallway or the garage, just make sure it can be heated in cold weather and cooled in the summer. Make sure that the surface is large enough to accommodate a number of books and papers on it as well as your computer. It does not have to be flash and one student had a door that was mounted with brackets against the wall. This was great, as it was a good length and managed to accommodate a lot of material. The same approach has been taken with a door mounted on sawhorses. Just make sure that it is stable, not too high and that your chair is at the appropriate level.

6.7.3

Make Sure Your Work Station Is Ergonomically Effective for You

You are going to be working at that location for quite some time, so it is worthwhile ensuring that your work station is ergonomically configured for you. There is

6.7 Establishing a Study Environment

169

nothing worse than giving yourself an ache in the back or neck because you are holding your elbows too high at the keyboard or sitting incorrectly. If you are going to a stationery or office warehouse to purchase your office furniture, don’t skimp on the chair. An uncomfortable chair will create health problems in the months or years ahead, and it is just not worth it. Go for the more expensive, but ergonomically adjustable chair. Remember, the better the chair, the longer you might be able to stay working. Some people prefer a standing work station or one that can adapt to standing or sitting postures. Whatever your preference it is worth investing in the right equipment. If you are going to be conducting telephone interviews or on the phone or Skype for long periods of time, it is worthwhile purchasing a lightweight headset with a microphone or, where appropriate, have a stand where you can rest your mobile phone with the speaker on. You will need to have one place where you can store all your material and where the majority of your files can be securely contained. Cabinets can be purchased from second-hand shops and auction rooms, and if they are looking a little bit worse for wear, get a can of spray paint and freshen them up. If you don’t have a lot of floor space, then consider putting shelves up on the wall. If possible, also use some wall space for a whiteboard, your calendar, your template for chapter headings and writing specifications for your thesis/dissertation/portfolio and, possibly, the structure of the chapter you are currently working on. Having these prompts at eye level and close to your computer is ideal. If you are keeping a hand-written research journal, give it an assigned resting place next to your computer as well. While you are setting up your work space, also give some thought to lighting (for both day and night). Usually, people don’t give enough consideration to the lighting in their home office. You will find that you are much more productive when you’ve got the right type of lighting. Use a light bright enough to read your papers easily without eyestrain but be careful to ensure that you don’t get glare or reflections on your screen (both from natural as well as direct light). Also ensure that there is no flickering associated with the lighting you choose as this can be very fatiguing. For equipment, you will typically need a good computer (you don’t want it failing half way through and losing valuable data or drafts), shelf space, and storage boxes. With respect to a computer, a desktop computer with a good-size monitor is easier to ergonomically configure; just ensure that the screen provides clear colour resolution (colours bleeding into each other on poorer quality screen can also be fatiguing). A Microsoft Surface (or Surface laptop) or Apple Macbook would also be quite workable (as well as portable, a distinct plus over a desktop computer), as long as you have a keyboard attached and, preferably a mouse (the tracking pad on these types of devices are not all that good for long-term use). (As an example, Ray wrote his chapters for this book using a Microsoft Surface Pro 3 with keyboard and arc mouse.) It is also well worth getting a printer, as often when you are preparing work, as a change or for proof-reading and checking readability (especially useful for diagrams and graphs, in which case a colour printer would be a better choice), you may want to see the material in print rather than on the screen. It is also worth paying for a good internet connection as you will often be sourcing your library’s database from a distance.

170

6 How Should I Manage My Time?

One final but important aspect of your physical work environment to attend to is to have a good back-up system available. Cloud based storage, offering password-access protection, means you can access from any location and your material is secure; examples include Google Drive, Apple iCloud Drive, Dropbox, Microsoft OneDrive (see https://www.pcmag.com/roundup/306323/the-best-cloudstorage-providers-and-file-syncing-services). It is also useful to diversify your backup strategies. For example, you might, in addition to using, say, Google Drive, also store key working files or maybe your completed documents on an external hard drive, which you connect directly to your computer and, as well, store printed copies of your drafted and/or finished chapters. The moral of the story here is to avoid ‘putting all of your eggs in one basket’.

6.8

Working in Different Locations

For effective time management, there is not only the work that you do in your normal work location, but additional work that can also be accomplished away from your standard work space. Most postgraduate students will be working in a number of locations in addition to their primary study location, and productivity can improve immeasurably if you are able to work effectively in different environments. Developing the skill to work in other locations will ensure that you are able to maintain momentum and may also provide some refreshing variety. A distinction can be made between desk time (at your normal work station) and away from desk time. A further distinction can be made between focused and non-focused time. For example: • desk time—focused: engaging in active thought, writing, theorising, data analysis, and so on. • desk time—not focused: sorting materials, filing, possibly routine data entry, and so on. • away from desk—focused time: data gathering, consultation with supervisors, reading journals/books related to your research (this location would often be in the library), and so on; and • away from desk—not focused: reviewing, proof-reading, making notes, checking footnotes, reflecting, and so on. The point here is that you can be just as productive away from your desk as you can be at it. For example, in a recent conversation with a well-known academic author of biographies of successful women, she indicated that it was her routine to write 500 words a day no matter what the day and where. She wrote on Christmas Day and confided to me that she wrote for some days while in hospital beside her ill father. She had her routine, despite the location, and was able to work in unusual circumstances (she said it helped her to deal with the situation by also being able to continue to focus on her work). If you travel you can still work. For example, two

6.8 Working in Different Locations

171

friends have worked as cabin attendants for Cathay Pacific Airways and both have undertaken postgraduate study. When they were not flying, they were studying in hotels in some of the most glamorous cities in the world. (This chapter is actually being revised in a hotel room in Hanoi, Vietnam.) For effective time management, work with fragments of time and you will be surprised how much it accumulates. Some years ago, I taught every weekend on an MBA program in Macau, across the Pearl River Delta from Hong Kong where I lived. The trip was an hour by jetfoil (no bridge then). I calculated that four teaching weekends in a month, travelling an hour each way, yielded eight hours per month— equivalent to an entire day in the office. With this realisation I decided that, rather than just slump in my seat and watch the water go by, I should use the time more productively. I would, therefore, always make sure I had material with me that was easy to work with on my knee, something like reviewing a paper, proofing a chapter, reflecting on data analysis or drafting another chapter. While that trip took an hour, the same approach applies for when you have the odd 15 min or so. You will be surprised at how it can accumulate into productive time, particularly if you are travelling or waiting for an appointment. So, try to squeeze time from the oddest moments; don’t just sit there having a haircut and chatting away to the hairdresser —proof-read. If you are waiting for the dentist—draft out a chapter outline or read a current research paper. You should always have your research journal or your latest chapter with you to work on, or the latest two papers relevant to your research that you have wanted to read.

6.9

Organisation of Materials

A very simple time management tool but one that is often overlooked is being organised. Disorganisation (e.g., piles of unsorted journal articles, papers and other paraphernalia covering other papers on your desk, stacks of books) is a common reason for students not performing to expectation. They have difficulty finding material because of the mess, thereby wasting a lot of time simply because they did not think to put a system in place. You need a system, so you can remember where you put things and can find them quickly when you need them. There are a variety of different ways and means of organising material. However, it is essential that you initiate a fairly disciplined approach to your material at the onset of your research journey. Having papers strewn all over the floor and a cluttered desk will probably start to slow you down as you get into the thick of your research work. So, set up your filing systems early. For right-brain thinkers, go out and buy covered folders, clips, notepads etc. as you may find them more engaging and memorable. “For me these coloured markers, tablets, and post-it’s are just as essential to my well-being as a balanced breakfast. They provide the momentum to keep me focused and motivated” (McGee-Cooper & Trammell, 1994, p. 93). For left-brain thinkers, keep things simple and linear but keep them tidy, which means you will have to have files, both electronic and physical.

172

6 How Should I Manage My Time?

The word ‘files’ conjures up visions of hunched Dickensian clerks shuffling through endless stacks of dusty paper. Unfortunately, given the degree of organisation demanded by postgraduate study, you’ve got to have organised files. Peters (1997) provides a non-exhaustive list of the types of files that, as a postgraduate student, you should be building up: • material from your applications; • research topic ideas; • research proposal material—drafts, supervisors’ comments, bibliography and so on; • files for your thesis/dissertation/portfolio itself—a file for each chapter and the bibliography; • interview notes, data gathering notes, data gathering protocols and forms (e.g., completed questionnaires, drawings by participants); • background documents (e.g., reports, organisational documents, media stories) for contextualisation purposes; • ideas for papers for publication; • notes from meetings with supervisor(s) and student support groups; • notes from conferences and seminars you have attended; • planning materials; • candidature progress reports and other institutional documents/policies; and • key contacts (adapted and extended from Peters, 1997, p. 134). Also remember that your research journal itself constitutes a very important file to keep up to date and several of the items in the above list could safely be recorded and maintained in that document (recall Chap. 3). Essentially, there are four types of material you need to keep track of: relevant literature, books and materials you have acquired; unsolicited material; your research data; and your actual thesis/ dissertation/portfolio document and related material. In the next section we discuss some suggestions for organising these types of materials.

6.9.1

What Are Some Suggestions for Organising My Literature and Accumulated Materials?

Wherever possible, try to retain copies of literature of material you have located relevant to your research as you will want to make notes. If you want to implement an environmentally responsive approach you will be scanning material and making extensive computer notes. Either way—hard copy, electronic or both—you are going to be doing a lot of filing, so you need to set up your systems early and have them flexible enough to expand as the material accumulates. It has been suggested that you should not have to waste time trying to work with a poor filing system or replacing work that is lost because you never created a back-up copy. If you need it, the IT Services at your university should be able to provide advice to you on

6.9 Organisation of Materials

173

managing electronic files effectively and how to create safe and secure back-up copies of your research and other work. For some other ideas, check out the following website: Succeed in your PhD/MPhil—Tip #2 manage your time and keep to deadlines; https://www2.le.ac.uk/departments/doctoralcollege/training/ eresources/study-guides/starting/time-management. As you start to acquire what, in the course of your studies, will be large amounts of material, develop a system for coding each item. For example, as you read to develop your topic or literature review, if you want to cover a large body of knowledge quickly, your first take on each paper will, in fact, be an initial scan, e.g., through reading the abstracts of journal articles. For hard or electronic copies, provide key words that the material relates to or reflects (some journals, such as those published by Emerald, provide these with each article as a matter of course). This could be for example, a variable or relationship you are examining, a foundational theory, or targeting a relevant chapter in your thesis/dissertation/portfolio. Save literature details in electronic form. This will give you quick recognition of what the paper is about. Start files named with each of these keyword headings and place the material into the keyword files as it is generated. Some students like to enter all their material into the EndNote, Evernote, Zotero or Mendeley bibliographic software system with notes. This is pretty time consuming for papers that may turn out to be only marginally relevant to your research or will not be used at all. If you use such an approach, it may pay to do a first pass sorting of your reference materials into three broad groups, essential/foundational for my research, relevant for a specific aspect of my research (e.g., theory, methodology, research context) and tangentially relevant to my research, and focusing on the first two groups for entry into your referencing software system. Other students prefer to enter items into EndNote when they commence writing, and to use it for articles which they have specifically referred to. This is your call and will be influenced by the volume of literature in your field. As you are writing, when an item is actually included or referenced in your writing, it is imperative that you enter it straight away into your EndNote system or bibliography. It is horrendously time-consuming to track down papers and references later, either because they were omitted, or some aspect of the reference was incorrect (although Google Scholar can be very helpful). Be very careful in detailing all aspects of the referenced paper, including author initials, publication, date of publication, DOI numbers, page numbers (as well as the page numbers for any specific quotes you want to use from the paper) and so on. It is worth investing in some storage boxes that can be labelled and stacked. Labels on the boxes can be changed so that as material starts to overflow you can always start a new box or go back and reclassify the keywords on the labels to make them even more focused. Don’t have material lying around. Have a periodic tidy up and set aside time each week for filing. Make this a regular part of your routine (Lewis & Habeshaw, 1997, p. 64). The naming of computer files can be particularly problematic to postgraduate students and, as a consequence, causes them frustration when trying to locate prior drafts or relevant material. Take a moment to consider how you might name your

174

6 How Should I Manage My Time?

computer files and the directories in which they are stored in such a way that they are meaningful to you. It’s advisable to actually embed the date in the name of the file. Regularly back up all your most recent material. Start a lending list, because if you lend out your papers or books, it’s quite possible you may never see them again (Webb & Ott, 2005, p. 7). Keep a list of where the paper is, who has it and the date that you lent it out.

6.9.2

How Do I Handle Unsolicited Material That Is Sent to Me?

We seem to get a lot of unsolicited email; it is time-consuming to read and can clutter your inbox. The same approach to dealing with hard copy unsolicited correspondence can be applied to unsolicited email. Handle the item only once, that is, read it, decide how to deal with it and take action. If you are going to file it, don’t leave it sitting in your inbox, just as you shouldn’t leave unsolicited mail on the bench in the kitchen or on your office desk. If you don’t want to file it now, place the material into a filing folder and file it in bulk later. However, this creates double handling so wherever possible follow the ‘touch it once’ rule. It has been suggested that to handle hard copy material, you should open unsolicited paper mail over a wastepaper basket, so the material can be skim-read and immediately dropped straight into the bin, if not worth keeping. For email, you can set up a filter to catch mail you do not want. Unsubscribe from as many low-value or irrelevant mailing lists as you can. Be ruthless as you do not need such distractions. You may also want to consider two Google Chrome plugins to assist you: AdBlock (https:// getadblock.com/) that enables you to remove distracting ads and StayFocusd (https://zapier.com/blog/stay-focused-avoid-distractions/#stayfocusd) which restricts the amount of time you can spend on specific websites such as Facebook.

6.9.3

What About Organising My Research Data?

Research data include not only data you have gathered and processed, but also your preliminary and final analysis documentation and contextual information that surrounds your sampling and data gathering efforts. Over the period of a postgraduate research project, these data can become voluminous and unwieldy if not managed properly. The most effective means for storing your research data is electronically on a computer, usually in the form of spreadsheets (incorporated into most statistical packages), written-up transcripts, voice files, document and image files and, where appropriate, multimedia files. As with your literature material, your research data also need to be stored in an appropriate location and organised under key headings (e.g., organised into logical directories on your computer or a logical hard

6.9 Organisation of Materials

175

copy filing system). Additionally, your storage method needs to be secure (e.g., password-protected or kept under lock-and-key). As there may be various iterations of your analyses, embed the date in the actual file name combined with a mnemonic name indicating what each analysis relates to. Think carefully about your file naming system and directory organisation and be consistent in your approach to implementing them. Record the details of your system in your research journal, so you have ready access to them if you can’t remember something. Remember to back up all files regularly as you do not want to be inadvertently losing data or the outcomes from certain analyses. Note that we dig deeper into issues associated with organising your data in Chap. 20. Your methodology chapter should be one of the easiest chapters to write, and yet, many postgraduates struggle with it simply because they have not maintained their research journal effectively and implemented effective data management procedures. For example, they may not be able to recall on what date they interviewed a participant, precisely how many questionnaires they sent out, how they arrived at their final research questions or why they carried out a specific analytical procedure. It can be very difficult to reconstruct such information after the event, particularly if several months pass. Thus, it goes without saying, that you should record all aspects of your data gathering and analysis processes (including how you have prepared your data for storage and analysis) in your research journal in order to have the information at hand when writing up your methodology. Also record any issues experienced that may have impacted on your data gathering, as they may become relevant at a later date when you are interpreting your data or writing up the limitations of your study.

6.9.4

How About the Organisation Around My Thesis/ Dissertation/Portfolio Material?

In preparing your examinable research outcome (i.e., thesis/dissertation/portfolio), don’t leave writing-up to the end but develop the chapters as you go even if they are just partial drafts. Unless you have skills in large document management, you may prefer to approach and store your evolving research outcome, not as one document, but as separate chapters or sections with bibliographies attached to each one. The bibliographies can later be merged into one big reference section. This will also assist the supervisory reviewing process as you can submit smaller component parts of your research outcome for review instead of an every-growing single document. It is critical to be systematic in your naming of component documents. We recommend that your file names always include a version number so that you can keep track of different versions of drafts. We would also recommend that you keep older versions of your drafts (i.e., keep archives). Many students find that their writing process evolves over time so that, occasionally, they find they have deleted something they wrote in an earlier version, then later decide they really did want it

176

6 How Should I Manage My Time?

included. If they retain the older, but clearly labelled versions, then recovering that previous writing would be a simple matter of cutting-and-pasting into the most recent version of the draft. Always ensure that you have backed-up the most recent work, preferably into the cloud but possibly also on an external drive, and have it stored in an alternative location. As mentioned previously, there is nothing more frustrating than getting to the end of writing your research outcome, checking the references, and not being able to locate an item. Don’t let this happen to you! From the beginning of your research journey set up your EndNote system or, if you are a technophobe, a card system or a simple word file, to record relevant details for each of your references. Similarly, when writing, make sure that all references are recorded at the time of writing. Don’t leave it until your next research work session!

6.10

Dealing with Interruptions and Distractions

A critical dimension of time management is being able to handle distractions such as interruptions and time-wasters. When trying to maximise your productive time, you may have put a number of effective measures in place, such as planning and setting deadlines, grabbing odd moments, working in a variety of locations, organising your materials and so on, only to be derailed by time-wasters. We all know what or who our personal time-wasters are and they become time-wasters simply because we are unable to say no, we lack self-discipline, we procrastinate, or we are just plain disorganised. Interruptions can create distractions (e.g., when a partner asks you to do a chore); equally, distractions can create interruptions (e.g., you see an email come in and stop to read and answer it). When looking specifically at interruptions and distractions, Bolker (1998, p. 180) makes a useful distinction between interruptions from the outside and interruptions from the inside. Interruptions from the outside are, essentially, events that happen to you, while interruptions from inside are those time-wasters that you initiate. Distractions can also be categorised in two ways. Those that cannot be ignored and need to be dealt with, for example, a computer breakdown, a sick partner or child, an emergent commitment at work, as well as the more severe circumstances of a significant loss or bereavement. The second category, and one that you may need to work on, comprises those distractions that can, with training and self-discipline, be controlled and/or deferred until a more appropriate time. These usually come in the form of social media, personal emails, text messages, telephone calls, internet surfing or just wandering off to look in the fridge. As one doctoral student observed about getting distracted “When you begin your day by reading messages, you are prioritizing other people’s requests before taking action towards your long-term goals. Responding to emails can distract you from your priorities and interfere with your concentration for the rest of the day” (https://finishyourthesis.com/time-management/).

6.10

Dealing with Interruptions and Distractions

177

6.10.1 How Do I Deal with People Coming into My Work Space and Interrupting Me? If you have a desk in a working office, dealing with interruptions can be awkward, particularly when colleagues knock on the door and are using you to entertain them during their breaks. There are three approaches to handing these. The first is a direct approach where you indicate that you are pretty busy at the moment, trying to get something finished, and that you will pop by later (thereby interrupting their productive time). Alternatively, you can be non-responsive, and not make eye contact. A gentle shake of your head can dissuade people from further communication; however, the downside is that you do come across as being rude and uncaring. The third and less confrontational way to handle an interruption is to excuse yourself, pick up a piece of paper and walk out of the room as if you are going to photocopy or drop it off somewhere. A quick walk around the block or trip to the loo is enough to break the interaction, thus enabling you to return safely to the solemnity of your work location. Of course, prevention of interruptions is far more preferable than dealing with them after the fact. One very useful preventative strategy would be to hang a sign on your closed office door, which says “Please do not disturb—in a writing frenzy!” You could even post office hours to indicate to potential drop-ins when you are available and able to be interrupted. Complete avoidance is also a viable strategy: Many students avoid interruptions by working predominantly from home or in a secluded spot in the library. During the day, while sitting at our desk, we are bombarded with what could be called insignificant urgencies. They are compelling and distracting and need to be managed if we are to manage our time effectively. Handling the distractions of calls, emails and text messages is hard and takes discipline on our part. The truth is that we generally welcome the interruption and convince ourselves that the interruption is important and, consequently, automatically drop what we are doing and respond. In doing so, we break our concentration, disrupt the flow of work and spend 5– 10 min on an activity which is not core business or our focus the day. The first step to managing these types of interruptions is to minimise the distraction. Turn your mobile phone off and only check it as a reward when you have finished a two-hour study session. Turn the beep off on your incoming email box. Just because the phone rings doesn’t mean to say you have to answer it. The next step is to have some self-discipline around responding. Interruptions of this sort can be very seductive as we think, ‘Great, someone needs me’, but each interruption is only serving to waste time and take you away from the task at hand. One suggestion for handling emails is that while you are working on your research, allocate only two or three opportunities in a day when you will receive and respond to emails or social media notifications.

178

6 How Should I Manage My Time?

6.10.2 How Do I Deal with Distracting Myself from My Work? As we noted above, interruptions from inside are, essentially, those things we do to ourselves. Bolker (1998, p. 180) refers to interruptions from inside as ‘static’ or unrelated thoughts, feelings or distractions that pass through your mind while you are writing or trying to write. Essentially, this is mental debris. Making up your shopping list may be useful, but it is a job for the end of the day and doing it in a work session is merely distracting. Panicking yourself is another example of an interruption from inside. If you are an anxious writer, then reciting negative thoughts about how bad you are and how important something is will not help and takes up mental space, time and effort. By way of advice, you don’t have to get unanxious first, you just need to learn how to work despite your anxiety (Bolker, 1998). Keep your sessions short and the pressure on and cut yourself short when you start to indulge in any negative rhetoric with yourself. Interruptions from inside can be both mental and physical but are always initiated by us. Many physical interruptions are self-induced. Wellington et al. (2005, p. 44) recalled one student who said that whenever she looked at the heavens for inspiration, she noticed that the windows needed cleaning! That may be true but once you have set aside your designated research time, you need to remind yourself firmly that you do not have to clean the windows, water the plants, rearrange the bookshelf, put a load of washing in or engage in any other task-avoiding strategy you can think of. The reality is that it is probably a lot more rewarding and fun to pick up a new project, the phone, a book, check out the internet or talk to someone who happens to be passing by, than to work on the task at hand, which can be tedious and boring. Recognise that interactions are not just external but internal, and that you could be promoting those interruptions yourself. Take some time to reflect on when and where, and how frequently, you interrupt yourself and how these interruptions might be managed. If, for example, you are an avid texter, perhaps limit your texting, both receipt and return, to one or two times in the day as a reward for when you have completed a research-related activity. For unnecessary interruptions, one suggestion I have heard of is to put a rubber band around your wrist and twang it every time you catch yourself going for a mental walkabout. The very task of twanging the rubber band along with the increasing redness of your wrist will signal the number of times you are distracted by ‘static’ and are not focused on the job at hand. We have just been discussing interruptions that we initiate during a research session but sometimes these time-wasters and distractions can kick in even before we get to the desk. For example, another distraction for postgraduate students can be taking on other commitments. Some students are new to academia and, as a doctoral student; you are in a more privileged place than an undergraduate or master’s student. You are also a useful resource within the department and will probably be called upon to assist with the activities of the department and senior academic staff. Activities such as the organisation of conferences, seminars and

6.10

Dealing with Interruptions and Distractions

179

events, teaching support and so on are good opportunities to establish relationships, build skills and create networks, although, be protective of your time in order to ensure that the distraction does not start to eat into your primary task. Don’t give away your time by taking on extra community assignments—a major building/ renovation project or another job. You have an excellent excuse—you are currently involved with your postgraduate research journey! In closing this section, it is appropriate to mention that, for some students, the number one time-wasters are watching television, internet surfing and watching DVDs. It may not be your vice but, if it is, consider restricting your usage if you want more productive time in your schedule. Take a moment to make a list of the inside and outside distractions that most commonly impact on you and rob you of time and consider some strategies for minimising these time-stealers. What things are time-wasters for you? A colleague recently noted that any time someone mentioned going for a coffee, he seemed to be eager to go. He wasn’t sure whether it was the coffee or the social interaction he needed, but the outcome was inevitable —loss of time in being away from his desk. Having recognised this as a time-waster, he then tried to limit himself to one coffee out per week, and only as a reward after he had done a concerted few hours of work.

6.11

Conclusion

Undertaking a postgraduate research journey is a big endeavour and can quickly morph into a very time-hungry activity. As a consequence, there is an ever-present need to find slots of time to devote to your research and ensuring you are producing outcomes directly focused on your goal. Regrettably, getting your best work out of these time slots can be frustrated by poor time management manifested in a variety of behaviours, such as not planning and scheduling, not prioritising, not finishing tasks, working in a poor environment, being disorganised, getting distracted, procrastinating, not saying ‘no’ to additional activities, lack of self-discipline, and even too much socialising. The keys to successful time management have long ago been identified as active implementation of strategies around activities, such as identifying key results to be achieved, planning and prioritising, setting specific outcomes for work sessions, staying focused, having a good work environment and organised work methods, as well as controlling distractions. Reading about these strategies makes them sound easy but putting them into practice is often more difficult and requires tailoring of actions that best suit each individual. Intriguingly, most postgraduate students talk about the importance of ‘managing time’ when, in fact, the true heading should be ‘managing oneself’, because in most circumstances, we are the chief architects of our own time-abuse. Try to identify the personal traits you think prevent you working effectively. For example, you might be: • A dawdler who just works through things without any plan or direction; • an impatient person who needs to do everything immediately, even if it means working excessive hours; • a perfectionist who misses deadlines to perfect your work;

180

6 How Should I Manage My Time?

• a procrastinator who puts off tasks and lets them pile up until they are really urgent; • a slow warmer who takes ages to get into a task; • a butterfly who flits from one task to another but doesn’t really finish anything; • an optimist who sets unrealistic expectations and deadlines and is constantly disappointed when outcomes are not achieved; • an over-committed person who takes on too much, not saying ‘no’; • a distracted person who is easily seduced by other demands or interests; • an interrupter who walks about and interrupts others; • a disorganised person who is messy and surrounded by mountains of paper and a lengthy inbox; or • a staller who waits for a good block of time before getting down to work (adapted from Webb & Ott, 2005, p. 3). Tick which ones you think relate to you. You could be a combination of the above, but they will give you some insight into potential areas to work on. Admittedly, time management is very much for left-brain thinkers. If you would like to read a book on time management for the right-brain thinker, try McGee-Cooper and Trammell (1994), Time Management for Unmanageable People. Another text which you could find useful is Time for Research: Time Management for Academics, Researchers And Research Students by Kearns and Gardiner (2013). You may have devised another time management system that is more appropriate to you and that you have been working with effectively for some time; if so, that’s great, stick with it. Your intention should be to identify areas for improvement, experiment with a variety of strategies to address the problems and then start adapting and implementing, on a regular basis, the action plan that works best for you. Consider making up a personal list of at least 5 actions that you might implement, such as: • • • • • • • •

limiting your social media interaction to lunch time and evening; discouraging drop-ins; handling each email just the one time; doing something once: hit the delete button, store it or file it; don’t say ‘I will get back to it’; processing emails in concentrated bursts about 3 times only in your work day; doing something in your ‘empty’ time, for example, while waiting for the dentist, outline your next chapter or sketch out some implications for your research; cease watching so much television; and/or unsubscribing from unsolicited or low-value email lists.

The benefits of time management are real and as you become more productive, the quality of your work will improve. You can avoid falling behind or leaving everything until the last minute but, more importantly, you will feel more confident and enjoy what you are studying without feeling guilty. As a consequence, you will probably be able to sustain high level of activity over many years and, on the odd occasion when you actually have some leisure time; you will actually enjoy it more!

6.12

6.12

Key Recommendations

181

Key Recommendations

• Develop a study schedule. Remember, this is your ideal schedule, not necessarily one that will work out all the time. Unexpected events do and will occur. Don’t feel bad about them; just make sure they don’t become a regular occurrence. • Understand the pattern of working that works best for you and capitalise on your body rhythms. Some people are morning people, some are night owls. Match your work efforts to your patterns. • Do some work on your postgraduate study every day and set a goal for that day, even if it is tidying up punctuation in a section you have written. • The longer the gaps between research sessions, the less productive the session will be as you will spend valuable time coming back up to speed and reacquainting yourself with where you left off. For most people, frequent short bursts of focused activity (constantly chewing away) are generally better than more widely spaced concentrated sessions (binging). • Be wary of commitments that can take you away from your work. For a period of time, you may have to downgrade or postpone commitments. Practise saying, ‘Unfortunately, given my work load, I don’t think I can handle that at the moment’, or ‘I have a deadline coming up which I can’t miss’. • If you have a whole day to do something, it is important to ensure that your day is structured and that there is variety in what you do. Change to a different task every few hours on. This way, you are maintaining momentum throughout the day and not becoming bored with one continuous activity. • Identify and prioritise your tasks. Use your research journal for writing down your To-Do lists. One of the reasons why you might feel overwhelmed is because you are trying to keep a lot of ideas fresh in your head. Take a moment to stop and write in your research journal any thoughts that have been occupying you but, more importantly, lists of things you need to do. • To encourage a sense of achievement, try to accomplish at least one important task every day. • Stay organised. Have a place for everything, and everything in its place. Place the most frequently used files and items close to hand. Related items should be grouped together. Keep de-cluttering. • Identify your own ‘research avoidance strategies’ and try to overcome them. Keeping a log for a week of 15-minute intervals will give you a good indication of how much time you actually waste. • Manage interruptions, don’t let them manage you. Minimise distractions during your focal work time—no, it is not rude to close your door, you most certainly should turn off your mobile, and it is OK to decide that you will call someone back later. • Don’t mistake effort for accomplishment. Remember Pareto’s 80/20 rule—you should only be working on the things that are important and are instrumental to the completion of your thesis/dissertation/portfolio and avoid tangential activities.

182

6 How Should I Manage My Time?

• Have a stack of small tasks you can do when your concentration is flagging but it is too early to finish your research session. • To create more time, learn to work in odd places, i.e., make the most of your travelling or waiting time.

References Bolker, J. (1998). Writing your dissertation in fifteen minutes a day: A guide to starting, revising and finishing your doctoral thesis. New York: Holt Paperbacks. Cryer, P. (2006). The research student’s guide to success (3rd ed.). Maidenhead: Open University Press. Farkas, D. (2018). 10 Surprising time management strategies to help you graduate. Retrieved December 21, 2018, from https://finishyourthesis.com/time-management/. Katz, R. (2009). Shorten the time to doctorate. A guide to managing your PhD project. Bloomington: AuthorHouse. Kearns, H., & Gardiner, M. (2013). Time for research: Time management for academics, researchers and research students (2nd ed.). Adelaide: Thinkwell. Lewis, V., & Habeshaw, S. (1997). 53 Interesting ways to supervise student projects, dissertations and theses. Bristol: Technical & Educational Services Ltd. McGee-Cooper, A., & Trammell, D. (1994). Time management for unmanageable people: the guilt-free way to organize, energize, and maximize your life. New York: Bantam. Peters, R. (1997). Getting what you came for: The smart student’s guide to earning a Master’s or a PhD (Rev ed.). New York: Noonday Press. Race, P. (2007). How to get a good degree: making the most of your time at university (2nd ed.). New York: Open University Press. Remember The Milk. (2018). The smart to-do app for busy people. Retrieved December 21, 2018, from https://www.rememberthemilk.com/. Rowarth, J., & Green, P. (2006). Sustaining inspiration and motivation. In C. Denholm & T. Evans (Eds.), Doctorates down under: Keys to successful doctoral study in Australia and New Zealand (pp. 112–120). Camberwell: ACER Press. Succeed in Your PhD/MPhil—Tip #2 Manage Your Time and Keep to Deadlines. (2018). Retrieved December 21, 2018, from https://www2.le.ac.uk/departments/doctoralcollege/ training/eresources/study-guides/starting/time-management. Webb, S., & Ott, B. (2005). Effective organisation and time management. In K. L. Allen, J. Bamber, & M. Flower (Eds.), Study skills: A student survival guide (pp. 3–18). Hoboken: Wiley. Wellington, J., Bathmaker, A. M., Hunt, C., McCulloch, G., & Sikes, P. (2005). Succeeding with your doctorate. London: Sage Publications. Wunderlist. (2018). Keep your life in sync. Retrieved December 21, 2018, from https://www. wunderlist.com/. Zerubavel, E. (1999). The clockwork muse: a practical guide to writing theses, dissertations, and books. Cambridge: Harvard University Press.

Chapter 7

How Do I Stay on Track?

7.1

Staying the Course

Staying the course and completing your postgraduate qualification is truly a wonderful experience. As one student commented, “On a beautiful sunny day, with clear Carolina blue skies, I turned in the final, signed copy of my dissertation. Graduate School staff member did some last-minute checks on the document and pronounced it acceptable. After 6½ years of toil and sweat, I was finally done! While walking back to the CS Department Building, I was sorely disappointed that the heavens didn’t part, with trumpet-playing angels descending to announce this monumental occasion” (Azuma, 2017). Even with anticipated jubilation of completion, all too frequently you can lose sight of your end goal and can get stuck in the process, making limited or no progress. The notion of the ‘wilderness years’ has been used to refer to those low points in one’s research experience, when you feel you are wandering around in confusion and getting nowhere (Wellington, Bathmaker, Hunt, McCulloch, & Sikes, 2005, p. 34). Problems with your research, self-doubt and flagging energy are quite normal; nevertheless, they can be debilitating (Cryer, 2006, p. 212) and can completely derail a postgraduate research student. However, if you anticipate that this can and will happen, you can be better prepared and can mobilise the necessary internal and external resources to deal with the issues before they become too problematic and cause you to stall or completely abandon your studies.

7.1.1

Do Most People Actually Finish?

Doctoral programs have high dropout rates with only 41% of students successfully completing within 7 years (Ampaw & Jaeger, 2012). In the US, universities expect approximately anywhere from 40 to 60% of their students to obtain their doctoral © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_7

183

184

7 How Do I Stay on Track?

degree (Spaulding & Rockin-Szapkiw, 2012). The reasons for non-completion are, commonly, that students find academic research is not what they expected and/or they were overwhelmed by the combination of course work, research and teaching expectations required of a PhD student (Noland, Francisco, & Sinclair, 2007, p. 67). After an examination of PhD submissions in Social Sciences in the UK, investigators reported that within a three-year period, only 4% had completed and, within five years, only 25% had completed (Salmon, 1992, p. 2). There is variability in completion rates depending on the discipline with science experiencing an approximate 76% completion rate (Jiranek, 2010) and education experiencing a lower completion rate of approximately 50% (Spaulding & Rockin-Szapkiw, 2012). Concerns about poor completion rates in doctoral education have tended to focus on improving the supervisory support mechanism but are now recognising that this focus is too narrow (Fenge, 2012). In Australasia in recent years, completions have increased and in general, reflect funding incentives that governments now provide for doctoral completions, which have shortened the completion times. Interestingly, completion rates are generally similar for males and females but higher for full time and younger students. The increase in younger student retention could be attributed to better pastoral care and selection of doctoral students, while the decrease in retention of older, part-time students is possibly due to high rates of employment, interfering life events and more rigorous monitoring of the progress of doctoral studies.

7.1.2

What Commonly Gets in the Way and Inhibits Completion?

In a review of prior studies of British PhDs who had either completed or were approaching completion of their doctoral studies, it has been noted that the most common difficulties experienced were in relation to coping with isolation, establishing an appropriate relationship with a supervisor and specific problems with the substantive elements of the research (Pole, 2000, p. 100). Interviewees were, however, quick to indicate that they saw their doctoral education as largely a positive experience, and the less positive memories “fade in the light of the overall achievement of the doctorate” (Pole, 2000, p. 100). Race (2007) also provides a good list of what could be described as ‘inhibitors to successful outcomes’. These are lack of confidence, not getting enough feedback (or not being receptive to feedback), through to not understanding expectations, not being able to make sense of the topic, not seeing the relevance of the topic, and not being able to master the skills necessary for the task (for example, data analysis). It has been noted that most often students undertaking empirical work experience difficulties in staying on track, either becoming paralysed when faced with literature or their data, or becoming so engrossed with the techno-colourings of analysis, that they lose track of time and drown (Delamont, Atkinson, & Parry, 2000, p. 110).

7.1 Staying the Course

185

Students often forget that they do not have to use all of their data or all of the literature they have read.

7.2

Strategies to Stay on Track

Advice can come from a number of sources or papers on the strategies for successfully completing a postgraduate research program (see, for example, Conn et al., 2014; Nettles & Millet, 2006). From our experience we have developed 12 strategies, discussed below, for you to reflect on if you wish to stay on track and complete your studies but are either currently wondering around in the wilderness or want to avoid getting stuck there.

7.2.1

Use Your Planning Tools and Deadlines

As most people have not previously embarked on a project that takes many years, they may not be aware that, at the beginning, there is usually a false sense of security and a belief that they have heaps of time. However, this is a fallacy and the sooner you come to grips with the fact that time marches on very quickly and that, if you are not careful, before you know it three months will have passed, you are behind and becoming severely disillusioned. In order to avoid, this engage in active and ongoing project planning, work out what you have to do, create timelines and set deadlines for yourself. With a project of this size, students are often inexperienced in the appropriate allocation of time. Initially, use the stages provided in The Doctoral Research Planning Guide in Chap. 5, and attempt to attach to each step an indication of how long you think that step will take. In order to estimate more accurately the time needed, it is useful to discuss with others who are working with similar research approaches and methodologies to your own. Ask how much time it took for the various components of their research. In addition, your supervisor(s) may help to identify likely timelines from which you can derive target dates for completion of each phase. From experience, there are areas where students often underestimate the time needed. Predictably, these are in the data clean-up stage and the writing clean-up stage; that is, where data have been miscoded and need to be checked, and when the research outcome is in its final writing stage and needs to be thoroughly proofed. If your research involves the gathering and analysis of qualitative data, another area that postgraduates tends to underestimate is how long it takes to prepare and analyse qualitative data. Having assessed the timelines involved with various activities you should now be able to set some specific targets or milestones. Make sure each milestone is evidence-based; that is, state what you should have demonstrably achieved at the completion of that step, for example, completed the first draft of the literature

186

7 How Do I Stay on Track?

review. Staying on track involves being aware of when these deadlines are due, by regularly reminding yourself, and trying to stick to those deadlines. Create an emotional attachment to the milestone—the irritating thing about a postgraduate research degree is that the targets are very arbitrary unless you instil some relevance in them. Having a date in a diary or on a wall planner is one thing, but actually having it as a significant date that drives you to put in the extra time, is what really matters. If that date is important and it would disappoint or embarrass you if you missed it, then you know that your emotional attachment to that date is meaningful. If it is a date you could go past without worrying, you are lacking the emotional attachment necessary to ensure appropriate motivation and attainment of that target by its deadline. Make the deadlines realistic and use the milestones to motivate yourself but be careful you don’t make things too stressful. For example, if you plan to have a chapter of your thesis or portfolio delivered to your supervisor the day you are departing for an overseas trip, it may just be too much given the other demands upon your time. A few days before departure would be a more appropriate deadline. As always you also need to be realistic about how much time you have available for study, as designing a regular work schedule may require a certain period of experimentation and adjustment. You may discover, for example, that, although you originally intended to devote 70 h a week to your research, it just isn’t doable, or the 6 h per night that you allowed for sleeping leaves you somewhat exhausted and prevents you from effectively keeping up with such a demanding work schedule for more than two days in a row (Zerubavel, 1999, p. 15). If you get behind, stop and reflect on why. Was the milestone unrealistic? Was your estimation of the time required inappropriate? Did you not keep to your routine and allocated study sessions? Did you get side-tracked on other tasks or requests? Did you lack necessary information or skills to achieve the task? Did you panic? Did you find the activity too tedious to pursue? Did you fill up your study time with tasks that were not really a priority? Did you feel stranded and need help, perhaps you couldn’t make up your mind, i.e., were there alternative courses of action you couldn’t decide upon where you should have sought supervisory advice? Or, did you just need to keep redoing something until it was perfect. Be careful of these pitfalls and try practising the following: • regularly remind yourself what your deadlines are and when they are approaching; • don’t create unrealistic deadlines that you know you cannot keep to; • try to be a little more disciplined in sticking to your study schedule; • catch yourself if you are feeling a little overwhelmed and/or incapable, and seek help; • don’t give up too easily when you hit a problem; • try not to over-analyse your data; • avoid the urge to get everything absolutely perfect; • don’t get side-tracked on other projects; and • don’t let the demands of other commitments override you.

7.2 Strategies to Stay on Track

187

If you miss a deadline, you need to be brutally honest with yourself as to why. Not achieving a deadline may be masking a personal frustration or obstacle that needs to be worked through with some assistance. Alternatively, you may need to build more leeway into your project plan. When you read the Table of Contents of a thesis or portfolio, it all looks so terribly ordered and well-thought out. The reality is that it is a reflection of an entire research journey at the end of the process where many decisions have been made along the way and problems solved. It doesn’t by any means reflect the variety of difficulties encountered or decisions that were made in undertaking the project. Not everything will go according to plan, so build extra time into your plan which will give you some safety periods of possibly a week each quarter, or two days per month in order to do a catch up. It is also important to keep your plan up to date. When you get stuck, the best antidote for lack of progress is, perversely, progress. If you are doing something (hopefully, every day) on your study, and especially, writing every day, achieving something will provide a sense of progress. This sense of progression, even if it is small, will tend to enthuse and motivate you. If your progress has, however, stalled or stopped completely, the feelings of guilt and anxiety for some curious reason stop you from returning to the project. The end point will not change but the allocation of time within each stage may be modified as you set new, more specific and accurate deadlines as you progress through the work. So, use your planning tools and deadlines. Bite the bullet, reorganise and acknowledge you are behind, take a deep breath and set new targets, allocate your work sessions and get back into it. The sense of relief after a few days that you are back on track will be enormous, so use the anticipation of that relief and the bliss of being back as motivators for getting going again.

7.2.2

Deal with Procrastination

In postgraduate studies, a vicious cycle can occur where your fervent desire to do well on a project can lead to procrastination, which can then lead to falling short of achieving the desired results. This then creates a sense of failure, which leads to low self-esteem and, as a consequence, even further procrastination. As you spiral down, you become less productive although the occasional spurt of activity may, deceivingly, just convince you that you are still on track. Procrastination is very self-directed and appears to come in two forms: not getting around to doing what you should be doing and/or doing other activities instead of the one that you should be doing. In not doing what you should be doing you are, essentially, putting things off, making excuses, postponing tasks etc. Throughout their PhD, a student we knew had a prominent sign on their desk which read, ‘It’s not the doing that scares me, it’s just the getting around to it’. So true! Getting around to doing something can be scary and procrastination can often be like quicksand, sucking you down as you increasingly become more nervous and anxious about a looming deadline and

188

7 How Do I Stay on Track?

feel guiltier for not getting onto it. However, once you get into the task and make significant strides toward completion, the feeling of concern diminishes and is often replaced with relief. When you are actively doing something towards achieving your goal and working on the task, it is more than likely to be less of a panic (one would hope!) and more of an enjoyable experience. The ‘doing’ part can actually be quite pleasant, but the lead up (if you are feeling disappointed with yourself and your progress) can be most unpleasant. When doing other activities instead of the one that you should be doing, you are deferring a high priority task to concentrate on ones of lower priority and, thus, never actually moving on to what you should be doing. Lewis and Habeshaw (1997) have suggested that people procrastinate for several reasons: • The need for perfection—by constantly striving for perfection we create delays. Clearly, we would always like to do things to the best of our ability, but sometimes we are not sure where that mark of ‘best’ actually is. It is probably appropriate, therefore, to do your ‘best at that moment’ and then polish things up later. • Boredom—some tasks are just not interesting or engaging and, as a consequence, we put them off. Human nature will always drive us towards activities that are more rewarding, interesting or seem like they would take only a little time to complete. • Hostility—there may be some resentment associated with the task or activity that creates, in the back of your mind, a resistance to doing it. The way to tackle this is just to get over it. For example, you may have lost your last 2 h of writing work through your own stupidity and must retype the last two pages. Acknowledge that you made a mistake and get on with it. • Deadline ‘high’—a deadline ‘high’ is the adrenaline rush you get when you are working under pressure towards a looming deadline. You may enjoy it, but it is not good for you and, one suspects, probably not good for those around you. • Deadline fear—the sheer enormity or importance of a deadline can sometimes paralyse you into doing nothing. That’s where you need to steel yourself and just get started. Remember, it’s not the doing that scares you; it’s just the getting around to it. Procrastination, as an umbrella term, can come in many forms and you will be familiar with symptoms such as time wasting, vacillation, displacement, fragmentation, and ‘busy work’. Naturally there is considerable overlap in the types of activities that occur in relation to these symptoms. There are as many forms of procrastination as there are students, so feel free to add your own personal favourites, but keep in mind that indulging in them, unless it is for the genuine purpose of relaxation, will be counter-productive to your studies. Some common procrastination activities are indulging in simple time-wasting such as watching TV, surfing the internet, reading irrelevant materials, socialising too much. Taking these descriptors of procrastination further, vacillation has been described as “changing one’s mind, postponing or putting off unnecessarily,

7.2 Strategies to Stay on Track

189

indecision, the inability to come to a conclusion” (Mackenzie, 1989, p. 69), whereas displacement activity is when you are not working on your studies but are doing something else. Petre and Rugg (2010) have a wonderful list of displacement activities which one can do instead of your usual study session. These include: • sorting out your old clothing; • getting rid of junk; or • tidying and filing. Fragmentation is a slightly different form of procrastination. Have you ever experienced the following? You’ve sat at your workspace and fired up the computer to polish up a piece of work. While the computer is starting up, you begin to file some papers that are on the desk. A particular paper captures your interest, so you start reading it. As you prefer reading in a comfy chair, you move to another location, on the way considering the possibility that a cup of coffee might be appropriate. You divert to the kitchen where you start to make some coffee. While preparing the coffee, you engage in a conversation with someone who reminds you of a telephone call you should have made. Feeling a tad guilty that you have used up a chunk of your two-hour work session, you go back to your workstation, but, rather than working on polishing up the piece of writing, you read but don’t answer a few emails … and so it goes on. This is fragmentation and, unlike procrastination where you know you have a task to do but are putting it off, here you are somewhat unwittingly being led by alternative activities and, in doing so, are not achieving closure on any specific task. There is a trail littered with incomplete activities and, and after a morning of this, you wonder why you have not completed anything and been robbed of that sense of accomplishment. Picking up and putting down activities, or fragmentation, is more problematic for those who work from home, as the number of potential divergent activities (laundry, cleaning, interesting magazines etc.) is even larger. Busy work is another area that you need to watch. Busy work is working on your research but not necessarily productively. We delude ourselves that we are working on our research, but the reality is that the work is merely intended to keep us busy and occupied and create a false sense of achievement. Busy work can include organising and labelling your folders, photocopying papers, cleaning up the data set. While busy work can provide light relief and a sort of mental down time enabling us to gestate ideas, you have to be careful when it goes too far and dominates your work sessions. There is a distinction between real work and busy work. Real work is actually quite productive; it pushes you along the path towards completion of your journey. Busy work allows you to respond to questions about how many hours you have put in. The reality is that while you have clocked into your work session, you have hardly been productive at all, you have just been engaged in activities which are tantamount to spinning your wheels, and there was no traction at all. Be honest with yourself—has this busy work gone on for a bit too long? Is it a veiled form of procrastination? If the answer is yes, then you need to get back on track.

190

7 How Do I Stay on Track?

A situation which is sometimes confused with procrastination is, in fact, lack of concentration. You may have experienced it, for example, when you find yourself reading the same paragraph three times and still not actually digesting it. If you are experiencing this, stop and consider what you are having difficulty with. Is it because you are actually doing the wrong task? Is there another task which is, in fact, more relevant and needed and is playing on your conscience? Is there something which is unresolved in your personal life that you need to redress, or are you just physically or physiologically tired? By reflecting on the possible causes of your lack of concentration, you will be in a better position to remedy them. Possibly, you need a brief (yes, brief) break. While procrastination can dissipate your creative energy and significantly reduce your writing time (Thody, 2006, p. 61), keep in mind that there can be a positive side to procrastination—some procrastination is our intuition telling us that this task may not actually be worth doing and we should drop it (McGee-Cooper & Trammell, 1994, p. 140). So what strategies could you use to deal with procrastination? Reflect on why you are procrastinating. Take a moment to think about the factors that are impeding you getting on with the job and try and unpack them. For example, you could possibly be procrastinating because of the level of importance attached to the task, you don’t want to screw it up, it’s really important that you get it right, or there is a lot of your own ego involved in it. As McGee-Cooper and Trammell (1994, p. 138) observed, “By being curious about the times when I don’t stick to my plan and exploring the reasons, I am improving my ability to stay on track”. Manage your fear—it is often fear of failure or of making a mistake that causes us to procrastinate. To alleviate this, Mackenzie (1989, p. 68) has suggested: • Accept that there is a risk involved and that decisions are required. Delaying a decision will impede your progress but making the decision is necessary if you are to get to the next stage of your research. • Reaffirm your priorities—why you are doing this? What tasks need to be done? • Schedule difficult or unpleasant tasks and set deadlines for their completion. If necessary, tell others of your deadlines to give you additional pressure. • Change your attitude regarding potential mistakes; treat them as learning experiences. Try to think of a range of likely consequences. • Set reasonable standards rather than reaching for unobtainable perfection. • Control interruptions and resist the temptation to drop tasks without having completed them. Use negative emotion positively—procrastination can make you feel awful when you reflect on what you have not done and what still needs to be done. Use this emotion to your advantage. What would you rather feel—this current feeling of disappointment, or the feeling of accomplishment? The antidote is quite clear. If you wish to shift emotions, you need to get cracking. Once you are aware of your emotional responses, contrast the awful feeling with the potentially good feeling. Know what you need to do

7.2 Strategies to Stay on Track

191

to get the good feeling, and then get started. As aptly stated, “Daily progress is the best antidote for procrastination” (Peters, 1997, p. 127). Try controlled displacement—if you sit at your desk and find that you just cannot get into your work, find another task which is work related and give yourself a time limit of how long you are going to work on that task, say, for example, you are going to do some filing for an hour. The time limit is extremely important because anything longer than that can end up with you procrastinating. Now do the task that you have been putting off, but once again give yourself a time limit, say only an hour. It may be that you actually engage in it and work longer but knowing that you only have to do it for an hour before switching back to another task, perhaps filing again, will create a tight time frame that you will see as being manageable. Keep switching backwards and forwards so that the onerous task is only allocated short intervals that appear manageable. Activities in small doses can be a valuable mental relaxation (Thody, 2006, p. 61). Procrastination kicks in when the mental breaks are extended into more significant distractions. Deal with indecision—some students have no problem sticking to their work session. The difficulty is in making some big decisions in relation to their research, knowing that once that decision has been made, they are committing themselves to a path that, while not impossible, would take some time to back track. For those suffering indecision, Mackenzie (1989, p. 70) has suggested the following considerations: • • • • •

What must be decided? What are the objectives or conditions that must be fulfilled by the decision? What are the viable alternatives? What is the critical information on each alternative? What are the potential negative consequences of each alternative, and what is the likelihood and seriousness of each consequence? • What is the best alternative? • What steps must be taken to implement the decision? Naturally it is always useful to seek advice. That is what your supervisor(s) are for. They have a lot more experience in the research game and can often effectively guide you over an indecisive hump. Manage fragmentation—for those who fall prey to fragmentation, here are some suggestions for overcoming it and evoking a little discipline: • First, recognise that you are a person who tends to get easily distracted, and acknowledge the fact. • Consider what it would be like to complete, at least by morning tea, a specific task—just visualise what it would be like to have that job behind you, and to feel the sense of accomplishment. • Be very clear what it is you wish to accomplish within the next time frame. State it out loud, “Before lunch I would like to …” (complete the sentence). This way you are giving the task a priority.

192

7 How Do I Stay on Track?

• Start the job in front of you and, just when you get the urge to do something else, catch yourself and pull yourself back to the first task. • If you find that you are all over the place with your activities in a day, try writing down your time-slots, i.e., what time you started real work and what time you finished, add up the gaps at the end of the day and you will recognise how much time you have wasted. Only record time when you are on task and record it in intervals of 10–15 min. • Get into the practice of handling less significant documents only once, particularly mail and emails. Don’t just read and say to yourself, “l will get back to that”. Deal with it and get closure. Work out what is important—procrastination often involves doing low priority tasks. These are tasks that are neither urgent nor important, and you appear compelled to do them rather than getting on and doing the high priority tasks. High priority tasks are those that are both urgent and important. High priority tasks should be done first but, for some reason, you have a mental barrier about doing them. What you are looking for is to engage in instrumental behaviour, that is, behaviour which moves you towards your goals (Petre & Rugg, 2010). Order your tasks—when looking at causes of procrastination; one is personality, i.e., a disposition towards procrastinating, and the other is overload. By now you will know enough about yourself to recognise if you have a personality that commonly procrastinates. If you do tend to have such a personality trait, you need to be continuously on the lookout for when your procrastination pattern emerges and take active steps to work around it. In the circumstance of overload, you just tend to be overwhelmed with the number of tasks that need to be done and, as a consequence, do nothing. In order to overcome this, be realistic and know that you will not be able to achieve all the tasks but, perhaps by knocking a couple off the list, you will create a sense of accomplishment, in turn, raising your self-esteem and creating even further momentum. So, if you are prone to doing other things, start with an easy task first, but give yourself a time limit for that. The next task should be the hardest task of the day. Don’t leave the hardest task until the final stage of your work session or day. Get the difficult job under your belt first and gain a sense of accomplishment. Also, work on your time, not others’, for example, if you receive a telephone call that you think will entail a lengthy conversation when you are right in the middle of achieving your tasks, place your task as being more important by taking the caller’s number and promising to phone them back later in the day.

7.2.3

Avoid Becoming Too Isolated

Coping with isolation is one of the most common difficulties experienced by postgraduate students (Pole, 2000, p. 100). One important lens for understanding a postgraduate student’s decision to persist or depart from a degree program is the

7.2 Strategies to Stay on Track

193

level of socialisation the student engages in (Gardner, 2008). Hockey (1994, p. 179) indicated one of the initial problems that new PhD students experience is the recognition that postgraduate education is not a collective activity and is characterised by large elements of solitude. This isolation can affect individuals in a number of ways and can potentially cause them to go off track. The isolation can come in two forms: intellectual isolation and social isolation. With regard to intellectual isolation, students can sometimes become so focused on their work and the specifics of their study that they are unable to share the complexities with others. Characteristically, as postgraduate research is a unique piece of work, other than your supervisor(s) usually no-one else is fully acquainted with your work and it is often difficult to discuss your journey with others. Other students enrolled in your department appear to be on their own journey and immersed in their own study and may also be unwilling or unable to converse. In addition to intellectual isolation, there is also social isolation, postgraduate study does not have the same structure as students have previously experienced at lower levels of study. There are usually fewer if any classes to attend on a regular basis and you will find you are spending hours studying on your own. The lack of structure also means that the traditional deadlines of assignments and examinations are no longer there as markers or motivators. When students have moved geographic location in order to study, the sense of isolation can be even more acute. International students, who must cope with a new cultural environment in addition to the demands of postgraduate study, can experience considerable social isolation. Social isolation may also be present for a domestic student, particularly one who is studying some distance away from their university. Such a sense of isolation can be profound and can result in self-doubt. As one student explained, “I was due to do my PhD upgrade panel and it was nerve-wracking because I had no-one to measure myself against. My supervisor said I was ready, and I trusted her, but if I had been able to be more present (at the university) I would have been better networked with other students. I would then have been able to make that sort of judgement on my own behalf” (Gatrell, 2007, p. 19). To avoid becoming too isolated, the strategies are fairly predictable and relate to identifying support networks, which can be in the form of fellow students, friends, members of interest groups (church or sport) and even neighbours. By interacting with students who have enrolled at the same time, you can get an indication of how they are progressing and the ups and downs that they are experiencing. Even if your projects are in very different topic areas or disciplines, there will be value in networking with other students simply so you share experiences and maybe talk about tough issues. Not only can student networks be supportive, they can also be motivating. Talking with someone who is further on in their postgraduate research journey can act as an impetus for you to pick up your pace. As one student has indicated, “I didn’t think that I was a competitive person but, when I sat down for a coffee with some of the other doctoral students and heard about where they were up to, it would give me a bit of spur on”.

194

7 How Do I Stay on Track?

If you are feeling isolated, consider the possibility of serving on a committee. One of the advantages is that it will help you integrate into the department and become more comfortable and collegial with the professors, as well as understanding how academic procedures and policies work (Peters, 1997, p. 148) but be careful that you don’t over-commit yourself with additional tasks. To avoid isolation, also consider non-academic interest or sports groups and even having regular chats with a neighbour. An international PhD student we knew could not afford to join a sports club and spent most of her time beavering away in her flat. She found it pretty lonely but every Friday evening she would go and have a drink with the neighbour in the flat below. It was infrequent but enough to break the cycle of work and alleviate her isolation. If you are a postgraduate student studying by distance education, the feelings of isolation can become pretty severe. An important strategy for coping with this isolation is to take every opportunity to visit the department, your supervisor and on-campus colleagues, if it is within your means to do so. If you cannot manage a physical visit, regular Skype or phone chats with your supervisor or another student can help to break down the feelings of isolation. One of the most powerful influences on your staying in the program is your supervisor (Barnes, 2010) as well as other students so consider how you can respectfully engage in order to avoid becoming isolated. Technology can create additional opportunities for you to reduce your feelings of both intellectual and social isolation. Engage often in email communication; get into a chat group, create a blog use social media (wisely). See if your supervisor or department can arrange to set up an electronic discussion board (via Blackboard or Moodle, for example) for postgraduate research students where regular interchanges and posting of resources can occur. It would be important for your supervisors to regularly access this discussion board as well; it can keep the intellectual fires burning and help you to maintain a sense of connectedness. One final suggestion is to investigate becoming a paid research assistant in the department. Not only will it help financially and with your socialisation but it has also been found that students with research assistantships have the highest likelihood of degree completion (Ampaw & Jaeger, 2012).

7.2.4

Don’t Hide but Engage with Your Supervisor

We dealt with managing supervisory relationships in more detail in Chap. 4. However, it is appropriate to emphasise that in order to stay on track, your engagement with your supervisor(s) is a critical factor, so it is worth making a few additional comments and recommendations: • Meet regularly with your supervisor(s)—even when there is little to report. Plan out a meeting schedule and try to keep them to a regular schedule. If you have two or more supervisors, try to meet with them all together. You need to conscientiously maintain communication with your supervisor(s) as this can not

7.2 Strategies to Stay on Track

•

•

• •

•

•

•

195

only help maintain your motivation but also tells them you are involved with the project and want to keep their involvement up to date (and it will help them feel needed as well). Arrive on time, be prepared, and listen. In the meeting, try not to be defensive, explore opportunities and alternative perspectives, be open to suggestions, agree on a To Do list, agree on the date of the next meeting, and don’t overstay your welcome. If you are working from a distance, regular email or phone/Skype contact is essential, for both you and your supervisor(s). Agree on deadlines—having a deadline that has been agreed with your supervisor creates the impetus needed to complete the work. Knowing that you need to front up to someone with your drafted chapter or with your theoretical framework developed will certainly put the pressure on you to achieve it by a certain time line. Choose a suitable place to meet—it’s best to meet in the supervisor’s office or in a departmental meeting room. Avoid loud and busy areas such as cafeterias. If you are a working from distance, check into the possibility of using Skype or other type of web conferencing technology (e.g., Adobe Connect, GoToMeeting, Apple FaceTime) to facilitate meetings with your supervisor(s). Don’t pester—try to avoid contacting your supervisor(s) unless absolutely necessary between your scheduled meeting times, they are busy people too. Don’t put off the meeting times—once you have started doing that it will just stretch out your program. If you have encountered a problem and, for some reason, have not been able to complete the To Do list agreed at the last meeting, still attend the meeting and explain your reasons. Don’t hide problems—your supervisor may be able to help with solving a methodological issue or simply provide you with the encouragement and motivation you require. Remember that experienced supervisors will have seen it all before, so will likely be able to offer sound advice on many of the problems postgraduate researchers encounter. Draw upon that experience. Schedule your work—when you leave a meeting with your supervisor, mark the date of the next meeting in your calendar, and then go back 4–5 days and make another mark on your calendar for when you want to ensure you have delivered any promised materials to them for review and discussion. This way your supervisor(s) will have enough time to review your material prior to your meeting. Do not arrive with material in hand or send it to them the night before —they will not have time to read it. If you are going to get maximum value from the interactions with your supervisor(s), give them time to consider their feedback carefully by submitting your materials early. Bring a hard copy of the material with you to the meeting—you may, in fact, wish to bring two copies in case your ‘absent-minded professor’ has mislaid theirs (they shouldn’t have, but it does happen). By now, you will have been into more than one office of an academic and will realise that they usually have 5 or 6 projects on the go at once—and it can all get rather messy. There could also be an accumulation of 20 or 30 years of prior learning material, which can quickly bury your recently submitted chapter if your supervisor is unorganised.

196

7 How Do I Stay on Track?

• At the start of the meeting provide a verbal introduction—although it may sound patronising, at the start of the meeting you may wish to take the lead and provide a quick summary to review the action points determined at the last meeting and summarise the material you have provided. This will also provide memory prompts for the supervisor(s) as they will have participated in number of different meetings, including with other students they supervise since your last meeting with them, which can interfere with their memory of what you discussed in your last meeting. You can then turn the discussion over to your supervisor(s). • Make notes in your research journal as the supervisor is speaking—not only will this keep the focus on the supervisor, and keep you from talking too much, it also demonstrates a degree of diligence and will certainly help with your recall what you need to do after the meeting. • Check with your supervisor whether they anticipate being away or on sabbatical or study leave for any length of time—a six-month leave of absence in Europe could significantly impact on your research progress and you might need to discuss alternative arrangements or strategies for communication for that period of time. Never leave one supervisory meeting without having the date set for the next one, with agreed outcomes to be completed by that next meeting. • Prepare just before the meeting—in addition to the work that you have done on your research, prepare specifically for the meeting. Make sure you jot down the questions you want to ask or issues you want to discuss during the meeting. As a rule of thumb, your preparation time for the meeting should be about as long as the meeting itself. • Go into the meeting with a clear objective of what it is you want to get out of the meeting—it maybe the supervisor has still not read a draft chapter and you want to speed things up; if so, put it on your list to discuss. Your intention is to get a date for receipt of the reviewed chapter. • Think of ways you can help your supervisor—supervisors are frequently under considerable pressure with numerous other commitments so keep an eye out for ways you may be able to provide some temporary assistance. For example, meeting a visiting guest lecturer at the airport could be an easy task for you but could save your supervisor two hours. Similarly, working on a literature search with your supervisor could help you both. • Be appreciative—without overdoing it, periodically mention your appreciation for the time and effort that your supervisor is putting into your studies, indicating that you value their expertise and input. • Be open to suggestions—if your supervisor proposes a different approach, don’t resist. Listen and engage in the process. If you still disagree, propose that you will go away and think about it. Away from the meeting, give it due consideration and compile your alternative suggestions and/or reasoned arguments. Some postgraduates are simply not aware of what a resource a good supervisor can be. As one student acknowledged, “Although my relationship with my supervisor was never bad, it was only when I was writing my thesis that I really

7.2 Strategies to Stay on Track

197

started to receive regular support from him. He helped me with the structure, encouraged me to set regular deadlines and meticulously referred to the drafts. Most importantly, he stopped me when it was good enough. Looking back, I wish I had asked for his input more often whilst I was doing the research itself”. So, create the relationship early, establish expectations and maintain regular contact.

7.2.5

Face Problems Head-On

Problems can come in many forms that relate to not knowing what to do, or perhaps knowing what to do but not how to do it. Any of these situations can paralyse your progress. Not knowing what to do, or not knowing how to do something is all about a lack of information, which creates an uncomfortable feeling of uncertainty, but nonetheless can be remedied. In these circumstances, you need to obtain some advice either from your supervisor(s) or, more often, from someone with specific expertise in the area. The beauty of working at a university is that you have a wealth of knowledge, all within a two-kilometre radius, that you can draw upon, so take the initiative and ask around about who may be able to help. More problematic is when you hit a big problem and, as the Kenny Rogers song goes, you have to “know when to hold them; know when to fold them”. If something is not working out, rather than blindly pursuing a course of action, stand back from the issue. It might be better to drop something now and spend a month or so retracing your steps, than to abandon it altogether six months later. A gaping hole in the literature could be there for a reason. For instance, it may be that getting access to participants to answer a particular question is nigh impossible; if that is the case, toss that line of enquiry in your research project. Facing a problem head-on is easier said than done when you are approaching what looks like a complete dead-end or a significant problem with your research. Nevertheless, be reassured that many a postgraduate research project has been resurrected from a false turn. What you need, however, is some creative and lateral thinking as to how the material you have generated to date can be worked into a productive outcome. This may possibly require going back and reframing your questions and re-scoping your literature, but it doesn’t necessarily mean that you have to abandon the project outright. There is usually always a way out—it may require some backtracking and some additional time, but once you face up to the fact that with some extra effort you will be able to complete, this should re-energise you. If you are in this situation, regular communication with your supervisor(s) becomes even more imperative. What if your problem does not relate directly to your research? Problematic issues that may confront a student are temporary difficulties with oneself or one’s family that require considerable expenditure of time, for example, a sick family member. In circumstances like these, you can look at suspending your candidature or registration for a period of time, and it is advisable to do so in order not to have to cope with too many things at once. Once again, speak to your supervisor(s) and

198

7 How Do I Stay on Track?

seek advice. Also, listen to your own intuition. If you feel that you are unable to cope with the additional demands you are now facing, as well as undertaking your studies, make the decision that is best for you. However, do not throw all your work out the window. A temporary suspension, formally approved by your institution, may enable you to clear the current issue and resume work after a period of time, without penalising you for the lost time in your candidature.

7.2.6

Be Persistent

Close on the heels of the discussion on facing problems head-on in order to stay on track, is the need for persistence and avoiding what has been termed ‘learned helplessness’. It has been observed that good work habits and perseverance usually separate those who manage to complete their research outcomes from those who do not (Zerubavel, 1999, p. 10). You will recall that, in the discussion of skills needed to complete postgraduate studies in Chap. 2, persistence was mentioned as a relevant skill. This is, in reality, “stick-ability”, so even when you feel like abandoning the project you manage to draw on some inner resources to keep you going and help you overcome some of the frustrations and problems you are experiencing. This inner resilience is necessary as often the problems you are experiencing are not just intellectual but also emotional. Studies reveal that student persistence is seldom the result of one factor; goal commitment, motivation, advisor support, academic and social integration, support of family and friends and use of student support services have all been commonly mentioned (Ivankova & Stick, 2007; Spaulding & Rockin-Szapkiw, 2012). Persistence is considered to be a key skill because it is frequently called upon when undertaking research of an extended nature. This is, however, not about doggedly adhering to your intentions when things get tedious or discouraging, it is also about being flexible enough to create new options or opportunities in order to keep the momentum going on your project. In contrast to persistence is learned helplessness. Postgraduate students are particularly prone to this feeling, and usually go through at least one phase of feeling that they are getting nowhere and that there is no point in keeping going. Petre and Rugg (2010) suggested that if you are taking ages to get nowhere it could be for a variety of reasons, the first being that you don’t have the faintest idea what you are doing and where you are going (hence, having an agreed topic and a project plan in place will be particularly helpful). The second reason is that you are in the middle of your journey and are in a natural lull, the second-year blues are a fairly normal part of doing doctoral research, for example. If learned helplessness and the feeling of getting nowhere is an accurate description of how you are feeling then pull yourself together long enough to either eat some chocolate, acquire a self-help book (e.g., Feel the Fear and Do It Anyway), have a good talk to yourself and set yourself some manageable goals. Talk things through with someone who can give

7.2 Strategies to Stay on Track

199

you supportive advice and get some exercise away from your usual haunts to acquire a sense of perspective. Some students notice that after a year or so they need to vary their work habits and change their schedule as they have tired of the sheer routine of it. As one postgraduate student commented, she was doing well working in the evenings as she had a full-time job. But after two years, she hit a wall and it just wasn’t working for her. She felt despondent and unmotivated, so she took a break from study, reflected on the situation and came up with a revised schedule in order to inject some change. To avoiding the mid-doctorate doldrums, the easiest way is to reflect, take a deep breath, make some changes (either your work place or work schedule), set new markers, and get some momentum going in your studies.

7.2.7

Start Writing Early on and Keep Writing

The first PhD to be awarded in the United States was at Yale University in 1861 for a thesis written in Latin which was only six pages long (Marshall & Green, 2007, pp. 3–4). Clearly, PhDs as well as professional doctorates and research masters have become a lot larger over the years and their research outcomes require much more than six pages. There appears to be a great deal of consistency in the recommendation that one should start writing early, and continue as you go through the research process, rather than leaving it until the end. The best way to get into a good writing rhythm is to write every day. You can define ‘every day’ as you please, seven days a week or only on weekends, or at least five days out of seven, as long as you define what you intend to do in advance, and don’t keep changing the rules as you go along (Bolker, 1998, p. 38). The continuous recording of your experiences, reading and observations will, thereby, create the first draft on which further revision can be undertaken and the final document more easily prepared. The only problem with this suggestion is that many students, despite many years of study, still find writing difficult. Apparently, most people would rather wash the bathroom floor than write (Bolker, 1998, p. 38). Couple this with the perceived importance attached to each chapter, and they can become virtually catatonic in their attempts at writing. Zerubavel described PhD students as long-distance runners, cross-country bikers and mountain climbers, and as writers they traverse a long road paved with serious doubts, wondering whether they will ever be able to reach their final goal. The actual distance from this goal, which, like the top of the mountain that lies behind a seemingly impenetrable screen of clouds they can only vaguely envision, understandably generates a paralysing amount of anxiety. Intimidated by the enormity of the task that lies ahead of them, many writers indeed break down somewhere along that ominous road and never complete their projects (Zerubavel, 1999, p. 37).

This phenomenon is not new, as Lord Byron the poet lamented, “I spent all morning putting a comma in, and the afternoon taking it out again.” While most students are drafting material as they go, the bulk of their writing will occur after

200

7 How Do I Stay on Track?

the data have been collected and analysed. As a consequence, difficulties with writing appear to be most problematic mid-way through postgraduate studies. The process of writing actually involves three stages: the planning, the writing and the tightening. In the planning stage, the structure for each chapter or section is developed. • Provide a rough outline of the chapter—it is pretty easy to ascertain that there will be an introduction, main points and a conclusion—it’s the middle part that gets tricky. Nevertheless, find the likely headings you will be using; something you can negotiate with your supervisor(s). For example, if you are writing up the theoretical foundations, what are the main theoretical domains pertaining to your research? They then become the subheadings. Once you have the subheadings, they will provide an appropriate structure into which you can begin to insert information. • Choose a method that works for you—some students like to draft out a chapter with subheadings by hand, others directly onto the computer, while others, who are more visual, use mind maps, a whiteboard or even Post-It notes on a wall. The Post-It notes, for example, signify the key components, thoughts or dimensions of a chapter. They then rearrange them until they feel they make sense visually before writing them down. • Consider layout—when using subheadings, probably no more than three levels of headings are appropriate. “Writers and readers begin to flounder when they get past the sub-sub level” (Wellington et al., 2005, p. 147). Also, try not to have a full page of block prose; it is somewhat daunting to the reader. Breaking it up into paragraphs is much more digestible psychologically. • Don’t be surprised if you keep rearranging text—you will frequently face the dilemma of “should I put this section here or there?” This is an issue relating to structure and is easily resolved by cutting and pasting. Always remain open to moving blocks of text around because, as you read what you write, you often find that this bit of text belongs there rather than here in order to make your point clear. • Chew, don’t binge—if you are waiting for a big block of time when you plan to write, it’s probably not going to happen or, if you are waiting for all other tasks to be completed before you get started, that’s also not going to happen. Writing in small bite-sized pieces, i.e., chewing, is often the better approach to getting the job done. The writing stage deals with the actual construction of wording, and initially, as has been recommended many times before, just getting the material down is more important than finely-crafted wording. As the saying goes, the reason people get writer’s block is because they have nothing to say. Sometimes, the need to create perfect prose can be a significant inhibitor. As a consequence, it is best to give yourself permission to lower your writing standards just to record your thoughts, and to reflect on what you are trying to say, and then just say it. One strategy for getting things started is simply to list ideas and things you know you will want to

7.2 Strategies to Stay on Track

201

say, separated by semi-colons in a document. Don’t fret about complete sentences, just use a few words to signal each idea; later you can flesh things out into complete sentences and paragraphs (this is one of Ray’s strategies for his own writing!). Get the words down and don’t go back to refine and polish until you have exhausted what you are trying to say. When you have got it all out of your system, you may then wish to reflect on the material and provide more structure. You can always go back and condense two points into one or split one point into two. The idea is just to start recording. To get the ball rolling, some people find it is easier to speak about what they are trying to say which can be recorded on a smartphone, then reorganise it onto a page. Postgraduate research is generally reflective, analytical, interpretive and theoretical and therefore often needs to be written down to find its direction. It forms on the page (not necessarily in your head) as you write. • Start writing early in your studies—do not wait until all your data are into start writing; write as the data analysis starts yielding stories. You can always write pieces of your literature review and your methodology as you go along, which can make pulling things together at the end a bit easier. • Don’t be influenced by others—being overly concerned about what others are expecting of you can create unrealistic expectations which you may feel you cannot fulfil. • Do not censor yourself—separate the writing process from the editing process. The writing is the creative side, editing is largely a critical/mechanical part. Get it down, in the first instance. If you feel it is just a draft and doesn’t have to be perfect, sometimes that will make the writing easier. Reflect and revise later; get the material out of your head, and into some form you can then review and revise. • Record your random thoughts—as you are writing in one area, thoughts arrive that relate to another aspect of your research. Go to that chapter and record your thoughts while they are fresh in your mind, but don’t linger there for too long, go back to where you were. You could use your journal to record such thoughts as well. • You don’t need to work sequentially—create headings and then insert material under each of the headings as it is generated. Once you have material in each area, you can go back and re-draft it in order to make it flow and be more readable. When you are happy with each section you can go back again and look at the chapter in its entirety, and re-draft yet again for greater coherence between the sections of the chapter. No-one says you must start at the beginning. • Chapters can be written out of order—you don’t have to write chapters in the order they will appear in the final document. Some students, for example, find it easier and more comfortable to start by drafting the methodology section(s) of their thesis or portfolio, as this is largely describing what they have done and why and provides an easy entry to the writing process. To ensure the thesis/ dissertation/portfolio ties together, the introduction chapter is commonly completed last.

202

7 How Do I Stay on Track?

• It is easier to go from larger to smaller, than from smaller to larger—insert all the material as it is generated; it is easier to cut out or condense than to squeeze additional material in. • Work out what is best for you—some experienced academics draft out their papers by hand before typing them up. For people who write long-hand and then type it up, the typing up process becomes the first editing stage. Delamont et al. (2000) recalled a student who confessed to only being able to work on scraps of paper, such as the back of committee papers, previous drafts by others, spoilt sheets from the Xerox machine etc., as facing a clean piece of paper paralysed her. Fortunately, she had worked out what was good for her. • Write to your audience(s)—have your potential readers seated in front of you (figuratively speaking) as you type. • Use your digital thesaurus—many times you will need to re-phrase things, and the use of alternative words or descriptors can certainly assist in providing variety and livening up the material for readers. • Keep to the maxim—‘two shorter sentences are better than one long one’. • Avoid being a perfectionist—you could continue to polish up a chapter, conference paper or journal paper but you will reach a point of diminishing returns, where additional polishing will not benefit the original work and may, in some circumstances, actually erode the quality of the work. You will realise when this has occurred when you are changing sentences that you have previously changed at least twice. Postgraduate research is, in reality, the beginning not the culmination of your career—don’t worry about making it your magnum opus. Get it out sooner, rather than later (Azuma, 2017). • Seek help if you get writer’s block—when writing, some students experience what is commonly called writer’s block. Writer’s block tends to inflict itself on individuals with extremely (possibly even delusional) high standards who refuse to seek help. So, don’t just sit there, go and talk to someone about it and get help. When dealing with writer’s block, Thody (2006) specifically recommends: – – – – – – –

don’t panic more than once a week!; reward yourself by completing your daily writing goals; change to another of your writing projects if one is proving intractable; set a time limit for relaxation activities, just as you do for writing; don’t expect perfection; reflect on your writing while taking breaks; and when you stop writing, make notes of your plans for the next sentences; recommencing is then less daunting.

The third stage of writing involves reading over and rephrasing, clarifying tightening and editing the material you have written. Finn (2005, p. 105) points out that writing a thesis does, in fact, involve three dimensions: the research content to do with issues such as originality; the actual research questions, framework, methodology, interpretation etc.; and the structure which involves the appropriate location of the different sections of the thesis, as well as sequencing and style,

7.2 Strategies to Stay on Track

203

which is a matter of writing with conciseness. Clarity and adherence to the rules of grammar only come about through the editing process. Good cooks recommend that, when you take meat out of the oven, you should let it stand as it will improve the flavour and texture. There is definitely an analogy here with writing and developing a good style. When you go back to material that has been standing for a while, you are often in a more objective frame of mind and able to improve on the flavour and structure of what you have written. The latter stage, where the material is tightened, is often more relaxing than the actual writing. Writers, therefore, often alternate between the two stages of creating and editing. • Re-drafting is just as important as the writing—regrettably, you do not have the luxury of raving on, given the word count constraints that exist with a thesis, dissertation or portfolio. The time you spend on re-drafting will ultimately be beneficial because each new draft will tighten the material, thereby providing opportunities to take out excess words. • The introductory chapter gets written at least twice—introductory chapters are virtually always rewritten at the end of the research process. When you go back to re-write the introduction, you will actually be reflecting more accurately on the final out-turn of the research outcome. In this way, the introduction will be a more appropriate prelude to, contextualisation and foreshadowing of the presentation of the research and the related discussions to come. Further reasons to re-write the introduction are to update relevant key literature that may have arisen during the course of your study, and to beef up the ‘original contribution’ element. Having concluded your study, you will have a better understanding of where the original contribution exists within your study. For some students it may, in fact, be in the methodology, or the approach taken in the study, rather than in the findings. As the original contribution is one of the principal elements on which your thesis/dissertation/portfolio is being evaluated, it is good to have it well-stated in the introduction. • Check for mechanical errors—in addition, check for grammar, punctuation, spelling and typographical errors, and ensure all references are included and internet sources cited. • Editing will involve a number of individuals—you, your supervisor(s), one or more members of the profession (if you are doing a professional doctorate) and, possibly, a professional editor at the final pre-submission point. However, the most important person to rely upon is yourself. The material you produce should be to the best of your ability. Number or use different coloured paper to differentiate the various draft versions. • Ensure consistency of headings and styles—adhere to the institutional requirements of presentation and write to the accepted presentation style required by your institution. These include the layout, size of margins, headings, subheadings, line-spacing, font style, point size and referencing (Brown, McDowell, & Race, 1995). • Keep to the word limit—the word limit will be prescribed by the regulations covering your qualification. Doctoral theses or portfolios, at least in the social and

204

7 How Do I Stay on Track?

behavioural sciences, typically do not exceed 100,000 words, or 250 pages, in total. Length doesn’t always correlate with quality, for example, the Lord’s Prayer is only 56 words long, Lincoln’s Gettysburg Address is 268 words long, the Declaration of Independence has 1,322 words, and the US Federal Government’s Cabbage Code, which regulates the sale of cabbages, has 26,911 words (complaint of a US Congressman, contained in Lewis & Habeshaw, 1997). • Don’t underestimate how long it takes to polish a thesis, dissertation or portfolio—this is one stage where students are often surprised at how long it takes. Our general suggestion is to write early in the research process, write often, and write in draft (it’s less scary). This latter recommendation has been summarised as “don’t get it right, get it written” (Delamont et al., 2000). To achieve this, Delamont et al. (2000) suggest: • • • • • • • • •

The more you write, the easier it gets. If you write every day, it becomes a habit. Tiny bits of writing add up to a lot of writing. Break your writing up into small bits. The longer you leave it unwritten, the worse the task becomes. Until it’s on paper or in electronic form, no-one can help you get it right. Draft, show the draft to people, re-draft. Writing is a vital stage of clarifying your thoughts. Start writing the bit that is clearest in your head, not necessarily the introduction. Drafting reveals the places where it isn’t right (yet) in ways that nothing else does (Delamont et al., 2000, p. 121).

To top off this section, Petre and Rugg (2010) have also provided some good tips for research students. They advise: • Write, write, write—the more you write, the easier it gets; write as you go and don’t throw anything away; revising is often easier than writing new. • Keep an annotated bibliography—it should be a personal tool for you. • Form an informal committee—a small set of reliable people willing to read and critique your material. • Expose your work—make your work public at research seminars and conferences, thus exposing your work to questions and criticism; this will also help you to network with interested individuals. • Learn to ask other questions—learn to go beyond your initial questions in order to expose other perspectives and look for alternative explanations. • Read at least one completed thesis, dissertation or portfolio cover to cover— reading something that has already successfully passed through the examination process will give you good insights. • A postgraduate degree, at the point of submission of your thesis, dissertation or portfolio, is a pass/fail proposition—part of the process is learning when enough is enough to produce an outcome that has a very high likelihood of passing (Petre & Rugg, 2010)

7.2 Strategies to Stay on Track

205

For an extensive provision of handouts and resource materials on writing, see http://writingcenter.unc.edu/tips-and-tools/.

7.2.8

Be Conscious of Security

Losing material or data can be a sure way of losing traction in your research. We have all heard of those horrendous cases in the old days when manuscripts were typed on manual typewriters and the only copy of the completed manuscript was either left in a duffle bag at a train station or on the backseat of a car that was stolen. Fortunately, the perils of a single typed copy are no longer the case. Today, with word processing, we are able to save materials to a hard drive or cloud drive with enormous storage capacity, or onto an external drive which, unlike the old floppy disks, can store an entire thesis, dissertation or portfolio as well as all of its relevant data, images and diagrams, electronic copies of literature and analysis outcomes. Despite such ease of storage, caution should still be exercised with respect to keeping all your materials in one place. A colleague was recently devastated by the loss of her nearly completed PhD. She had been saving the material on her computer and also had been dutifully backing the material up on an external hard drive. The only problem was that both the computer and the hard drive were in the same location. When her office was broken into, she lost both the computer and the external hard drive. Her comment was, “Ironically, my distrust was always with computers, not people. I was so careful to back up material because of my distrust of computers and the fear that mine may crash that I just didn’t think it would be people or, more specifically, thieves that I should be most concerned about”. If you aren’t backing up files into cloud storage (such as Google Drive, Microsoft OneDrive, Apple iCloud or DropBox, to name a few), you may wish to make it a regular practice each week to update your material and deposit it in another location, e.g., with your mother, partner or best friend. At least in the event of your computer crashing, or your flat being burgled/burned down and your computer going missing, you are secure in the knowledge that you have nearly all of the most recent material. You may lose a few days work between the last back up and the next and, from experience, we know that backtracking a few days work can be soul-destroying as you reflect every three minutes, “But I’ve already done this!” For this reason, you should ensure that the period between backup occasions should be shorter (hours or days) rather than longer (weeks or months). Often, it is not the actual dissertation material that is of most importance, but the data files, so the same rule applies. A copy of them should be kept in another location. Set up your systems for backing up your data and written material early in your studies and regularly implement them.

206

7.2.9

7 How Do I Stay on Track?

Present at a Conference

A great way of stimulating your flagging interest levels and gaining a confidence boost is to present a paper at a conference. While we recognise the limitations imposed by lack of funding and time, if you do have the opportunity of presenting your material at a conference, you will find it extremely rewarding, particularly where your paper is presented as a work-in-progress and you have identified yourself as a postgraduate student. In doing so, you will tend to find that the audience is supportive and constructive. Nearing the end of your study when you have some results, the feedback and discussion can become extremely valuable, as differing perspectives on your findings may be proffered. For example, a researcher was presenting a dimension of her work at a conference. The study was on values education in primary schools, and her central findings were that schools that had introduced values education had noticeable improvements in several dimensions such as student discipline and higher levels of morale. However, one finding was an anomaly in that graffiti and vandalism had not reduced. This finding had puzzled her for some time but, upon presenting to a wider audience, an experienced principal indicated that most of the vandalism occurring in schools was, in fact, not done by school members and, as a consequence, would not have been influenced by the values education process. This was a fairly obvious conclusion but an important one that the researcher had missed. The reaction at the conference was extremely beneficial in gaining an alternative perspective and greatly assisted with the preparation of a forthcoming journal paper. In addition to receiving useful feedback on your research, the experience of presenting at a conference should also be stimulating because of the people that you will meet, particularly those who are also interested in your field. A further aspect is being exposed to other research papers that may have a contributory value to your own research. You will also be able to build networks, possibly identify future examiners, and alleviate some of the isolation you may be feeling in relation to your research, as well as taking an intellectually invigorating break from your routine.

7.2.10 Check in with Your Motivations As the saying goes—the most important thing you will do in your doctoral program is finish it. However, a common challenge for doctoral students is keeping their “eye on the prize” (Conn et al., 2014). Completing a postgraduate research degree is a long, hard road with many pot-holes and washed-out bridges along the way. You may run into a mine-field and have to stop, turn around and explore other routes. If the goal is important enough to you, these obstacles will not prevent you from completing your journey but, if you don’t know why you are on the road, you will become discouraged and probably leave without finishing, having wasted several years of your life (Azuma, 2017).

7.2 Strategies to Stay on Track

207

Staying on track is about staying motivated and there are two intricately entwined dimensions. The first is the physical dimension of completing tasks, collecting data and writing chapters. The second is the psychological dimension that impacts significantly on your performance. To ensure that you stay on track, it is advisable to check in with your motivations as to why you are doing it. Is it career aspirations, or a personal sense of accomplishment, or perhaps to satisfy a sense of enquiry? Keep your reasons and goal(s) firmly in mind. It may help to post your reason(s) in a highly visible place, along with your planning chart which details all the future tasks. Laminate your goals and planning chart and have one copy by your computer, and possibly as one student did, a copy in the bathroom. This visibility is important as it will be a constant reminder of what you want to achieve, the tasks you are working on and the tasks looming in the future. If your initial motivations are stretched a bit thin and you find you are getting despondent or have simply had enough of the research because the project has been hanging around for too long, go for a walk. During the walk, recognise that your project is finite, and that upon completion you will, in fact, be able to re-join the human race and participate in social and sporting activities again. If this is just too far in the future, use some short-term motivation, something that is just around the corner and relates to your current activities. For example, reframe the current situation and set a time line with statements such as, “If I just keep going, I should have all the data collected by Christmas”. Part of what keeps many students motivated is a sense of investment. As you have already spent numerous hours (and money on enrolment), you should be reluctant to stop as that would equate to many wasted hours and dollars. Some students may enjoy the security of the academic environment, the significant autonomy of being a research student and the lack of organisational pressures usually experienced in a ‘normal job’. The warm, supportive and intellectually stimulating environment may, in fact, keep them working on their project longer than they should. Fortunately, economic drivers often force you out of the cocoon. For additional motivation, consider all the new job opportunities that may be open to you once you have completed. Alternatively, your motivation for undertaking postgraduate studies may not relate to your career but to a personal goal that you wish to achieve. Take a moment to reflect, once again, on why you are doing it. Checking in with your motivations is not a quick process, if you truly reflect and get back in touch with your reasons and consider as well some of the supplementary benefits that you might also derive from the accomplishment.

7.2.11 Evaluate Your Productivity A method for keeping yourself on track, used by both fiction and non-fiction writers, is evaluating your productivity and then trying to maintain a consistent level of output. There are a variety of productivity indicators that can be used by postgraduate students, for example:

208

7 How Do I Stay on Track?

• Completion of tasks—set yourself one to three tasks for the day, of varying intensity and duration, and try to achieve them. If you achieve more, that is a bonus. • Working consistently for a specific period of time—specify your work sessions and keep to them, i.e., one two-hour session each morning for six days. • Total number of productive hours in a week—snatch work sessions whenever you can and see how many hours of real work you can accumulate in a week. Record the time and try to match or better it the next week. • Number of pages written—a common method used by writers. Work until you have finished three pages per session. • Number of words—similar to pages but the productivity indicator is words not pages. This works well when you are creating the first draft, but less well when you are in the editing and refinement stages. • Sections within chapters—once you have framed out a chapter, you may put in a productivity indicator related to the achievement of, say, 7 sections in the chapter. • Completion of chapters—here you reflect on how long it takes you to create the first draft of a chapter, or to revise it, or to edit it. Note how long it took and try to better it the next time. Possibly, the most common and useful quantitative indicators of productivity are word counts and pages when you are writing up research, and productive time (in minutes or hours) when you are carrying out general postgraduate research work. Remember, initially, rather than having a word-perfect, absolutely tight-as-a-drum thesis, your aim should, in fact, be to present a first draft. This less-than-perfect standard will possibly free your thinking up and enable you to get the material down in a rapid fashion, knowing that later on you will be doing the tightening and the tidying. For measuring productive time, use a time-tracking method where you merely record what you are doing, in intervals of 10–15 min. If you were truly going to track your time, you would actually record each attention shift, for example, when you were interrupted by phone calls, unscheduled visitors, or switching to a lower priority but more entertaining activity in the middle of pursuing a tedious high priority one (Lewis & Habeshaw, 1997). Three columns are required: time of day, activity, time taken. Hopefully, if you do this for a few days, you will start to feel a wee bit guilty about the amount of time you are actually wasting. You may also wish to consider other indicators more directly related to the research project itself, for example, the number of companies approached, questionnaires coded, interviews undertaken, data analyses completed, etc.

7.2 Strategies to Stay on Track

209

7.2.12 Reward Yourself It is well accepted that modification of behaviour can occur through both punishment and positive reinforcement. With punishment, it is possible to train ourselves to do things by punishing ourselves when we do something wrong; however, this method can be inefficient, inhumane and damaging to your self-esteem. On the other hand, positive reinforcement, particularly where we reward ourselves as we accomplish small goals on the way to achieving our larger ones (which animal trainers call ‘shaping’) is both more pleasant and more effective (Bolker, 1998, p. 36). As we all work better with positive reinforcement, if you use indicators of productivity you will be able to reward yourself upon completion of milestones or stages and at various way points. Rewards can be effective for both long-term and short-term efforts. However, to be effective, a reward needs to be personally meaningful and desirable, i.e., something you enjoy. For one student, it could be a three-mile run, for another it could be something more relaxed like a night at the movies. Ideally, to ensure that your rewards retain their value, they should be used periodically and there should be a variety of rewards at your disposal with varying degrees of enjoyment. A weekend off is a greater reward than a chocolate bar. In your journal, create a list of two levels of rewards, interim rewards and more significant rewards, and add to the list of rewards when a new treat comes to mind. They do not need to cost a lot and could just entail some quiet time; in the end it will be a list of things you enjoy. Provide yourself with both short-term and long-term rewards. Short-term rewards can, for example, be obtained at the conclusion of a session (e.g., my friend who rose at 4.50 a.m. to write each morning rewarded himself at 7.20 a.m. with coffee and a perusal of the newspaper at a local café), through to more long-term rewards where the task has run over many weeks or months. For example, reward yourself on the completion of a section of work such as pilot testing, completion of your field work, entering all your data or finishing a chapter. Choose a reward which is meaningful for you but not too expensive; something that is a bit of a luxury. A friend would buy a trashy woman’s magazine as her reward item; another bought fillet steak. I note that these examples are consumable, but they could just as easily be experiences such as attending a sports game, a binge session on Netflix, playing a round of golf or going out for a meal. One of the best rewards is, in fact, rest. Allowing yourself to sleep-in on the odd morning or take an afternoon nap is nothing short of delicious. A nice bottle of wine, employing somebody to clean the house or do some literature searches for you, or more expensive rewards such as a massage, should be used in recognition of the more long-term accomplishments. Once you have a list of potential rewards, they can be matched with a variety of productivity indicators. For example, you could reward yourself for the achievement of a deadline. As part of your planning process, you have already broken your work down into manageable chunks and set milestones or deadlines, so that the

210

7 How Do I Stay on Track?

achievement of a milestone or deadline would be an opportunity for you to reward yourself. The best reward in these circumstances is something that will regenerate or revive you. For short-term task accomplishments, consider the rewards you might give yourself for sticking at it and completing a specific task or a work session. The reward could be 15 min of internet surfing or answering emails. One student we know realised they were easily distracted with reading their favourite blogs, news sites and Facebook and, before they knew it, a significant amount of time had elapsed. In an effort to manage their time more appropriately, they used their internet surfing as a reward. Ten minutes of surfing for every two hours of hard work. Socialising can be a good reward and can help release the pressure you may be experiencing, as well as getting some additional input into your study. However, it should be scheduled appropriately, and not too frequently, so that it doesn’t erode your work. Treat socialising as a reward for a day’s work well done. Going for a drink with colleagues could be the motivator for keeping your head down and working continuously during the day. Consider also the possibility of group-orchestrated rewards. With group rewards, you meet with fellow students, set targets as a group and meet again a week later to review whether you have achieved what you set out to do, then possibly reward yourselves with a shared meal. The benefit of the group approach is that it is good for short-term goal setting and not only can it enhance motivation (“I must get that done before I meet the group”) but it can also help alleviate isolation.

7.3

Conclusion

It has been suggested that there are two classic ways of undertaking a postgraduate research journey, one involves knowing just what you are doing, travelling down a clearly defined path, suffering only the occasional fit of gloom and despair and, unless you do something remarkably silly or give up, emerging with your degree before proceeding smoothly with the next stage of your career. The other way involves stumbling in, wandering around in circles for months, if not years, suffering frequent fits of gloom and despair, and probably, but not necessarily, emerging with your degree; the latter experience being not uncommon (Petre & Rugg, 2010). Completion of your postgraduate research is your task and no-one else’s, and it will not necessarily be one continuously interesting and productive activity. Recognise from the beginning that your motivation, self-confidence and self-efficacy will ebb and flow during the course of your journey, it is only natural. You will, therefore, need to push through the ebbs and capitalise on the flows. When things are going well, push ahead, and when things are starting to ebb, recognise that you are slowing down. The slowing down is more understandable and manageable when you are waiting for something else to happen, perhaps a draft chapter to be returned, as there are always other tasks you can do. It is, however,

7.3 Conclusion

211

more difficult when you are the cause of the ebb and, as a consequence, you are stalling and losing traction. If you are caught in an ebb situation and wish to regain momentum, we have suggested a number of strategies; however, in the first instance the quickest approach is to talk through the type of issues you are facing with other students, colleagues and your supervisor(s). All of them can help whether it is just listening to your woes or giving advice. As you will be reviewing a lot of interesting (as well as uninteresting) information, you could have a natural tendency to indulge in lots of reading, so try to avoid becoming side-tracked on informative, but not immediately relevant, material and potentially distracting issues. Keep asking yourself, is this relevant? Leave departmental politics to the others and, remember, it is not necessary for you to be on every or even any committee. If something doesn’t feel right, it probably isn’t right. Trust your instincts. One reason why many students slow down or get stuck is because they have the niggling feeling that something is ‘not quite right’ with their research, and then become reluctant to proceed. Hence, they slow down and find tangential tasks to fill the void without really tackling the substance of the problem. If you find that you are in this circumstance, trust your instincts and face up to what you think the problem might be. Acknowledge it and share the problem with your supervisor or others. The outcome may not be reassuring, but at least you and your supervisor can develop a strategy for remedial action. This will enable you, in time, to move through the problem and transition into the stage of fixing the problem. It is far more rewarding than remaining stalled and initiating at least some action will get you over the hurdle and back on your productive way. A lack of focus and direction resulting in procrastination appears to be most problematic for postgraduate students. It is, therefore, appropriate to identify what is stopping you from achieving your next objective and what is concerning you at the moment. Are there any additional resources or help you require? Is a deadline unrealistic? If you have been too hard on yourself, now is the time to stop, set a new deadline and give yourself some moral support. One of the best ways of dealing with procrastination is to reaffirm your priorities. Go back to your schedule of tasks and your time lines. What is the priority for this phase of your research? How long do you have to do it and what needs to be done? This should be enough to startle you into realising you cannot squander precious time and that you need to be conscious of keeping up your momentum. You could be in a mid-candidature dip, in which case, try to inject some novelty by varying your work place or schedule. Often students do not think about their bio-rhythms, their location, their posture, or their preferred facilities and that they need to vary them. Experiment with silence versus background noise (one of the best appears to be classical music, although sporting events also appear to be quite popular), being alone versus being in company, the early morning versus the early evening, the clipboard in an armchair versus a laptop, the pen versus the typewriter, the desk versus the coffee shop (Delamont et al., 2000). When you are in the final writing-up stage, keep up momentum by trying to write something every day.

212

7 How Do I Stay on Track?

Whenever you get stuck or have doubts, always consider the range of options open to you. One of the last resort options could, of course, be to quit or pursue a downgraded qualification. However, if you truly want a postgraduate research qualification, go back and connect with why you are doing it, why you are undertaking postgraduate research degree and what your ultimate goal is. Take a moment to visualise a successful outcome. This could be visualising yourself picking up the final, bound document from the printer, or walking across the stage to receive your degree or to be capped or hooded. It is your experience and, at the end of the day, if you have lost track and still want to complete it, it will be up to you to re-engage with your study and push to ensure completion.

7.4

Key Recommendations

• Keep planning and re-planning, checking and re-checking your progress against the plan, as well as revising your plans and setting realistic deadlines. You also need to have some emotional attachment to your target dates. • Post your goals in a highly visible place along with your planning tool (e.g., Gantt chart). • Should you miss a deadline, you need to be brutally honest with yourself as to why. Not achieving a deadline may be masking a personal frustration or obstacle that needs to be worked through. • Acknowledge that your motivation will ebb and flow. When you are lost, seek help no matter how small or big the problem is. • Keep persisting in the face of problems. Things are going to happen; it is part of the research journey, but you need to keep sailing forward. With help, all problems are resolvable. • Recognise the signposts of risks to achieving success—if/when you stop meeting with and start hiding from your supervisor(s), this will put your program completion at grave risk. It may be your research project and journey, but you should not undertake it alone. • Try to avoid getting side-tracked on interesting but tangential issues. Many a research journey has been diverted or thrown off-track by following some tangential issue down the proverbial rabbit hole. Keep your eye on the main game. • You don’t have to use all your data. You can be selective in order to focus more acutely on a particular area if you feel there is enough material to produce a convincing research outcome. The additional data can be used for later projects, further analysis and future publications. • Writing is actually the thinking part of the research process, not just the recording part. You don’t have to have your thoughts completely well-formed and structured in your head before you start writing. Analysis of data often

7.4 Key Recommendations

• •

• • •

213

comes through the writing process, where insights suddenly occur. If you wait for them to arrive and then record them, you could be waiting a long time. The reality is that very few people actually write sequentially, i.e., it is more common to develop the introduction, subheadings and a conclusion and to write randomly under each of those headings. Take any opportunity to present your material—at workshops or conferences— as mixing with like-minded people and getting positive as well as constructive feedback can be motivating and help you move closer to producing a convincing outcome. Be conscious of security as losing research-related materials can be devastating and debilitating and, where data are involved, extremely difficult to overcome. Keep active. Do something regularly on your research; don’t leave your work untouched for longer than a week. Make this a rule. Evaluate your productivity and reward yourself for both short- and long-term achievements.

References Ampaw, F., & Jaeger, A. (2012). Completing the three stages of doctoral education: An event history analysis. Research in Higher Education, 53, 640–660. Azuma, R. T. (2017). So long, and thanks for the PhD! Retrieved February 6, 2018, from http:// www.cs.unc.edu/*azuma/hitch4.html. Barnes, B. (2010). The nature of exemplary doctoral advisors’ expectations and the ways they may influence doctoral persistence. Journal of College Student Retention: Research, Theory & Practice, 11(3), 323–343. Bolker, J. (1998). Writing your dissertation in fifteen minutes a day: A guide to starting, revising and finishing your doctoral thesis. New York: Holt Paperbacks. Brown, S., McDowell, L., & Race, P. (1995). 500 tips for research students. London: Kogan Page. Conn, V. S., Zerwic, J., Rawl, S., Wyman, J. F., Larson, J. L., Anderson, C. M., et al. (2014). Strategies for a successful PhD program: Words of wisdom from the WJNR Editorial Board. Western Journal of Nursing Research, 36(1), 6–30. Cryer, P. (2006). The research student’s guide to success (3rd ed.). Maidenhead, UK: McGraw-Hill Education. Delamont, S., Atkinson, P., & Parry, O. (2000). The doctoral experience success and failure in graduate school. London: Falmer Press. Fenge, L. (2012). Enhancing the doctoral journey: The role of group supervision in supporting collaborative learning and creativity. Studies in Higher Education, 37(4), 401–414. Finn, J. A. (2005). Getting a PhD: An action plan to help manage your research, your supervisor and your project. London: Routledge. Gardner, S. (2008). Fitting the mold of graduate school: A qualitative study of socialisation in doctoral education. Innovation in Higher Education, 33, 125–138. Gatrell. C. (2007). Should you go for that PhD at home or away? Times Higher Education. Retrieved February 25, 2018, from https://www.timeshighereducation.com/features/shouldyou-go-for-that-phd-at-home-or-away/208073.article. Hockey, J. (1994). New territory: Problems of adjusting to the first year of a social science PhD. Studies in Higher Education, 19(2), 177–190.

214

7 How Do I Stay on Track?

Ivankova, N., & Stick, A. (2007). Students’ persistence in a distributed doctoral program in educational leadership in higher education: A mixed methods study. Research in Higher Education, 48(1), 93–135. Jiranek, V. (2010). Potential predictors of timely completions among dissertation research students at an Australian faculty of science. International Journal of Doctoral Studies, 5, 1–12. Lewis, V., & Habeshaw, S. (1997). 53 interesting ways to supervise student projects, dissertations and theses. Bristol, UK: Technical & Educational Services. Mackenzie, A. R. (1989). Time for success: A goal-getter’s strategy. New York: McGraw-Hill. Marshall, S., & Green, N. (2007). Your PhD companion: A handy mix of practical tips, sound advice and helpful commentary to see you through your PhD (2nd ed.). Oxford, UK: How to Books. McGee-Cooper, A., & Trammell, D. (1994). Time management for unmanageable people: The guilt-free way to organize, energize, and maximize your life. New York: Bantam. Nettles, M., & Millet, C. (2006). Three magic letters: Getting to PhD. Baltimore, MD: John Hopkins University Press. Noland, T. G., Francisco, B., & Sinclair, D. (2007). Pursuing a PhD in accounting: What to expect. The CPA Journal, 77(3), 66–68. Peters, R. (1997). Getting what you came for: The smart student’s guide to earning a Master’s or a PhD (Rev ed.). New York: Noonday Press. Petre, M., & Rugg, G. (2010). The unwritten rules of PhD research (2nd ed.). Maidenhead, UK: Open University Press. Pole, C. (2000). Technicians and scholars in pursuit of the PhD: Some reflections on doctoral study. Research Papers in Education, 15(1), 95–111. Race, P. (2007). How to get a good degree: Making the most of your time at university (2nd ed.). New York: Open University Press. Salmon, P. (1992). Achieving a PhD: Ten students’ experiences. Staffordshire, UK: Trentham Books. Spaulding, L., & Rockin-Szapkiw, A. (2012). Hearing their voices: Factors doctoral candidates attribute to their persistence. International Journal of Doctoral Studies, 7, 129–199. Thody, A. (2006). Writing and presenting research. London: Sage Publications. Wellington, J., Bathmaker, A. M., Hunt, C., McCulloch, G., & Sikes, P. (2005). Succeeding with your doctorate. London: Sage Publications. Zerubavel, E. (1999). The clockwork muse: A practical guide to writing theses, dissertations, and books. Cambridge, MA: Harvard University Press.

Chapter 8

How Do I Maintain a Good Work/Life Balance?

8.1

Maintaining a Work/Life Balance

Undertaking postgraduate study is unquestionably a long haul and, no doubt, if you are motivated to finish you will put in the hours. However, the reality is, if you are to remain sane and healthy and actually enjoy your research journey, you will need to consider how to achieve work/life balance. Here is how one postgraduate student described their experience: while I did not realize this at the time, being an overachiever led to poor time management and reduced productivity. I set very ambitious goals, and when I did not meet my deadlines, I drove myself harder. I worked longer hours, sometimes to the point of complete exhaustion. By my third year, I had experienced several episodes of burnout. I constantly felt guilty about not living up to my supervisor’s expectations, and I started to lose motivation. I considered quitting graduate school because I did not see a way of out the dark tunnel I was in. I decided to stay in my program and I got my PhD but the long hours at work impacted both my mental and physical health (Finish your Thesis, https:// finishyourthesis.com/time-management/).

For more thesis survivor stories, see Waring and Kearins (2013). In a study of over 8,000 doctoral students, Mason, Goulden, and Frasch (2009) observed that this generation of doctoral students has different expectations and values from previous ones, primary among them is the desire for flexibility and work/life balance between career and other life goals. They also noted that changes to the structure and culture of academia have not kept pace with these major shifts. The term ‘work/life balance’ would suggest that there are two elements, work (your study) and life (your well-being), and that they should be kept in equilibrium. In one study, international graduate students were asked about their descriptions of well-being and their responses included physical health, happiness, joy and pleasure, harmony of body, mind and soul, feeling of security, sense of satisfaction, fulfillment or achievement, and feeling as if their life is meaningful (Wen-Chih & Newton, 2002). Exploring how female PhD students perceived their well-being, a Swedish study identified being true to oneself, being in the sphere of influence, and © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_8

215

216

8 How Do I Maintain a Good Work/Life Balance?

performing a balancing act (Schmidt & Umans, 2014). For a postgraduate student, there appear to be several dimensions of your life that require juggling, in addition to the research/study dimension. It is unlikely that you can realistically aim for equilibrium between all dimensions, but you will need to make arrangements to ensure that significant imbalances do not occur.

8.1.1

What Constitutes Work/Life Balance?

For postgraduate students, we can identify five important life dimensions necessary to keep in balance, or avoid a distressing imbalance, during your research journey: • Employment—increasingly, postgraduates are already employed or undertake full-time or part-time work while studying, on top of their research work; • Mental well-being—such as recognising and managing stress as well as seeking mental stimulation in areas other than your research; • Physical health—including attending to what you are consuming, managing weight, exercise and rest, as well as attending to ergonomic considerations; • Social interaction—maintaining communication with the people you care about, and looking after key relationships; and • Spiritual well-being—while meaning different things to different people, depending on their spiritual orientation, essentially involves engaging in activities that create positive interactions between your inner-self and the world. This can be achieved usually through giving back to others, maintaining religious and cultural practices, providing service, being with nature, meditation, prayer and/or even positive visualisation.

8.1.2

So Why Is Finding Some Sort of Balance Between These Life Dimensions Important?

As a postgraduate student, you should expect to be challenged, but it is important that the demands of your research are kept in perspective, and do not adversely affect your health and happiness. It is also important to recognise the connection between an unhealthy, unbalanced lifestyle and the inevitable loss of productivity and subsequent non-achievement of the milestones attached to your research. There is a difference between your productivity (outputs) and your productive capability (capacity to produce desired outcomes). An imbalance can stifle your productive capability thereby compromising the necessary fuel that drives your performance over an extended period of time-term. Attending to your work/life balance is important because it means you will be able to stay the distance. If you focus only on your research and little else, you will soon burn out and wonder why you are losing interest and motivation and possibly falling behind on your research plan.

8.1 Maintaining a Work/Life Balance

217

As the saying goes, while there is life after a postgraduate degree, there should also be life during the journey toward that degree. As has been pointed out, “undertaking a long research programme has to be a way of life, not just a job, because it cannot simply be locked away into office hours inside the institution or other place of work” (Cryer, 2006, p. 27). Given the age range in which most individuals undertake a postgraduate program, it can be a busy time for other life events such as a job, courting, marriage, community activities, career developments and children, so you may have a lot to contend with while undertaking your research in addition to managing your own physical and mental well-being. Being aware, early on, of the need to accommodate all of these dimensions can enable you to adequately structure your life during your postgraduate studies and to set up early warning systems for when activities are getting out of alignment or leading to negative outcomes. Once again, by balance, we do not mean absolute equilibrium in all areas but rather at least some form of tolerable approximation. This is important because experience has shown that if one element starts to overly dominate your life, it can become detrimental to your well-being. However, keeping things in perspective and attempting to actively look after each dimension in your life does enhance your productive capacity and, ultimately, the likelihood of successfully completing your postgraduate studies. Given that equilibrium amongst all dimensions is unlikely, we are starting to see a preference for terms such as ‘work/life harmony’, rather than work/life balance. For the purpose of our discussion, we will use the more common term, work/life balance, but recognise that it is not about equity in your life, it is about ensuring a healthy environment for you to work in and sustain your work practices. So, let’s look at these five dimensions inherent in achieving well-being while you are undertaking your postgraduate research journey.

8.2

Employment Dimension

Increasingly, given the costs of higher education, many students are now undertaking their postgraduate studies part time while also working, either part-time or full-time, often within an academic environment. Using semi-structured interviews, Garber and Gopaul (2012) found that many of the issues for part-time doctoral students involved concerns related to balance, support, and fitting the mould of a ‘traditional’ doctoral student. Importantly, while some of these issues are consistent with the experiences of full-time doctoral students, the experiences of part-time doctoral students are more complicated and required sustained and more flexible approaches. In some countries, the rapid growth of candidate numbers has been associated with a more diverse postgraduate student population and more flexible patterns of research and study (Pearson, 1999). If you have work commitments, you must be careful of ‘work creep’, that is, where working on your job or your teaching or marking commitments, incrementally creeps into the time you have specifically allocated for your research. When you see this happening, it should ring an alarm bell for you. You need to stop and re-assess, re-schedule and possibly re-negotiate

218

8 How Do I Maintain a Good Work/Life Balance?

with your employer or even drop a commitment. For those who work in an academic environment, for example, the UK guideline is six hours of teaching per week during term time (this includes preparation and marking) while undertaking doctoral studies. For students working outside of the academic environment, however, the situation is more difficult as the pursuit of a postgraduate qualification is frequently viewed as a personal indulgence rather than as genuine professional development. If you work for an organisation that is supporting your studies, either with time or finance, stay where you are as you are most fortunate! Working students may also need to manage guilt as some experience conflict between their research and their own jobs. This is a very real concern because, if not handled correctly, you could feel that you are robbing your employer in order to work on accomplishing your own goal of a postgraduate degree. For this reason, it becomes even more important to ensure appropriate scheduling, so that your employer does, in fact, receive your full commitment and that you do not short-change them. This is especially important if you are completing your postgraduate study part-time. You will want to keep focused on your study and to remain in contact with your supervisor(s) and peers, but not to the detriment of your focus on your job and vice versa. Keep the dialogue open with your employer and your supervisor(s) regarding expectations, your study schedule and your progress. You do not want brooding resentments of your study to surface during your next performance review or a discussion of how your work commitments are impeding your progress during a postgraduate candidature review.

8.2.1

But What if I Am a Full-Time Student?

If you are a full-time on-campus student, then the university is your work environment and you need to consider your role in the organisation. Your role in the department can vary according, from being purely a student who happens to be at the postgraduate level, through to their consideration of you as an intellectual and academic colleague, possibly also engaged in some teaching. You will likely be somewhere along this continuum. If you are a full-time or part-time distance postgraduate student, then you may seldom be on campus. You are likely to be employed full-time. This means your home also becomes your research work environment and most of your research work is done after hours. It can be much harder for distance students to build up and maintain relationships with on-campus academic staff, other than supervisor(s) and, as a consequence, they can feel more disconnected from the nexus of research energy and activities associated with other staff and postgraduates. You are also less likely to be considered for any opportunities to support teaching and research activities within the department, simply because you aren’t physically there. This disconnection makes it much easier for your employment to interfere with your research time, so you must be diligent in allowing sufficient room for both. If you are in this situation, our best advice is to take every opportunity to cultivate

8.2 Employment Dimension

219

relationships, by attending any postgraduate events on campus or via web conferencing (e.g. an annual or semi-annual postgraduate seminar; connecting with other postgraduate students in your geographic area in a study group; arranging with your boss to have time off for campus visits or for ‘research days’). For students who have come from professional careers where they were well-respected, finding a place in an academic department is often more difficult than for younger students. A mature-age student may experience difficulties with returning to student life. These difficulties may sometimes relate to the tasks involved with being a student (e.g., you may not have written anything for years, you may have forgotten how to study or how to use a library, you may not be computer-savvy), but more often relate to the lack of recognition for your prior knowledge, experience and accomplishments. This is particularly problematic for individuals who have been senior managers and are used to a certain level of respect and power. Such individuals often find themselves relegated to a lower status as a postgraduate student and can find it uncomfortable. If this is relevant to you, to manage this change, acknowledge that you are now in an entirely new field and need to earn your stripes, just like entering a new organisation, and this will help keep your ego in check. If you are not already aware, you will soon become cognisant that academic departments are quite political places. Academic staff, despite their intellectual veneer are actually quite competitive as they vie for internal and external funding opportunities, journal publications, conference funding, promotion and recognition among their peers and managers. By your vicarious association with an academic staff member as your supervisor, you may inadvertently get caught up in a tableau of petty jealousies. At all costs, try to stay clear and remain on good terms with all academic staff and other postgraduate students in the department. Do not engage in conversations that appear to verge on back-stabbing, gossip or speaking inappropriately. Simply excuse yourself from the situation, citing the need to go to the library, pick up the children or whatever other excuse you can think of, in order to avoid being drawn into the conversation. Don’t burn any bridges! Full-time on-campus postgraduate students are frequently asked to assist with teaching, research or other projects. If you are a full-time student contemplating taking on a short-term or temporary teaching or research obligation, try to ensure that the work allocation does not exceed 25% of a normal workload (Kehm, 2005, p. 18). Full-time students should avoid becoming overloaded, given the large number of requests that could potentially come your way. Think carefully before you commit to any additional projects. Weigh up the amount of time and effort that will be required and the benefits that might accrue from the activity. Try to assess what is involved in the project, how many meetings, the number of training sessions and the travelling time involved and form a reasonable estimate of the likely total time commitment. If you don’t know, ask. It is better to err on the side of factoring an allowance for additional time into your assessment as projects often balloon out. Having now estimated the total time, consider whether you are willing and able to put that time aside. Are you prepared to incorporate it in your calendar, and what might need to be pushed out (for example, personal activities) in order to

220

8 How Do I Maintain a Good Work/Life Balance?

accommodate the additional project? Also consider the benefits. Be generous in visualising what those benefits might be, as they may not be immediately discernible. Helping to organise a conference could put you in contact with others in your field, future examiners or employers, but will take a lot of time, especially when conference papers start rolling in. Be honest with yourself. If your reflections generate a dull ache of concern that the work is just going to be too much in light of all the other things you have to do, you need to be honest with those feelings and acknowledge that it is perhaps best to turn the project down. Saying ‘yes’ to a project or task that is not on your project plan will slow you down and could compromise the achievement of your milestones. Remember, there will always be potential consequences to agreeing to additional activities in the form of delays to your research and/or less time do other things you enjoy.

8.2.2

How Do I Go About Turning Down Requests for Additional Commitments?

Keep an eye on your end goal. The more activities you put between you and your goal, the more you will delay its attainment or possibly detract from the quality of what you are trying to achieve. When turning down an extra activity, go and see the person face-to-face and indicate that it was a difficult decision as you would like to have been involved, however, you are putting your research first and your desire to complete it within a timely fashion is your priority for the next six months or year, whatever the timeframe. If need be, practise what you want to say before speaking to the person. In this way, you will sound convincing and be relaxed in the process of delivering the bad news. Saying ‘no’ can be particularly difficult when there is a power imbalance, that is, when your supervisor or the Head of Department asks you to be involved in a project. However, remember that your primary responsibility is to yourself and to your project and, if you have a gut-wrenching reaction that the requested task is going to be a big distraction to you, that is the time to say ‘no’. Don’t wait until you are involved in the request and expectations regarding your contribution have been established. It is better to pull out right at the beginning rather than get involved and then disappoint people by pulling out later. As soon as possible, indicate that, as much as you would like to participate, you really need to keep your eye on the ball in your research. A prompt response will enable the requester to quickly move on to another option. Deep down you may, in fact, find that they respect you for taking this stance and for your single-mindedness to achieve your goal. Whether you are a full-time or part-time student, the work dimension of your work/life balance will predictably dominate, hence the importance of using scheduling and time management tools. However, traditional time management typically ignores the need for recreation and/or play time, and views time spent on such things as socialising as wasted. In your time management system, however, consider what you have left off your To Do list, that is, those things that should be in the list in order to safeguard the totality of your being. Sometimes students can

8.2 Employment Dimension

221

become so pre-occupied with their postgraduate research that all else falls by the wayside, including family, friends and activities they have previously found enjoyable. While you may find that outcomes are quite productive early on, the long-term effect is that you can quickly become isolated, unfit and despondent. The reality is that activities such as recreation and exercise and interactions with people we like and love can actually benefit our research. It is, therefore, most important that you consider maintaining those activities and schedule them into your time allocations. If your entire emphasis is on getting your research done and you just give personal, partner and family time whatever time is left over, you may find that both you and others start to resent your study. So, while it might seem quite calculating, don’t forget to put into your To Do list and weekly scheduling, things that create balance for you. It may sound somewhat business-like, but we suggest that you diarise such activities with an appointment to make them happen. You won’t get to see all the people you want or do the amount of exercise (or binge watching) you used to, but it is important that you maintain alternative interests, in addition to your research, that give you a break away from your work. Speaking of To Do lists, if a task remains on your To Do list for too long, don’t be afraid to acknowledge that you are probably never going to get around to it and be brave enough to drop it. It may free up some time during the week for something else (McGee-Cooper & Trammell, 1994, p. 83). Non-focused time, that is, time away from your research can, in fact, still be productive as it may provide time and space for reflecting on your research. This is the time to consider possible phraseology for what you are currently writing, or to contemplate alternative interpretations of your data, etc. The answers to current research issues you are experiencing can come at the most unpredictable times and often when you are not at your desk. The good idea you have in the shower or when you are out for a run has come about because you have let your mind relax and allowed your subconscious to work. I have a former colleague who refers to this as ‘tummy rubbing time’ and he considers this integral to good research and writing as it is time when you reflect, and gain insights related to your research.

8.2.3

What if I Want to Shift from Full-Time to Part-Time Study or Even Suspend My Study?

If, because of work or family circumstances, the unexpected happens and you need to modify your enrolment pattern (e.g., suspend or extend your postgraduate study) it may, in certain circumstances, be appropriate, albeit perhaps disappointing. In doing so, it will reduce stress and provide you with the ability to deal with a situation such as a home relocation, new job, new baby or a bereavement, and enable you to return with full productive capacity to the task at hand. This is not the end of the world and may make good sense—exchanging a short-term loss for a longer-term gain. However, make sure you know the institution’s policies and regulations governing such modifications, and the processes for obtaining approval for any changes. As

222

8 How Do I Maintain a Good Work/Life Balance?

one student commented, “Moving from full-time to part-time candidature was one of the best things I did. While it lengthened the time taken on the thesis, it did however enable a continuation of a career, full-time employment, extra income and access to data sources that I would previously not have been able to generate”.

8.3

Mental Well-Being Dimension

The mental dimension of work/life balance relates to your emotional well-being and it is not uncommon for students to experience a wide range of emotions. Emotions can cover the full spectrum from elation through to the more negative and potentially dysfunctional emotions. Negative emotions, such as as disappointment, boredom, insecurity and anger, can arise at any time during your postgraduate journey. These emotions are predictable and are significantly entwined with performance. A difficult head space could result in poor productivity and, conversely, achieving outcomes can have a remarkable and positive impact on your mental attitude. As your emotional well-being is an area that appears to be most problematic for postgraduate students, we will spend some time on this dimension. One emotion, at the lower level of the spectrum, that is not often discussed in the postgraduate research context is resentment. It usually occurs when the student is about two-thirds of the way through their study and is beginning to resent the time their research is taking, particularly if they have other significant commitments such as a family, sports or strong friendships. One possible way of trying to alleviate resentment is to give your-self a mini-reprieve. If you are working every weekend, you may wish to look at one weekend per month being a reprieve weekend for a period of, say, two months just to give yourself a bit of a break. The resentment phase does pass. As you near the end of your postgraduate research program, the time commitment required will increase quite dramatically. Fortunately, this coincides with a spurt in motivation as you start to see the light at the end of the tunnel. Your energy is also stimulated by finding your study to be increasingly more interesting as you start to pull it all together and draw conclusions. Just when you think you are almost finished, resentment will often rear up again as you envisage getting your life back and you simply can’t wait for that to happen. If you are starting to feel negative emotion creep in, stop and reflect on what might be generating it. Try to actually isolate and name it, rather than wallow in the general negativity that surrounds it. Having identified the emotion, or possible mix of emotions, now look at the causes. The reason for this introspection is that sometimes you can take the causes away. For example, disappointment could be a result of your frustration that, despite putting the time in, you feel that you should be further along in your research than you are. High-achieving individuals appear to be very good at ticking themselves off for not accomplishing, so be careful of that word ‘should’! Be realistic and be kind to yourself. If the cause of the negative emotion is not within your control, recognise that fact, acknowledge it and adopt a healthy perspective towards it. If, for example, you have

8.3 Mental Well-Being Dimension

223

just learnt that your supervisor will be away for the next three months and you are really annoyed, you have a choice to stay annoyed, frustrated and angry, or you can look at your work plan and make constructive changes in order to make best use of the time. Speak to your supervisor and have them approve the revised plan and ask if they would mind email/skype contact while they are away, or you could negotiate for another supervisor to temporarily step in. Yes, it is a hurdle but get over it. You can stay locked in the emotion or you can make practical steps to resolve the situation. What is the most constructive, adult, healthy response to the situation (even if you don’t want to do it)? Identify an appropriate response and get on with it. To manage the emotion, you may find it helpful to talk to other postgraduate students who will also be grappling with similar mental states or circumstances. They may have some useful strategies to address the situation. However, if you find the situation and the associated emotions overwhelming, you should naturally seek more formal support from the university counselling service.

8.3.1

Why Do I Sometimes Feel Out of Control and Really Stressed?

While postgraduate study is designed to enhance your knowledge, understanding, cognitive and intellectual skills, it is also seeking to develop behavioural skills, such as self-management, decision-making, resilience and stress management. The difficulties you experience are also part of the learning process but, on occasion, may become too much to handle. This is not uncommon and Ramiro Valdez, in a study of stress in doctoral students in the USA, identified that on a scale where 100 equalled the amount of stress experienced by someone whose spouse had just died, doctoral students in their first year scored, on average, a whopping 313 points (Peters, 1997, p. 5). Following a systematic review of a number of research articles, Ribeiro et al. (2018) highlighted the negative association between stress and the quality of life in university students, and its relationship to the deterioration of various aspects of physical and mental health.

8.3.2

What Exactly Is Stress?

Let us take a look at what stress is and isn’t. Here are some interesting points about stress: • Stress is not always negative. When talking about stress, we are usually talking about the negative side of stress, not “eustress” which is the positive or motivating stress that you may experience as you are approaching a deadline. • Stress is not anxiety. Anxiety is often a symptom of stress, but stress, unlike anxiety, also operates in the physiological sphere, as well as in emotional and psychological areas.

224

8 How Do I Maintain a Good Work/Life Balance?

• Stress is not nervous tension. An individual could be experiencing stress without exhibiting nervous tension. If present though, nervous tension, like anxiety, is often a symptom of stress. • Stress is not only caused by bad occurrences. Good things happening in a person’s life can cause stress. Attending too many parties at Christmas time or getting ready to go on holiday can get one stressed. • Stress is dynamic. We experience different levels of stress at different times depending on our workload or life events. We do not necessarily have to experience a continued or sustained level of stress to qualify as being stressed. This is because stress tends to have a cumulative effect over time. • Stress can be passed on to others. Partners and secretaries of stressed individuals can feel stressed themselves. In fact, close associates may experience even more stress, because they are often unable to control or influence the stress you are feeling. • Stress is not entirely caused by external stimuli. Stress is more the result of an individual’s cognition and subsequent responses towards external stressors, particularly the subjective experience and appraisal of events by a person. In other words, your interpretation of the situation can play a dominant role in stress formation. Some people, in fact, enjoy getting themselves stressed. So, stress is not so much caused by what you eat, but by what is eating you. • Stress causes the body to release cortisol. Cortisol is the hormone that has a direct impact on the brain causing the cortex to shrink. Further stress releases adrenaline, the ‘fight or flight’ hormone. The combination of these hormones produces a perceptually narrow tunnel vision necessary for survival and our brains lose their short-term memory capacity. These conditions might save our lives when escaping from a burning building, but they work against our being able to study successfully (or sit for an exam or a job interview). When studying, we need to be relaxed to utilise our short-term memory and to develop a breadth and depth of vision (Burns & Sinfield, 2016, p. 56).

8.3.3

How Do I Recognise When I Am Stressed?

Most postgraduate students will experience some type of stress during their journey. Some individuals have a less emotional resilience and experience more stress. Stress may be even more problematic for students who are from minority groups. International students usually have the double-whammy of being from a minority group and away from their usual support networks. Older students may also find postgraduate study problematic, as it is a significant change from their normal working environments, in the sense that they have little in common with the younger cohort of fellow postgraduate students. All students will experience varying levels of stress. Learn to recognise your stress levels and be aware of the symptoms. There are different categories of symptoms of stress, and each has differing manifestations:

8.3 Mental Well-Being Dimension

225

• Physiological symptoms—headaches, backaches, lower immunity and physical ailments; • Psychological symptoms—tension, anxiety, irritability, depression, confusion in logical thinking, etc.; and • Behavioural symptoms—changes in productivity, absenteeism, presenteeism (not wanting to stop work), increased consumption of alcohol, sleeping disorders, etc. Closely related to stress is the concept of ‘burn out’. Put simply, we all experience varying levels of stress and there is a considerable degree of personal variation in our ability to handle stress. What may be a niggling irritation for one individual is overwhelming for another. The key difference is the emotion that we attach to the circumstance. When the stress levels and emotions attached reach the point of creating dysfunctionality, it can become burn-out and usually involves a number of psychological, physiological and behavioural responses such as depression, health issues and increased consumption of alcohol and/or drugs. Signs of burn-out are: tiredness, emotional responses, reduction in motivation, poor productivity, depression, nervousness, irritability, and changes in appetite. Another manifestation of extreme stress is anger and even rage. “It isn’t surprising that people under stress often feel full of rage, which is often not specifically directed. People often become very frustrated and they feel powerless. Anger, once generated, can be spread in many directions, and the most harmful of these is when the anger is directed inward” (Race, 2007, p. 244). As Peters (1997, p. 268) pointed out, psychological stress can exacerbate existing psychological problems including eating disorders and abuse of alcohol or other drugs. Be sensitive to your own levels of stress. The warning signs are persistent, low mood or irritability, loss of concentration, feelings of hopelessness, worthlessness, weakness, inadequacy, or guilt or loss of interest in activities that you normally enjoy (Peters, 1997, p. 276). For more information on healing from stress and regaining your energy, see Dow (2018) and Sauser (2012).

8.3.4

How Can I Manage My Stress?

With the demands of new types of learning, deadlines or experiencing, a sudden turn in your research that can cause delays and frustration, you may experience levels of stress that you have not previously encountered. Negative stress can actually decrease your productivity and inhibit your intellectual capabilities, so the sooner you learn stress handling strategies, the better. In resolving stress, quite simply, there are healthy and unhealthy strategies for attempting to cope with stress. The unhealthy ones entail: • denial—not realising you are experiencing stress; • procrastination—doing anything except the task you need to focus on;

226

8 How Do I Maintain a Good Work/Life Balance?

• wallowing—recognising you’re in stress and wallowing in it, “Oh, woe is me…”, but not pulling yourself out; or • distraction—reaching for something as a means of distracting you from the stress. For example, self-medicating with alcohol or drugs in order to mask the stress, continuously checking your social media or YouTube videos.

8.3.5

What Are Some of the Tactics for Managing Stress?

• Look at the sources of stress—Self-talk or writing can be useful, that is, just recording your concerns in a diary. Do not resort to emailing other people. Put the sources of your stress into one of two categories—‘controllable’, I can do something about this, or ‘uncontrollable’, this is out of my hands and I need to get my head around the situation and make the most out of it. • Do not anticipate difficulties unless you can develop a concrete way to deal with them—Some individuals have a natural inclination for anticipating and then hyping difficulties and, in doing so, they stress about situations before they’ve actually arrived. This is especially true if you imagine difficulties without thinking about what you might do if they do arise (thinking about positive steps before they might be needed, which helps to reduce the uncertainty associated with generalised worries about things that might happen to you). If you know you have a predilection for that type of activity, try to work on that area of your psyche, as it is only creating additional stresses that you do not need. Try using the mantra ‘It’s all ok until it’s not ok’. • Distance yourself from the problem by engaging in some physical exercise—Go for a walk or a swim, or do a work-out, whatever is your preference. However, when exercising, don’t force yourself into a routine or over-exert yourself. Slavish or overly vigorous adherence to a fitness program could also be stressor injury-inducing. • Take responsibility—It has been mentioned previously that, as an independent researcher, you will be taking responsibility for your project and, in doing so, will need to have a positive attitude towards the many challenges that will come your way. When things get difficult it is often tempting to blame others for the problems you are experiencing. This is a natty form of displacement but is neither constructive for your mental health nor helpful for your growth within your research journey. Recognise that the difficulty is now yours, and yours to solve. Seek advice on how to best minimise the impact of the problem or to rectify the situation. If you are going to have a healthy progression through your postgraduate research journey, putting the responsibility squarely in your camp for maintaining progress will help in the times when you do get stuck, when it does become frustrating and/or when you are feeling particularly bottomed-out.

8.3 Mental Well-Being Dimension

227

• Take an action-oriented approach to solving the problem—Be curious as to how the problem can be solved. This may involve voicing your concerns or getting feedback and assistance. Sometimes it is not the work that is creating stress but another factor. One student confided that she was happy with the progress on her study but, with a full workload (she worked full-time) and her sporting interests, the relationships with her partner and her study, the one thing that kept falling through the gap was the cleaning. This in itself made her very stressed as she realised it was mounting up on her and every time she looked at a dusty surface she was irritated. She solved this by sharing the problem with her partner. They decided to pay a neighbour’s high school aged daughter to do the cleaning. She said the relief was immense. Identify and escape from traps, solve the problem—e.g., if you have two supervisors disagreeing, get them together and work it out (Peters, 1997, p. 277). • Experiment with different approaches to solving the problem—You could remove the cause. For example, if your flatmates are a constant cause of stress, it may be better to shift house; or negotiate to reduce the cause. For some people, the actual writing process, that is, putting fingers to a keyboard, can be daunting. Experiment with different approaches. Speaking is less permanent and onerous. The trick may, therefore, be to capture your thoughts using your smartphone or voice recognition software or give a presentation and then sit down to the computer, using the notes from the presentation, as the basis for your writing. Alternatively, speak to friends and colleagues and summarise your discussion with them. Try different strategies in an effort to resolve the problem. • Increase your competency—Consider what skills you could improve. Decrease anxiety by becoming more competent in a particular area, for example, public speaking, handling statistical data, writing, etc. You may also wish to improve your interpersonal skills in areas such as assertiveness, empathy, conflict resolution or communication. • Control what you can and accept what you cannot—With an understanding of what creates stress for you, work on those circumstances that are stress-inducing. Investigate the source and work with relevant factors—it could be, for example, the relationship with your supervisor. Spend a bit of time reading and reflecting on how this might be improved. One of the common causes of stress for postgraduate students is information overload and not knowing how to process the sheer volume of information which emerges as part of the research journey. If you get a handle on this, it would certainly assist with your stress levels. On the other hand, there will be some areas that you cannot control, and you will need to face the equally demanding prospect of developing strategies to merely accept and minimise the impact of those factors on your research and on your life. This can be difficult to do but is a lot healthier approach in the long term. • Be careful of emotional spirals—Negative thoughts can get in your way. If things are going badly, you are stuck on a problem, you have lost some data, your analysis is not working out, or you have forgotten to save a file, it is tempting to get into an emotional spiral, particularly if it is late in the evening. Call a halt and start fresh the next day when you have more emotional resilience,

228

• •

•

•

•

•

•

8 How Do I Maintain a Good Work/Life Balance?

otherwise you could be working yourself up into a longer-lasting negative mental state. Be thankful for what you have—Work on the positive, possibly keeping a gratitude journal and always celebrate your successes. Put some order your life—Productive ways of dealing with stress are to take control, organise your time efficiently; manage your finances, develop a filing system, and de-clutter. It is also the urgent stuff that makes you stressed so use a To Do list and set priorities. Practise good time management and project planning—preparation takes a lot of uncertainty out of your project (Webb & Ott, 2005, p. 6). Stop the spinning—Take some time on your own, stop the world for a moment and quietly reflect. Slowing the pace can have a very soothing effect. This may also help settle your mind and make circumstances appear less devastating. Mediation can be very helpful practice. If you play an instrument, you may find it relaxing just to take some time out to play it; if you have a hobby, it may help to focus on that for a short period. What you want is to distract your brain, so it can recover from the spin. Increase human contact—Get together with friends and family, talk to others who are going through their own postgraduate research journeys or have completed and appear to have coped well (although you may be interested to find that they also had their low or stressed times). Remember, you only need one or two fellow students (either in person or in a chat group) you get on with to create an effective support group. Spend a bit of time speaking to other students until you find the one or two you would like to engage more frequently with. Act undepressed—If you act happy, sometimes your feelings will fall in line. Just as there are different working habits, there are also different thresholds for withstanding stress. So, if you don’t have a high threshold, pretend that you do. Fake it, until you make it. Avoid dwelling negative thoughts—Such thoughts are unrealistic, emotionally draining and self-defeating. Ensure that you maintain positive self-talk. Saying that you can’t do something usually means you won’t be able to do it and, as somebody quite rightly pointed out, negative speech patterns are like writing yourself personal, poison-pen letters. Why do something so nasty to yourself? Stop being hard on yourself; avoid being overly critical of what you have not yet achieved; refuse to give into worrying about being too slow, not being smarter, not understanding that concept more clearly; don’t fixate on why you can’t type faster, why you are so slow at data entry, why you can’t understand statistical analysis material, etc. etc. All of these are negative comments that you are making about yourself. These inner dialogues can be extremely discouraging and you are not doing yourself any favours by ticking yourself off on a regular basis. Nurture yourself—Be kind to yourself and leave your self-deprecating comments aside. Instead, although it may feel a bit strange to start with, be positive; say good things to yourself when you accomplish something. How about saying ‘Well done, you managed to stay on task for two hours’ or ‘Well done, you managed to do the hard job first rather than procrastinating’. Don’t hesitate to

8.3 Mental Well-Being Dimension

• •

•

•

•

229

congratulate yourself or forgive yourself for mistakes. A nice way to end the day is to think about what good things happened during the day; even small things can be quite meaningful. Get moving on your work—Much of postgraduate student depression comes from procrastination and isolation from academic staff. So, re-engage and get moving again. Seek counselling—If you are feeling that you just can’t break the barrier and climb out of the negativity you are experiencing, you may require additional counselling. It has been suggested that as postgraduate study involves significant personal, psychological and academic risk and emotional investment, counselling can offer an important source of additional support for students (Wright, 2006, p. 352). Counselling can reduce levels of psychological distress and blocks to potential academic progress (Wright, 2006, p. 359). Institutions will have an ombudsperson or a counselling service so seek help from university counselling centres. They may suggest relaxation techniques or cognitive therapy, where you may be asked to retrain and take a more proactive approach to problem solving. Counselling can also assist in changing your response to stress (perceptual restructuring), acquiring coping skills, keeping things in perspective, learning to let go and learning to forgive others. Learn relaxation techniques—Peters (1997, p. 220) referred to an interesting study by the American National Academy of Sciences that concluded that subliminal suggestion tapes don’t help people lose weight, quit smoking or earn more money selling real estate. However, scientists concluded that the one thing that does work to improve performance is plain old relaxation. A relaxed person will out-perform a stressed person on everything from a maths problem to tennis. For example, for academic writing, this means that in order to get started, you should get comfortable and nice and relaxed. Use relaxation techniques such as meditation, positive imagery, massage and anxiety management training, breathing exercises, progressive muscle relaxation (concentrating on each joint progressively) and mental relaxation, such as using imagery to create pleasant and relaxing scenes in order to promote a relaxed state. Change your perceptions—The philosopher Epictetus is credited with observing that people are disturbed not by things, but by the views they take of them. Cognitive techniques for managing stress focus on changing labels or cognitions so that we appraise situations differently, avoiding over-generalisation or taking a situation too seriously or personally. So, don’t panic yourself or exaggerate a situation. By keeping things in perspective, you are better able to cope not only with the amount of work that you must do, but also with the additional pressures that may pop up as a result of unforeseen problems, hurdles or difficulties. You will be in a better space to actually manage problems and, rather than throwing up your hands in despair and going into a funk, you will be able to take a more reasoned approach to a problem merely by changing your perception of it. Reduce consumption of stimulants—Stimulants continue to promote the red alert reflex in our bodies and restrict our parasympathetic system from doing its job and releasing tension. So, aim for moderation in all things and, at the very least,

230

•

•

•

•

8 How Do I Maintain a Good Work/Life Balance?

reduce the amount of caffeine you consume. Also, our parasympathetic system works extremely well when we sleep so get as much of it as you can. Keep breathing—You will be amazed how shallow our breathing becomes when we are stressed. Take a moment to re-centre yourself by inhaling and expelling all the air in your lungs. Do this two or three times, and then for a while with your hand on your stomach. Concentrate on breathing so that your stomach becomes extended. The way you stand is also important, as slouching for any length of time can put stress on muscles and promotes shallow breathing. The interesting thing is that we often don’t realise that we are slouching. Are you slouching now? Maintain other interests apart from your research—Make time for things that you enjoy. Concentrating on just one project, even though it is an important one, may ultimately bore you. As humans we thrive on variety (rarely do we eat the same meal every day for a week). It is the contrasts that help maintain your interest. In a study session, this can be achieved by varying the tasks that you do and, for a long-term project like a writing a thesis, dissertation or portfolio, by introducing different dimensions to your day or week and maintaining other interests. Use wearable technological supports to help you manage your health—nowadays, there is a plethora of different technologies for monitoring and managing various aspects of your heath and exercise. FitBits and other smart watches like Apple Watch, mobile phone apps like Samsung S Health and other health tracking and monitoring devices and software may be able to help you reduce stress through helping you to monitor and record things like exercise regimes, food and water intake, heart rate, weight and sleep patterns. As well, these technologies can usually be configured to provide you with activity reminders, exercise and health milestone achievements, weekly reports as well as allowing you to share information with significant others. Why might this help you with stress management? By giving you control over certain aspects of your life that you may previously have been unable to control and perhaps helping you to place boundaries around what you do. Check out some possibilities: https://blog.sisuguard.com/howwearable-technology-is-transforming-health, https://thiswayup.org.au/12-freeapps-to-help-you-beat-stress/ and https://www.dummies.com/health/exercise/ what-is-wearable-fitness-technology-and-how-can-it-help-you/. It is important to note that such technologies can help you to build a bridge between the mental well-being and physical dimensions of your work/life balance (to be discussed below). Enjoy the breaks—When you do break for renewal, take care that it is a truly rewarding experience. McGee-Cooper and Trammell (1994, p. 164) refer to the concept of ‘contaminated time’. We contaminate time when we are not able to stay focused in the moment or when we are trying to do one thing but are thinking of another. If our time is contaminated, we don’t get the full benefit of work, play, or total relaxation. Contaminated time is when we feel guilty about taking a break, resting, being with friends or doing something different and this contamination undermines the benefits that we could realise. Contaminated time also occurs when you are working but you are too tired, too worried or too distracted to do anything really meaningful.

8.3 Mental Well-Being Dimension

8.3.6

231

What Should I Be Aiming for?

When you think of a continuum between boredom and extreme stress, midway there is an optimal performance zone. That’s where you would like to be. At that point, you are thinking clearly, being energetic and performing at your best. Frustrations will arise but remind yourself that developing your ability to cope with uncertainties in contextualising, framing, configuring, sampling, data gathering, analysing, interpreting and writing original research is part of becoming an independent researcher. In time, you will appreciate that the satisfactions derived from research are intimately associated with efforts to investigate, understand and reduce uncertainty (Finn, 2005, p. 8). Whether you interpret something as a stressor or as a challenge will depend on the way you appraise the situation and the extent to which you feel you can control it. A resilient individual tends not to fall off the deep end when a problematic situation arises, avoids being a drama tragic, and remains optimistic that there is a solution out there, it just needs to be found.

8.3.7

What Are Ways of Dealing with Frustration and Anger?

The following strategies can be useful in dealing with frustration and anger, i.e., being positive in the face of adversity: • • • • • • •

asking yourself, ‘am I over-reacting?’; not losing your sense of humour and seeing the funny side; recognising that set-backs are all part of the learning curve; turning mistakes into experiences and opportunities to learn; avoiding self-blame; keeping things in perspective—this is only one stage of your life; and remaining optimistic and enthusiastic.

Have you ever noticed that sometimes you are working well and are engaged, and time seems to fly by; it’s fantastic when you are in ‘the zone’. Alternatively, you may feel trapped in a vicious cycle where the more you work the more tired you get, the less creative or open-minded you are and the less you accomplish, the further behind you get and the more you don’t think you deserve the time off for other things. If this does occur and you find yourself becoming resentful, tired, overwhelmed, negative or discouraged, that is when you need to stop, stand back reflect, refresh and renew yourself. How you do that is up to you, but you do need to have an arsenal of activities that balance out what you are doing in relation to your research. These activities may be physical, going for a walk, a bike ride or a work-out, or it may be more spiritual, such as praying or doing meditation, or it could be purely entertainment, listening to music, watching a movie, or reading a book that has nothing to do with your research. If you know that you are a bit prone

232

8 How Do I Maintain a Good Work/Life Balance?

to excess in these down time activities, put a time limited on them, e.g., engage with Facebook for only 20 min. Take a moment to consider the activities you enjoy that could be utilised as part of the balancing act when you do find that you have pushed for too long and have become indifferent to your research journey. As a word of caution, for some, household chores and routine family activity, although not ‘work’, can be seen as another commitment, where an individual doesn’t receive the necessary refreshment and renewal that is required. You may, therefore, need to combine activities such as going for a walk with family and friends. You may find doing something entirely different from your routine will act as a refresher. One student I know took an evening drama course and found the physical and mental activities, as well as the engagement with people who knew nothing about her research, were distracting. Another volunteered at a charity. You may, therefore, wish to look at community noticeboards and newsletters in order to see what night-school classes are available. Usually they are of a fairly short duration, say, six weeks, and involve a few hours once a week, so may make an interesting diversion, as well as opening up some new connections. Chose a pursuit that will provide a contrast to your studies. Although you may find it difficult to give yourself permission to take the time out, the benefits may far exceed the cost or the time.

8.4

Physical Dimension

Curiously, constant work apparently destabilises our immune system. Therefore, maintaining your health is particularly important if you are going to meet your completion dates for each of the phases of postgraduate study. The concern for loss of time through ill health should be enough to motivate you to maintain your immune system by eating properly, sleeping well and avoiding excess consumption of stimulants.

8.4.1

How Can I Ensure My Physical Well-Being While Studying?

This discussion is going to sound painfully obvious as the strategies for maintaining one’s physical well-being have been around for ages. Your mother harped on about eating properly and all those self-help books seem to suggest similar approaches, so, how can we add value here? The first is by repeating those essential tactics, and the rest is up to you. Remember that, in our previous discussion of the mental well-being dimension, there may be useful wearable technological support devices and software you can use to help monitor/record aspects of your physical well-being (e.g., heart rate, sleep patterns), food/calorie and water intake and exercise. We suggest that you consider, from the following list of strategies, what

8.4 Physical Dimension

233

you are doing correctly and what you could improve on. Be honest with yourself and then set a 30-day plan (that is roughly how long it takes to forge a habit) to implement some changes.

8.4.2

What Are Some Strategies for Improving One’s Well-Being?

Be conscious of ergonomics As you will be spending a lot of time in front of a computer and at work stations, the last thing you want is a stabbing shoulder, wrist or neck pain that can significantly impede your concentration or, worse still, a repetitive strain injury (RSI) that can cause lasting damage, perhaps requiring physical therapy or even surgery. You may wonder how seemingly innocuous activities such as typing and clicking a mouse button could be potentially harmful, but they are. There are a number of suggested recommendations for improving the ergonomics of your workplace that are intended to alleviate poor posture, poor technique and overuse (Scott, 2015): • Where possible, position your workstation in such a way that glare from overhead or desk lights and/or the sun is minimised. Prolonged glare is very fatiguing. • Keep your feet flat on the floor, knees directly over feet, bent at right-angles or slightly higher, and pelvis rocked forward. • Keep your lower back arched in and supported by your chair (use a towel roll or cushion, if needed), make sure your upper back naturally rounded, your shoulders and arms are relaxed and at your side and that your neck is not jutting forward but rather upright. • Ensure your keyboard is positioned above your thighs. You should be able to reach the keys with your elbows close to your sides and bent with your forearms at a slightly downward slant from the elbow (for better blood flow). • When typing, keep your wrists straight, as the straighter your wrists are, the less strain you put on the tendons. Let your wrists float. The mouse should be placed to the dominant hand side of the keyboard and the mouse buttons configured for correct use by your dominant hand. You should be close enough that you don’t have to lean or stretch. • The monitor/screen should be directly in front of you, such that your eye-level is somewhere between the top of the screen and 20° down from the top. You should be looking slightly downward at your monitor/screen. The screen should be at least 30 in/approx 70 cms away from you. If it is too close. you will tend to hunch forward. • If you read papers, articles and reports on your monitor, make sure the font size is comfortable for long-term reading and that the page background is white with black text. You may also find it more comfortable to dim the brightness of your monitor, especially at night.

234

8 How Do I Maintain a Good Work/Life Balance?

• Your monitor should not flicker; if it does, you will find this very fatiguing and may induce headaches. If you are using a laptop, try to use a full-size keyboard and monitor attached as accessories for longer-term working comfort. • Set the zoom size for your word processor at a level where longer term screen work and reading of fonts is easy to do without eye strain (http://web.eecs. umich.edu/*cscott/rsi.html). To protect against RSI, take a moment after you have been doing a lot of typing, to rotate and flex your wrists and to stretch out your fingers. When you are next in a computer showroom, have a look at keyboards and mice that can protect against RSI. If you find that your wrists or hands hurt, you may definitely be in need of a new keyboard. Ergonomic keyboards look a little weird as they are slightly splayed and may take a bit of getting used to but, nevertheless, may be worthwhile. However, you should know that such keyboards have been optimised for touch typists. A standard keyboard may be best if you are a ‘hunt and peck’ typist. Make sure when you are setting up your monitor that you are looking straight at the screen, and not having to lean forward. The top of the monitor should be slightly below your forehead. This is in order to reduce the strain on your neck. Avoid crossing your legs when you are typing for long periods of time. Some people swear by footrests as a way of taking the pressure off your feet, legs and lower spine. Personally, I just kept on kicking it. Some people these days prefer to have a standing desk, so try a few options out to see if this works for you. Change your working location Your working space can play an important part in how well we feel about our work and how we use our time when we are supposed to be working. If you are finding that you are not being as productive as you need to be, consider whether your regular work space might be part of the problem. Are there too many distractions? Is it too noisy? Is it uncomfortable? See if finding a new regular working space helps. Schedule in short rest breaks Short rest breaks are where you get up from your work station and undertake exercises or go for a 10-min work around the block. Longer breaks are where you actually give yourself several hours or even a day off. Anticipating that time off may motivate you to work harder and having had that time off, you will hopefully feel more replenished and able to better resume your work. You cannot work 100% of the time; if you try, you will simply burn out. You need to give your brain a break from research in order to re-energise your capacities for intellectual and creative work. Do desk exercises You don’t necessarily have to move away from your office in order to get some exercise. Desk bound exercises will help you fight the neuromuscular fatigue that can significantly slow down your mental and physical performance. They help to keep you focused and alert, but they have to be done regularly and will involve standing and stretching. Work from your ankles through to your neck, rotating your joints and stretching your muscles. Think of the exercises that airlines recommend you do while flying.

8.4 Physical Dimension

235

Schedule a couple of 1-h exercise sessions It doesn’t have to be very energetic, it could be walking, gardening, bike riding or going for a run, but make sure you put in some significant time at least two or three times a week. Try massage Don’t under-estimate the power of a good neck and back massage. While they are readily available from professionals at a price and are as close as your nearest mall, for a cheaper version, ‘bribe’ your partner on a reciprocal basis. Giving a 10-min massage is worth it for the 10 min you will get in return. Sleep As we have already discussed, sleep is designed to help our body repair, re-organise thoughts and re-energise. We can do without it on a temporary basis but without adequate levels of sleep, our systems just do not operate as efficiently as they should. Covey (2013) reinforces the need for balance in his story of the saw. A man was vainly attempting to cut down a tree in the forest with a dull blade and making heavy work of it. It was suggested that he sharpen the saw, to which he replied he was too busy to sharpen the saw. It’s a simple story, but it sends a reminder of how we sometimes beaver away with blunt instruments, the blunt instruments being our body and our mind, when actually we sometimes have to take time out to refresh and enable us to achieve the task more effectively. Eat well Eating well is just as important as sleeping well. You would not think of putting a low-grade fuel in your car so why, if you are trying to get maximum performance out of your body, would you not put in the best ingredients to ensure it works well? A diet of fast food lacking in fruit and vegetables will undoubtedly slow your mental capacity. If you experience recurring dips in energy and concentration around 11.00 a.m. each day, the chances are that you are not eating an adequate breakfast. Include protein as well as carbohydrates in your first meal of the day, for example, by eating an egg, with cheese. If you need to top up during the day, try nuts which are very filling and minimise sugar-loaded treats or caffeine hits. Don’t overdo stimulants As we know, stimulants come in various forms and we are always amazed when students delude themselves into thinking that they work better having consumed their favourite stimulant (e.g., coffee, an energy drink): “But I can think so clearly …”. If you ever had to read the work of a student in this state, you can be assured that it is most definitely not their best work. They just think it is! Make it a rule not to mix stimulants and work. Leave stimulant consumption to leisure time; this includes coffee. You may also want to create a few parameters around social use. The problem is not the few beers or glasses of wine that you have in the evening, but how they can disrupt your sleep and slow you down the next day. You likely know what your own consumption patterns are, so consider where improvements could be made in order to help you work better.

236

8.4.3

8 How Do I Maintain a Good Work/Life Balance?

Why Is Exercise so Important, When All I Want to Do Is Work?

There is an anecdotal correlation between exercise and mental health. To stay on top of things, you also need to maintain your physical health. Exercise also provides a fantastic opportunity for reflection and for your subconscious to work on the problems you are currently facing in your postgraduate research. Repetitive exercise like running, walking, swimming, and cycling allow you time to think about the events of your day at work. It can become a time to connect with your feelings about work and gain a perspective about the day as well as planning for the next day. Apparently, exercise can also improve cognitive ability and it appears that moderate exercise will make you feel more alert and less tired. The trick is to remember how energised you felt when you exercised yesterday and do it again today. Be wary, though, of over-doing exercise as this can actually deplete your energy, reduce work performance and, at least for a time, weaken your immune system (https://health.usnews.com/health-news/blogs/on-fitness/2010/11/05/10-signsyoure-exercising-too-much, accessed 3rd Feb 2019).

8.5 8.5.1

Social Dimension I Am a Single International Student; How Does This Social Dimension Affect Me?

As often discussed, one of the more common difficulties experienced by students, particularly single international students, is a sense of isolation and solitariness (Wright & Lodwick, 1989) and the consequential inability to seek help or support to work through some of the many barriers that they encounter along the way. It has been found that single full-time doctoral students with no children actually faced more challenges in efforts to achieve a school-work-life balance in comparison to their peers with families (Martinez, Ordu, Matthew, Sala, & McFarlane, 2013). Social isolation can be particularly irksome for students who are used to frequent interaction with other students. The feeling of isolation is also particularly problematic for international students who are not only dealing with new material and experiences due to cultural variances but are isolated from their normal cultural support networks of family and friends. International students can experience a considerable amount of stress as they acculturate to a totally new environment. Social support has found to have a buffering effect on life stressors and in relation to international students and their management of stress with Mallinckrod and Leong (1992) finding that support from their families had a positive direct effect on stress symptoms and support from their academic programs had both direct and buffering effects. A recent meta‐analysis investigating the relationships between various types of social support and student burnout also showed results suggesting that school or

8.5 Social Dimension

237

teacher supports have the strongest impact on reducing to student burnout followed by support from parents and peers (Kim, Sooin, Lee, Sung, & Sang, 2018). Stress and isolation can also be very acute for students who are working from a distance, that is, they are registered with an institution, but are not on campus on a regular basis and live some distance away. For both domestic and international students, it is not just social isolation, but also intellectual isolation that may be experienced given that no-one else is studying your topic and your research is entirely of your own orchestration. Additional thought, therefore, needs to be given to mechanisms by which communication can be maintained and isolation avoided. Morales, Ntontis, and Kyprianides (2019) point to the importance of support provided by both the supervisor and faculty members in helping to avoid burnout and enhance engagement among doctoral students. They also found that a students’ identification with supervisors and faculty members together with clarity of role were positively associated with students’ work-related well-being. Hockey (1994) identified a feature which helped counter isolation and sustained students in the face of the trials and tribulations of the postgraduate research journey. It hinges on the formation of a research students’ sub-culture. Other postgraduate students can provide an immense amount of peer support for you, so it is worthwhile making connections with other students. Make specific arrangements to meet up with fellow students when on campus, make it a routine, every Wednesday afternoon, for instance, at 3 p.m. in the coffee shop. If you are seeing your supervisor weekly, plan to get together with your colleagues straight after that meeting. Some students like the idea of a study group where they just meet and informally share their experiences and get unsolicited advice. While postgraduate programs are similar for all students, the actual research content is clearly going to be different. As a consequence, some students find it difficult to discuss their research with others, although rehearsing your elevator pitch may help in this regard. You may wish to pull back from the detail and talk about some of the more general areas you are struggling with, for example, gaining access to a data source, how to pass a reluctant gatekeeper, organising your literature review and determining what to put in and what to take out of the literature review, the mindless task of entering data or reviewing transcripts. Finding common ground with issues you discuss may spark a conversation with a fellow postgraduate student. In doing so, you may be able to share experiences, or possibly pick up a good strategy for dealing with the issue and avoid some of the inherent isolation which is endemic in postgraduate study. My niece once came to have a chat with me as she was heading to visit Egypt. I dutifully provided her with my nicely typed up travelogue and regaled her with my experiences. After she left, I realised on reflection that while the information might be of value, she will have her own very different experiences. Similarly, with your postgraduate study, while it is useful to speak with others about the highs and lows of their research, the reality is that their journey isn’t going to be the same as yours. So, by all means, speak to others but perhaps phrase your questioning on a more positive note in order to generate ideas and tips. For example, in discussions with recently completed PhD students, pose questions such as if you had

238

8 How Do I Maintain a Good Work/Life Balance?

undertaken your research again, what would you have done differently? What is the best piece of advice you could give to a new researcher embarking on a PhD, etc.? This type of question will avoid the complaint of how hard and how difficult it was. You are not interested in how high or how wide, but how to actually cross the river without drowning or getting stuck on the rocks! Some students benefit from a more formal approach where they circulate pieces of work for critique and review. It may, in fact, not be necessary to meet face-to-face but merely to have email interactions. If you are studying at a distance, your department may be able to make provisions to set up a virtual discussion board or a chatroom viaa learning management platform (e.g., Moodle or Blackboard) or other social networking platforms (e.g., Adobe Connect, Zoom) for your research group. On the other hand, some students actually enjoy the independence of their study. The trick is to find out what suits you and to recognise when the particular approach you are using is no longer working and when you might be losing motivation, or are possibly becoming somewhat dysfunctional, that is, too stressed and unhappy to function efficiently. As a word of caution when interacting with other postgraduate students, being around other very driven and intelligent individuals may also eat away at your confidence. Keep in mind that they are going through the same journey as you are and hitting similar obstacles. They may look as if they are on top of it but, deep down, they are often struggling with some key issues. Maintain your self-esteem by not being hard on yourself; negative self-talk doesn’t assist anyone (Covey, 2013). For further discussion on establishing an organisational framework for dealing with social isolation to minimise postgraduate attrition, see Ali and Kohun (2007).

8.5.2

I Am a Married Student; How Does This Social Dimension Affect Me?

Well, there is good news and bad news here. Encouragingly, when investigating whether marital relationships of doctoral students were affected while they were enrolled in graduate programs, Brannock, Litten, and Smith (2000) concluded there were no significant differences among the marital satisfaction levels of graduate students at different stages in their program. Spouses who also were students scored significantly higher in marital satisfaction than spouses who were not students. Studies have shown that students with supportive spouses, ready to give encouragement and logistical support, have a better chance of finishing than single students (Peters, 1997, p. 284). However, the bad news is that postgraduate study can be particularly hard on one’s spouse or partner because of the need to commit such large chunks of time to the endeavour. Being holed up in your office and absent, yet again, from another Saturday evening or family event can become annoying and, just as you may be experiencing a sense of isolation and loss of your normal life, so too may your partner. To counter this, remember that relationships are like plants,

8.5 Social Dimension

239

they need to be regularly tended, so don’t fall into the trap of thinking, when this is over, I will be able to concentrate on my partner and give them the time that they need. I will make it up to them then… Sorry, but this will not wash. They still need to have your attention for the full duration of your study, obviously, not to the same extent they did before or will have after, but they do require your consideration and you need to set time aside to be with them. Put a date night into your planning schedule, be creative (take-aways with a tablecloth and candle at the park/beach, regularly go to gym or take a walk together), don’t resent the time away from your study, catch up on what has been happening and plan for the next date night. Work hard to maintain the connection or you will lose it.

8.5.3

I Am a Student with Children; How Does This Social Dimension Affect Me?

Undertaking postgraduate study with children is undoubtedly difficult but can and has been done before; it just requires a lot more planning and flexibility. In a study by Leonard, Becker, and Coate (2005) of British doctoral students, half of the respondents had children living at home while they were studying, and half of them had very young children. If you think you can study with young children running around you, I admire your powers of concentration! Lesser mortals find it virtually impossible and are either driven mad by the frequent verbal or physical interruptions or experience an attack of guilt because they feel they are ignoring and neglecting their children. For postgraduate students with children, scheduling becomes really important. Uninterrupted study time is usually only possible when the children are asleep. Research on this area has noted that doctoral students in this circumstance often preferred to study when the children were in bed with study hours of 9 p.m.–2 a.m. (Leonard, Becker, & Coate 2005, p. 375). Find study times that suit you. As one student commented, I chose to get up at 4 am to study, to have quiet time and to be available to the family in the evenings. I gave up TV almost entirely and worked library time around the schedules of others. The entire process required juggling to maintain roles of school administrator, wife, mother and student. I managed with less sleep and a sense of humour. (Dinham & Scott, 1999, pp. 53–54)

Investigate a variety of possible childcare arrangements. When your child is in care, resist the temptation to do chores around the house; they can wait. This is valuable study time so, don’t squander it. Specifically, in relation to female postgraduate students, research suggests that female doctoral students struggle with their well-being, including managing role conflict due to their multiple roles, have difficulty developing coping skills, maintaining social support. They were also found to perceive their well-being as an individual and social process that is constantly evolving and that each woman

240

8 How Do I Maintain a Good Work/Life Balance?

determined their own best personal practices to balance the multiple roles they played (Hayes et al. 2012). Postgraduate students who have a partner and a family are under considerable pressure. While postgraduate study has the potential for damaging yourself, it also has the potential to have a negative impact on others around you. You, therefore, need to be very conscious of this possibility and consider strategies you can use to avoid affecting personal relationships. Ways by which this might be achieved are: • Get commitment—Gain an early commitment from family and friends to your goal of a earning a postgraduate degree and enlist their support. • Communicate—Be realistic about the commitment of time needed and discuss this with everyone who matters. • Negotiate—Discuss, negotiate and agree about time allocations for your research, your partner, your children and for some personal time for you. Also negotiate some rules surrounding your work space for when you are working on your research. • Schedule—Develop a schedule to accommodate what you have agreed with your partner/family. • Stick to it—Honour the times that have been designated specifically for them; don’t steal their time. • Put a bit of thought into it—For designated partner/family times, make sure they are enjoyable activities for both you and them—it doesn’t have to be expensive —such as making pancakes on Sunday morning together, then going for a walk or to a park. Be creative; look up free local public events that you could attend together. One postgraduate student would take her children on the weekend to the school fairs and while there would pick up a second-hand toy or book. • Be in the moment—When you are with them, be ‘in the moment’ and enjoy them and the activity that you are sharing. • Recalibrate—Check in every so often as to whether your schedule needs changing, particularly around holidays. • Update—Keep your partner/family abreast of where you are in your studies. • Celebrate—When you are going through one of your research highs, share that with them—communicate the positive experiences to them. • Let them help—When you hit a research low, don’t ignore your partner/family, enlist their active support by asking them to be sounding boards or your emotional anchor. Let them help you through the rough patches. • Appreciate—Express your appreciation for their support by saying thank you often. For additional informative texts, especially for mothers in academia trying to find their work/life balance, see Connelly and Ghodsee (2011) and Evans and Grant (2008). In addition to maintaining your social networks, in order not to become too isolated, a novel suggestion by one PhD student is that socialising may actually improve your time management suggesting

8.5 Social Dimension

241

there is no better booster of your time management than having plans after work. If you know you need to leave office/lab at the certain time suddenly your productiveness increases significantly. PhD students without plans for the rest of the day very often tend to stay until late hour, being tired and slow which may have a very bad impact on work-life balance and mental health. (Agata, 2017, 6 Time Management Tips for PhD Students, http://blogs.nottingham.ac.uk/studentlife/2017/11/01/6-time-management-tips-phdstudents/, accessed 27th Jan 2019).

8.6

Spiritual

Have you ever looked at a beautiful picture, a magnificent view, or listened to a piece of music and been moved? As a consequence, something inside you just settled, and you felt a little better because of it. If you have had that sort of experience, that was you interacting with your spiritual self. The reason that spirituality is so personal is because the interaction is so internal, so subtle and stimulated by a myriad of different sources all unique to yourself and no-one else. For some, that connection is made by hiking up a mountain to admire the vista, while for the more sedentary, it could be just having some quiet time on your own, listening to music, a meditation tape, or engaging in prayer. As an adult you will know what activities enable that connection with your spiritual self, but further thought could be given to how this element could be enhanced. This would involve two strategies. The first is to reflect on what existing catalysts work for you and then to experiment with an open mind and to consider a few that may be new to you. Ever tried yoga? Give it a go. Do a trawl of the internet for activities that you could learn or do, or possibly just stick with the practices that you know nurture you. The second strategy is to incorporate some of these activities into your life. You will be pleasantly surprised not only at how little time they actually take but also how beneficial they could be to your well-being. However, to receive the benefits you must allow sufficient down time to enable the connection between you and your spiritual self to be made or refreshed. You cannot rush it or fake it. As a postgraduate student, you deserve some time to call your own; you shouldn’t regard it as a luxury but part of your conscious efforts to maintain your wellbeing.

8.7

Conclusion

Apparently, being a postgraduate student is like becoming each of the seven dwarves. In the beginning, you are Dopey and Bashful; in the middle you are usually sick (Sneezy), tired (Sleepy) and irritable (Grumpy); and at the end they call you Doc, and then you are Happy (Azuma, 2017). Undertaking postgraduate research can be like an emotional and intellectual rollercoaster with the associated highs and lows or, as one PhD student described it, a busy mental and emotional journey:

242

8 How Do I Maintain a Good Work/Life Balance?

in this research project, the places I have visited are not physical locations, because the goal of a thesis is indeed to draw separateness into a meaningful whole. The travel is thus a busy mental and emotional journey of comings and goings that include false starts, wrong timetables and getting lost, balanced by exciting discoveries and serendipitous encounters, all free of physical constraint but embodied experiences nonetheless (Meldrum, 2008, p. 50).

Student well-being is seen as a critical determinant of future completion success (see, for example, Pyhalto, Toom, Stubb, & Lonka, 2012; Sverdlik, Hall, McAlpine, & Hubbard 2018; Vekkaila, Pyhalto, & Lonka, 2013). In a study examining the mental health needs, knowledge, and utilisation of counseling services among graduate students at a large university in the western United States, Hyan, Quinn, Madon, and Lustig (2006) observed that almost half of graduate student respondents reported having had an emotional or stress-related problem over the past year and they knew of a colleague who had also had an emotional or stress-related problem over the past year. Self-reported mental health needs were significantly and negatively related to confidence about one’s financial status, higher functional relationship with one’s advisor, regular contact with friends, and being married. In their research. Martinez, Ordu, Matthew, Sala, and McFarlane, (2013) determined that full-time doctoral students strived to achieve a school-work-life balance by (a) purposefully managing their time, priorities, and roles and responsibilities; (b) seeking well-being by managing stress levels, maintaining their mental and physical health, and creating personal time; (c) finding support from various individuals and their institution; and (d) making trade-offs between the various demands they encounter. The trick is therefore to remain primarily both mentally and physically healthy throughout the years of your postgraduate research journey. This is not to say that you will not have down times and periods when you are extremely frustrated, disappointed and confused. That is normal. When the emotions become extreme and result in dysfunctional behaviour which either stops you from working or damages your relationships with other people, things have gone too far, and you end up in a danger zone. You will need to stop and address the situation. Learn to listen to your body and to your emotions. When you find yourself starting to slip, get despondent, stressed or depressed, you need to take a break from the activity in order to reflect, build up your resilience, or ‘grit’ as it is also referred to (Hodge, Wright, & Bennett, 2018). Then return to the task at hand with a more appropriate and constructive attitude to the problem. This is not always easy to do when you are in the thick of it. For example, we have all been in the circumstance of forgetting to save a file, accidently hitting the delete button, losing data or being bluntly critiqued. However, by taking a break from the situation, realising that things happen, stepping up to the problem (e.g., realising that unsaved material is not going to magically re-surface and that you should have saved it appropriately) and then biting the bullet and getting back into it, you will get one step further towards your long-term objective—your goal of completion. As frustrating and potentially devastating as they are, these are problems that just need to be dealt with. The more significant problems come from the sheer enormity, complexity and length of postgraduate research project where you need to watch for signs of stress

8.7 Conclusion

243

and burn out. How do you know when you are getting close to it? Your body or your head will tell you. If it’s your head telling you, you will probably be somewhat despondent, bored or resentful of your research and on the verge of giving up on it. If it’s your body telling you, usually you will experience illness, fatigue and general malaise. These are signals that your ability to produce, that is, the engine driving you intellectually and physically needs to be replenished. If you need to make the journey, just think of it as an old, coal-fired train heading up the hill, unless you keep stoking it, it is not going to be able to make it to the top. We sometimes wonder why it is just chugging along or virtually stopping. When this happens you need to stop, possibly take a break, re-schedule your work in some way, or refresh it through a change of task or routine. Better still, try to avoid getting to that point. Look at the five dimensions of work/life balance and see if you have any significant current imbalances. You may have inadvertently dropped the things that give you enjoyment and provide some insulation from your study. Now consider what you can do about it. What actions can you take to address the situation? The undertaking of postgraduate research spans a considerable period of time. The work will always be there, and you need to set up appropriate practices early on in order to protect yourself and your family from what can become, for some, over-zealous and excessive behaviour. We would all like to complete the project within the minimum time, but at what cost to yourself, to your health, to your well-being and to the well-being of others around you? All too often, students view the writing up of their thesis, dissertation or portfolio as the final goal of their journey, yet it is important to realise that it is not just the end result. It is a reflection of what you had to do and go through in order to achieve it and this is what truly constitutes the learning experience. Writing up and submitting your major research outcome does not define you as a researcher, it is how you negotiated your entire journey. There is an undercurrent of learning occurring throughout the research process. Sometimes, in the headlong rush to get things down on paper, to complete the chapter, to finish the thesis, you don’t appreciate the full benefit of the research exercise. At the beginning of your research journey, you may wish to reflect on what you hoped to achieve from a learning perspective. In doing so, you may, in fact, relax a little more when the odd curve ball is sent your way, knowing that it is part of the evolutionary process and that the curve ball may, in fact, improve your game. To finish up here, consider a few apps that you might find useful in your pursuit of work/life balance: • Headspace (https://www.headspace.com/)—This app can help you to clear your head thereby making living mindfully and meditating easier. • 7-Min workout (https://au.wahoofitness.com/fitness-apps/7-minute-workout)— This app aims to help you get in at least 7 min of exercise by providing 12 high intensity bodyweight exercises, 30 s per exercise. • Tomato timer (https://tomato-timer.com/)—We recommended taking periodic breaks when working, but not for too long. This app is a timer you can use to schedule both short and long breaks.

244

8 How Do I Maintain a Good Work/Life Balance?

• f.lux (https://justgetflux.com/)—This app can reduce or eliminate the eerie blue glow from your computer monitor/screen so that it adapts to the time of day or night, thereby reducing fatigue.

8.8

Key Recommendations

• It is important to adopt a positive mental attitude and to be enthusiastic. New challenges give you a chance to tackle new tasks and acquire new skills. • If you are experiencing problems, get support—it could come in the form of a partner, your supervisor, a fellow student, childcare or even a cleaner. • Seek professional counselling if things start to really get on top of you. • Stop yourself from worrying when you’re at a vulnerable time. The end of the day when you are tired is not a time to start worrying about things. • If you are going to take a break, protect the time you have allocated for your day (s) off to do something different. Take the time off and do not feel bad or guilty. This is your way of re-generating yourself. • Use your research journal. Dump all your thoughts, concerns and floating ideas into it. • Be organised. Work from your To Do list, rotate tasks so that there is a variety of jobs and/or try different working hours and locations. • Be kind to yourself. Try to avoid engaging in negative self-talk—remove ‘I should have’ from your vocabulary. • Say ‘no’ and simplify your life. Better to say no now than to disappoint later when you have to drop a project (and don’t wait too long to do this as they may be counting on your assistance). • If you are a full-time student, try to adopt a routine working day, that is, 8.30 a.m. – 6.30 p.m. and then take time off to enjoy yourself with friends or family on at least one day on the weekend. • Plan quiet time each day to think, meditate, practise yoga, tai chi or just blob out. • Keep your sense of humour. There is life after the postgraduate research journey, but, for now, this is it, so find the funny parts in it. • Look after your health and diet (avoid junk food; ensure you have good nutrition), avoid too many stimulants, exercise, sleep and, at the risk of sounding like your mother, eat breakfast. • If you do face difficulties, try to remain positive. Most things can be resolved and worked through with time, energy and insight. • Be appreciative of the things that are working and thank others who are helping you in the process. • Talk with your supervisor(s), peers and family members about issues that arise. Not only may they have informative solutions (particularly your supervisors), but those who are immediately involved with your project may be able to place things more in perspective for you. Yes, it is a postgraduate research project; yes, it is important to you, but it is not your sole purpose for being.

References

245

References Agata. (2017). 6 time management tips for PhD students. Retrieved January 27, 2019, from http:// blogs.nottingham.ac.uk/studentlife/2017/11/01/6-time-management-tips-phd-students/. Ali, A., & Kohun, K. (2007). Dealing with social isolation to minimize doctoral attrition—A four stage framework. International Journal of Doctoral Studies, 2(1), 33–49. Azuma, R. T. (2017). So long, and thanks for the PhD! Retrieved February 6, 2018, from http:// www.cs.unc.edu/*azuma/hitch4.html. Brannock, R. G., Litten, M. J., & Smith, J. (2000). The impact of doctoral study on marital satisfaction. American Counselling Association, 3(2), 123–130. Burns, T., & Sinfield, S. (2016). Essential study skills: The complete guide to success at university (4th ed.). London: Sage Publications. Connelly, R., & Ghodsee, K. (2011). Professor mommy: Finding work-family balance in academia. Lanham, MD: Rowman & Littlefield Publishers. Covey, S. R. (2013). The 7 habits of highly effective people: Powerful lessons in personal change (25th Anniversary ed.). New York: Simon & Schuster. Cryer, P. (2006). The research student’s guide to success (3rd ed.). Maidenhead, UK: McGraw-Hill Education. Dinham, S., & Scott, C. (1999). The doctorate: Talking about the degree. Kingswood, NSW: University of Western Sydney, Nepean, ERIC, ED435305. Dow, M. (2018). Healing your drained brain. Carlsbad. CA: Hay House Inc. Evans, E., & Grant, C. (Eds.). (2008). Mama, PhD: Women write about motherhood and academic life. Piscataway, NJ: Rutgers University Press. Finish Your Thesis. (2018). Retrieved December 18, 2018, https://finishyourthesis.com/timemanagement/. Finn, J. A. (2005). Getting a PhD: An action plan to help manage your research, your supervisor and your project. London: Routledge. Gardner, S. K., & Gopaul, B. (2012). The part-time doctoral student experience. International Journal of Doctoral Studies, 7, 63–78. Haynes, C., Bulsan, M., Citty, J., Grant-Harris, M., Hudson, J., & Koro-ljungberg, M. (2012). My world is not my doctoral program … or is it?: Female students’ perceptions of well-being. International Journal of Doctoral Studies, 7, 1–18. Hockey, J. (1994). New territory: Problems of adjusting to the first year of a social science PhD. Studies in Higher Education, 19(2), 177–190. Hodge, B., Wright, B., & Bennett, P. (2018). The role of grit in determining engagement and academic outcomes for university students. Research in Higher Education, 59(4), 448–460. Hyun, J. K., Quinn, B. C., Madon, T., & Lustig, S. (2006). Graduate student mental health: Needs assessment and utilization of counselling services. Journal of College Student Development, 47 (3), 247–266. Kehm, B. M. (2005). Developing doctoral degrees and qualifications in Europe: Good practice and issues of concern—A comparative analysis. In J. Sadlak (Ed.), Doctoral studies and qualifications in Europe and the United States: Status and prospects (pp. 279–298). Bucharest: UNESCO. Kim, B., Sooin, J., Lee, J., Sung, H. A., & Sang, M. L. (2018). Relationships between social support and student burnout: A meta-analytic approach. Stress and Health, 34(1), 127–134. Leonard, D., Becker, R., & Coate, K. (2005). To prove myself at the highest level: The benefits of doctoral study. Higher Education Research & Development, 24(2), 135–149. Mallinckrod, B., & Leong, F. T. L. (1992). International graduate students, stress, and social support. Journal of College Student Development, 33, 71–78. Martinez, E., Ordu, C., Matthew, R., Sala, D., & McFarlane, A. (2013). Striving to obtain a school-work-life balance: the full-time doctoral student. International Journal of Doctoral Studies, 8, 39–59.

246

8 How Do I Maintain a Good Work/Life Balance?

Mason, M. A., Goulden, M., & Frasch, K. (2009). Why graduate students reject the fast track. American Association of University Professors. Retrieved January 28, 2019, from https:// www.aaup.org/article/why-graduate-students-reject-fast-track#.XFAJ4MaP5us. McGee-Cooper, A., & Trammell, D. (1994). Time management for unmanageable people: The guilt-free way to organize, energize, and maximize your life. New York: Bantam. Meldrum, R. J. (2008). A curriculum for entrepreneurial creativity and resourcefulness in New Zealand. Unpublished PhD thesis, Deakin University, Melbourne. Morales, S., Ntontis, P. J., & Kyprianides, A. (2019). PhD supervisors and faculty members might help to avoid burnout as well as enhance engagement and organisational citizenship behaviour (OCB) among PhD students. Technical Report. University of Sussex, Retrieved January 28, 2019, from http://dx.doi.org/10.20919/Psych(2019).001. Pearson, M. (1999). The changing environment for doctoral education in Australia: Implications for quality management, improvement and innovation. Higher Education Research & Development, 18(3), 269–287. Peters, R. (1997). Getting what you came for: The smart student’s guide to earning a Master’s or a PhD (Rev ed.). New York: Noonday Press. Pyhalto, K., Toom, A., Stubb, J., & Lonka, K. (2012). Challenges of becoming a scholar: A study of doctoral students’ problems and well-being. ISRN Education, 1–12, Article ID 934941. Race, P. (2007). How to get a good degree: Making the most of your time at university (2nd ed.). New York: Open University Press. Ribeiro, I. J. S., Pereira, R., Freire, V. I., de Oliveira, B. G., Casotti, C. A., & Boery, E. (2018). Stress and quality of life among university students: A systematic literature review. Health Professions Education, 4(2), 70–77. Sauser, K. (2012). Exhausted & drained? It’s not just in your brain: Identify and heal from adrenal stress and fatigue to regain your energy. Indianapolis, IN: Dog Ear Publishing. Schmidt, M., & Umans, T. (2014). Experiences of well-being among female doctoral students in Sweden. International Journal of Qualitative Studies on Health and Well-Being, 9(1), 23059, https://doi.org/10.3402/qhw.v9.23059 . Scott, C. (2015). Repetitive strain injury. Retrieved Feburary 26, 2018, from http://web.eecs. umich.edu/*cscott/rsi.html. Sverdlik, A., Hall, N. C., McAlpine, L., & Hubbard, K. (2018). The PhD experience: A review of the factors influencing doctoral students’ competitions, achievement and well-being. International Journal of Doctoral Studies, 13, 361–388. Vekkaila, J., Pyhalto, K., & Lonka, K. (2013). Experiences of disengagement—A study of doctoral students in the behavioral sciences. International Journal of Doctoral Studies, 8, 61–82. Waring, M., & Kearins, K. (2013). Thesis survivor stories: Practical advice on getting through your PhD or masters thesis. Wollombi, NSW: Exisle Publishing. Webb, S., & Ott, B. (2005). Effective organisation and time management. In K. L. Allen (Ed.), Study skills: A student survival guide (pp. 3–18). Chichester, UK: John Wiley & Sons. Wen-Chih, T., & Newton, F. B. (2002). International students’ strategies for well-being. College Student Journal, 36(4), 591–598. Wright, J., & Lodwick, R. (1989). The process of the PhD: A study of the first year of doctoral study. Research Papers in Education, 4(1), 22–56. Wright, T. (2006). Issues in brief counselling with postgraduate research students. Counselling Psychology Quarterly, 19(4), 357–372.

Chapter 9

Why Should I Think About Guiding Assumptions?

9.1

What Are Guiding Assumptions?

This is a critical question and forms an essential consideration for your research journey. The choice is tantamount to you clearly stating, ‘these are the assumptions I am willing to make to guide my research’. Discussions about guiding assumptions in social and behavioural research are heavily influenced by philosophical discourse (e.g., discussions of ‘ontology’—the nature of the world and reality, and ‘epistemology’—the nature of knowledge) and have traditionally been associated with Kuhn’s (1970) concept of a research ‘paradigm’. A research paradigm is a system of commonly-shared (at least amongst a cohort of researchers) assumptions and expectations for helping researchers decide what counts as the type of knowledge one seeks, what counts as something worthy of research in pursuit of such knowledge and what constitutes ‘good research’ (i.e., quality criteria) for generating that knowledge. [For example, in the natural sciences (e.g., geology, chemistry, physics, astronomy, biology, botany, zoology and so on), the dominant research paradigm is called ‘positivist’ (sometimes ‘post–positivist’ or ‘normative’) and embodies what most people have learned through their early years of education to refer to as the ‘scientific method’ (sometimes called the ‘hypothetico-deductive approach’).] However, in social and behavioural research, it turns out that there are potentially many different paradigms or sets of guiding assumptions you could adopt in pursuit of relevant knowledge and understanding (see Neuman, 2013, Chap. 4, for a very clear discussion of paradigms and their implications; Crotty, 1998, provides a very coherent discussion of the philosophical underpinnings of social research paradigms; Lincoln, Lynham, & Guba, 2018, provide an excellent critical discussion of paradigm assumptions and controversies). In each case, the role of a paradigm is to provide you with (1) a coherent set of assumptions that can be used to guide your research focus, methodology, data type and data source choices and activities and (2) a perspective that allows you to critically evaluate the strengths © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_9

247

248

9 Why Should I Think About Guiding Assumptions?

and weaknesses, opportunities and constraints that adopting that particular set of guiding assumptions affords you. Often, paradigms and their attendant guiding assumptions are given a covering label, such as interpretivist, constructivist, positivist, critical, feminist, critical realist, indigenous, post-modernist and so on; each label intended to convey something central about the paradigm. For a very long time, paradigms were viewed as being in competition with and critical of each other, often to the point where one paradigm defined the dominant approach taught and implemented within specific disciplines. For example, in the disciplines of medicine, psychology, marketing and economics, the positivist paradigm remains the dominant set of paradigm assumptions implemented by many researchers. The interpretivist or constructivist paradigms tend to dominate much educational and sociological research. However, in these as well as other disciplines (e.g., management, public policy, nursing and health, criminology), a shifting balance in focus can be observed. We advocate a more systemic and deliberately pluralist perspective, one that values the potentials as well as acknowledges the limitations of a range of paradigm guiding assumptions. We emphasise complementarity and synergy rather than competition or rivalry. We will also try to look behind the labelling of paradigms by focusing on the pivotal questions you must ask yourself when trying to pin down just what guiding assumptions you will adopt. It is the pattern of choices made in your answers to these pivotal questions that creates the association with a specific paradigm label. We will argue that there is no one right set of guiding assumptions to implement, it is a choice you deliberately make and, in making that choice, there then follows a cascade of more focused methodological implications and consequences. We will see that achieving clarity in one’s guiding assumptions has clear benefits in terms of narrowing and focusing the field of choices confronting you and offering some criteria relevant for assessing the quality of research guided by those assumptions. We will also see that your adopted guiding assumptions may introduce blind spots that you must grapple with. Thus, you must take the good with the bad with each pattern of guiding assumptions and, for this reason, research anchored by only a single set of guiding assumptions must be viewed as necessarily partial. Importantly, as a consequence, you need not be restricted to a single set of guiding assumptions for the entirety of your research journey. This is one important aspect of the concept of ‘pluralism’ that we employ throughout this book. As we argued above, guiding assumptions are those assumptions you explicitly or implicitly adopt when undertaking your research journey. In Table 9.1, eight pivotal questions are presented along with a range of choices or positions you could adopt with respect to each question. The choices shown should not be considered exhaustive of the possibilities as there is always room for growth, evolution and creative insight with respect to guiding assumptions. The first six pivotal questions in Table 9.1 can help you to identify the pattern of guiding assumptions that makes most sense for your research: Control, Perspective, Locus, Stance, Theory and Learning.

High: High or full control over what happens in the research context, as in a laboratory

Partial: At least partial control over what happens in the research context as in a field experiment, intervention or evaluation of an innovation

Researcher: Your perspective, as researcher, should drive your research process (i.e., your ideas are privileged and will be centrally tested) meaning that power over conceptualisation and theorising is concentrated in your hands

Participants: The perspectives of participants and other data sources should drive your research process, and therefore the balance of power over conceptualisation and impact on theorising is shifted toward participants (i.e. your perspective, as researcher, becomes subordinate to those perspectives emerging from or reflected by your data sources)

Externalised reality: The locus of ‘social/behavioural reality’ where the data patterns/relationships/meanings (including any causal inferences) to be understood are considered to be exterior to and generalisable from any specific individuals/groups (i.e., a nomothetic or ‘average person’ interest)

Internalised realities: The locus of ‘social/behavioural reality’ where the data patterns/relationships/meanings (including any causal inferences) to be understood are considered to be interior and relative to specific individuals/groups (i.e., an idiographic or individualist interest)

Hybridised loci: There are internalised views of realities that can be mapped, at least to some extent, onto phenomena external to the individual, which means that some elements of an externalised reality can be at least partially understood and that it is possible for some internalised views to be fallible

Balanced: Your perspective, as researcher, and the perspectives of participants and key stakeholders/users should all have some impact on driving your research process (i.e., power over conceptualisation and theorising in your research process is distributed)

Minimal: Minimal or no control— control over what happens in the research context is infeasible or to be avoided; context and the events to unfold within it are to be taken as given without your intervention

Possible Choice

Objectivist: You, as researcher, wish to approximate an objective disinterested observer who is

Subjectivist: You, as researcher, wish to adopt a subjectivist stance, that of an interested and empathic researcher

Critical/Emancipatory: You, as researcher, wish to adopt a Critical/ Emancipatory (i.e., a change/

4. Stance: What will your value-oriented observational stance as a researcher be with respect to what and whom you are researching?

Possible choice

3. Locus: Where will the locus of ‘reality’ reside for purposes of your research?

Possible choice

2. Perspective: Whose perspective should drive your research process?

Possible choice

1. Control: To what extent do you need to be able to exercise control over your research context(s)?

Guiding assumptions pivotal question

Table 9.1 Eight pivotal questions for identifying a pattern of guiding assumptions for your research

(continued)

Pluralist: You, as researcher, wish to adopt a position that acknowledges there is an important interplay to be

External: External agency/forces control what happens in your research context as when a policy change is enacted, or a natural or societal event/ disaster occurs

9.1 What Are Guiding Assumptions? 249

who is value-neutral yourself but interested in the value positions of others; in other words, where you, as far as possible, attempt to keep your own preconceptions from influencing your achievement of understanding of other’s perspectives and where no perspective is seen as better or worse than any other or as being right or wrong. Here, you are seen to be a co-creator of data in conjunction with participants

Apriori: Theory takes the form of logical and systematic arguments that identify key constructs, patterns and relationships of interest to you; you set out these arguments prior to the conduct of your research, thereby providing the basis for deducing research questions and/or hypotheses to be tested. Primary logical processes are deduction (from theory to testable hypotheses) followed by induction (from observations to refining the more general theory)

Emergent: Theory takes the form of a story or some other description of how meanings, patterns and relationships are co-created and sustained in context; theory is intended to emerge as a consequence of your research instead of being formulated apriori. The primary logical process is induction from data to more general theory

Critique: Theory takes the form of a story or critique that tries to show true conditions and power relationships in a context and provide guidance for people to take action and produce change for a better world. The primary logical process is critical argumentation

liberating/action-oriented) stance; you are an active, often participatory, change agent whose position is value-laden; in other words, where you deliberately seek to influence and facilitate or foment change in power relationships (hence the relevance of emancipation) and where particular viewpoints are seen or shown to be privileged or marginalised (encompasses indigenous or feminist research orientations as well) or where change/improvement is deliberately being sought)

Description: Theory has minimal formal role in your research; your goal is to provide characterisations and descriptions of events, policies and practices, persons, groups, organisations, institutions, governments, and/or communities

understood between the objective manifestation of reality (e.g., cause-effect relationships) and the subjective perceptions of that reality. Thus, both stances need to be explored in order for a more complete understanding to be achieved

Possible Choice

Explanation and Prediction: Learning/knowledge from your research will enable explanation of

Deep understanding: Learning/ knowledge from your research will enable deep understanding and sharing of worldviews

What works/What and How to change: Learning/knowledge from your research will enable actions, changes and improvements to be planned,

(continued)

What is/Patterns of association: Learning/knowledge from your research will enable insights into what is or has happened to people, groups,

6. Learning: What is your downstream intention for learning and knowledge produced by the research you conduct (i.e., how will you use your learning and knowledge)?

Possible Choice

5. Theory: What is the role of theory with respect to your research intentions?

value-neutral; that is, where you, as far as possible, cannot influence the data collected and where no findings are seen inherently good or bad, right or wrong. Here, you are seen as having a passive role, once data gathering has commenced

Guiding assumptions pivotal question

Table 9.1 (continued)

250 9 Why Should I Think About Guiding Assumptions?

assessed and evaluated and/or decisions to be made

Mode 1: Knowledge building occurs relatively independently of interactions and engagement with or inputs from key stakeholders and end users (prototypical classical discipline-centred academic/ university research). Here, your research will serve to achieve discoveries and you intend to disseminate what you have learned via peer-reviewed outlets such as academic journals and other academically-respected forms of research outcome (e.g., a thesis/ dissertation/portfolio, conference paper)

Mode 2: Knowledge building depends heavily, if not completely, upon high-level interactions with and inputs from key stakeholders and end-users; that is, knowledge is co-created and participatory (inputs may be physical, emotional, cultural, intellectual, resource-based, power/control over access to contexts, data sources …). Here, your research will have a practical and user-focused outcome or intention, such as facilitating the development and adoption of innovations or changes in practices

Culturally-privileged: Knowledge is seen as culturally and historically anchored, relational and privileged and permeates all aspect of life, the cosmos and relationships; you, as researcher, work to gain access to this privileged knowledge for potential sharing with the wider community or for creating additional benefits to communities

organisations and so on, often in the context of a specific situation as well as identifying patterns of associations between/within key entities

Possible Choice

Fixed: You intend to adopt a fixed or single consistent set of guiding assumptions throughout your research process Varying: You will deliberately plan and implement different sets of guiding assumptions at different stages of your research

Flexible: You will maintain a more flexible stance on guiding assumptions, evolving/changing them to meet emergent research needs

Creative: You will look to move beyond paradigm-defined constraints on guiding assumptions through creative re-imagining of patterns of guiding assumptions

8. Assumption Diversity: To what extent will you adhere to a singular set of guiding assumptions or be open to adopting multiple/diverse sets of guiding assumptions (i.e., adopt a pluralist attitude) through the stages of your research?

Possible Choice

7. Knowledge Building: What mode of knowledge building best suits your research intentions?

observable phenomena and prediction of future phenomena

Guiding assumptions pivotal question

Table 9.1 (continued)

9.1 What Are Guiding Assumptions? 251

252

9 Why Should I Think About Guiding Assumptions?

The first question concerns Control. When we consider the connection between your context(s) and the research context, one key assumption is the extent to which you need to be able to influence or change what happens in the research context. For certain types of research, it is essential that you be able to control what happens to data sources in a research context, particularly if you want to gather evidence for externalised cause-effect relationships (i.e., causal relationships that are externally verifiable rather than perceived). The issue of control is of paramount concern in natural sciences research and when we import this natural sciences logic into the social and behavioural sciences, control becomes of paramount concern as well. It is through exercising control that you attempt to ensure that only the causal factors of interest have changed when you observe or measure the effects of interest. Full researcher control is only really achievable in a laboratory and in social and behavioural research, you can be assured that even in a laboratory, control cannot be complete where humans and their idiosyncrasies are concerned. As soon as one moves out of a laboratory, capacity for you to control the research context declines dramatically, which makes externalised cause-effect relationships that much harder to convincingly demonstrate. The problem becomes more acute in cases where some external agency controls/determines what happens in the research context, rather than you, as the researcher. The second question concerns Perspective. Here, we consider the major driving perspective behind the research. You may choose to conceptualise everything about the research and pre-configure all aspects of your research to be consistent with that conceptualisation prior to gathering data. Alternatively, you can choose to remain in the background, to some extent, opting instead to try and learn about the perspectives of the participants (i.e., data sources) in the research. These are diametrically opposing approaches to addressing the perspective question: in one, your preconceptions count for everything; in the other, you try to withhold your own perspective so that you can form empathetic insights into/understanding of the perspectives of others. Of course, you can opt to try and maintain a more balanced or egalitarian approach to managing perspectives; a choice that is more challenging to implement, but in the end, may be what is required in order for your research to be convincing (as in, for example, research into the developmental evaluation and adoption of innovations, see Patton, 2011; Cooksey, 2011, for discussions of this approach). The third question concerns Locus. Here, we intersect the philosophical concept of ‘ontology’ or the nature of the world and reality. However, we need not deeply engage with this concept in order to make use of it. Basically, you can attack the issue of reality by asking where you are assuming it is located, for purposes of your research. If you assume that the story of reality you are pursuing lies exterior to any particular individual or group, then you are considering reality to be externalised. The implication here is that if you wish to look for externalised cause-effect relationships, you are interested in a generalisable picture, independent of any particular person. If, on the other hand, you adopt a more relativist positioning on locus by assuming that any story of reality you might gain access to is interior to each individual or group, then you are considering locus to be internalised. The

9.1 What Are Guiding Assumptions?

253

follow-on implication is that any question of cause and effect will always be relative to an individual’s or group’s perspective, i.e., their construction of cause and effect in their own life space. It is also possible to adopt a hybridised loci perspective whereby both external and internalised worlds and their interplay in context are of interest. Such a perspective is a key feature of the critical realism pattern of guiding assumptions (Sayer, 2000; Mingers, 2014). The fourth question concerns Stance. This is probably the most complex and multidimensional of the questions, invoking as it does considerations of values and observational/psychological distance (from data sources). An objectivist stance means that you try to maintain distance from, and minimal connectivity with, data sources, so as not to influence or taint the data they may be providing. At the same time, an objectivist stance implies value-neutrality in that you intend to make no judgments about the rightness, wrongness, goodness or badness of whatever you observe. In contrast, a subjectivist stance means that connectivity between you and data sources is explicitly sought so that understanding of different points of view can be achieved. This connectivity means that any data that result are essentially co-created between you and data sources. Simultaneously, the subjectivist stance also implies value-neutrality in that you intend to make no judgments about the rightness, wrongness, goodness or badness of whatever you learn, but you are interested in the value positions held or reflected by others. There is a third important stance that you can adopt and that is a critical/emancipatory stance. Here, you adopt an explicitly value-laden stance where change and improvement are desirable outcomes to pursue, where there are good and bad ways for social systems to work. Your role is to help bring true social conditions and power relationships into the light (revealing illusions) so that people can be liberated from restrictive social conditions (hence the emancipatory aspect) and ultimately improve their situation. This is a key feature of critical social science (Lincoln et al., 2018). A fourth important stance is a pluralist one where you accept that there is value in examining phenomena from an objective stance as well as from subjective stances in order to understand the interplay between them—another key feature of critical realism (Sayer, 2000). Intrinsic to the pluralist stance is that objective reality is only imperfectly perceivable and partially understandable by individuals or groups and that those subjective understandings and perceptions may be flawed. The fifth question concerns Theory. Here, we focus on the conceptualisations you develop and/or use during the course of your research. Theory has different meanings under different sets of guiding assumptions. One way of looking at theory is as a network of constructs and links between constructs that you assemble prior to the gathering of any data. Theory in this sense is called apriori and provides the means for deducing specific hypotheses that you wish to test. Apriori theory generally reflects your preconceptions about what you expect to show in your research. Once data have been gathered and relationships and patterns assessed, you then use logical induction to infer back to the theory and possibly refine it. Another sense in which theory may be taken is as emergent. Here, you do not start with a theory, rather theory emerges from gathered data as a way of accounting for why the data and attendant meanings and patterns unfolded as they did. This is a process of

254

9 Why Should I Think About Guiding Assumptions?

induction from specific data to a more general theoretical account (in grounded theory, early data-driven theory development can lead to deduced hypotheses which can be tested by gathering more contextualised data, see discussions in Charmaz, 2014). Thus, the emergent choice for theory means that theory is constructed from the data, not prior to the data. Theory may also emerge as a collaborative phenomenon between researcher(s) and participants, a key aspect of the participatory inquiry paradigm (Lincoln et al., 2018; Heron & Reason, 1997). A third possible sense in which theory may be taken is as a critique. A critique is a story or critical argument that illuminates the true nature of social conditions and power relationships with the intention of providing guidance for further action. Of course, it is entirely possible that you do not intend for theory to play a role in the research, especially if your intention is straightforward description. The sixth question, pertinent to identifying your desired pattern of guiding assumptions, concerns Learning. Here the focus is on what you intend to do with what you learn and with the knowledge you assemble from your research. One orientation is for using the knowledge you assemble and learning you achieve to enable explanation and prediction of social and behavioural phenomena. This is most often associated with a strong emphasis in showing externalised cause-effect relationships as once a generalised cause-effect relationship is understood, it can be used as a predictive rule as well as serving as an explanation for why things happen as they do. A second possible orientation for using the knowledge you assemble and learning you achieve is to provide and share a deep understanding of worldviews. A third orientation is to use the knowledge you assembled and learning you achieve to inform yourself and others about further actions and decisions that might/should be taken; thus, learning is transformed into action. Finally, a fourth orientation is to use the knowledge you assemble and learning you achieve to show others what is happening or has happened in a specific set of circumstances and contexts. Specific patterns of choices across the six questions are typically given labels as ‘paradigms’. Figure 9.1 shows six concrete examples (again, not exhaustive), where a line connects specific guiding assumption choices (symbolised by a specific geometric figure) consistent with the intention of a specific paradigm label. In some cases, more than one possible choice may be signalled for a question, whilst still remaining consistent with the intention of the paradigm. Consider, for example, the ‘Positivist/Normative’ solid line with circles. The line traces the pattern of guiding assumption choices typically associated with the positivist paradigm [Control/High or Partial + Perspective/Researcher + Locus/Externalised reality + Stance/ Objectivist + Theory/Apriori or Description + Learning/Explanation and Prediction or Learning what is]. In certain instances, a specific question may point to a distinguishing feature of a ‘paradigm’. For example, the paradigm of Critical Realism (see Sayer, 2000; Mingers, 2014) is distinguished particularly by adoption of the Hybridised loci choice under Locus coupled with a pluralist Stance. The Critical Social Science paradigm (triangles connected by a long dash-double dot line) is particularly distinguished by its reflection of the critical/emancipatory stance and the critique approach to Theory. The Indigenous or Feminist paradigm

9.1 What Are Guiding Assumptions? Pivotal Question

255

Assumption Choice A Assumption Choice B Assumption Choice C Assumption Choice D

Control

High

Partial

Minimal

Perspective

Researcher

Participants

Balanced

Locus

Externalised reality

Internalised ‘realities’

Hybridised loci

Stance

Objectivist

Subjectivist

Critical/ Emancipatory

Pluralist

Theory

Apriori

Emergent

Critique

Description

Learning

Explanation & Prediction

Deep understanding

What works/ What & How to change

What is/ Patterns of association

External

Positivist/Normative paradigm Interpretivist/Constructivist paradigm Critical Social Science paradigm Indigenous or Feminist paradigms Critical Realist/Postpositivist paradigm Participatory Inquiry paradigm Note: branches connecting to a line show alternative choices consistent with the same paradigm assumptions

Fig. 9.1 Patterns of guiding assumption choices labelled as ‘paradigms’

is distinguished by its reflection of the critical/emancipatory stance and the critique or descriptive approach to Theory coupled with a goal to achieve deep understanding and/or what is/patterns of association with respect to Learning. The Participatory Inquiry paradigm (octagons connected by a long dash-single dot line) is distinguished by its unique combination of subjective participant focus, emergent theorising and action-oriented learning (although a critical/emancipatory pathway is also possible). The seventh question in Table 9.1, Knowledge Building, is a higher-order question about your guiding assumptions and concerns the mode of knowledge production you intend to pursue in your research, based on the pattern of guiding assumptions you have chosen. This question is intended to invoke systems thinking about the overall role your guiding assumptions might play in generating knowledge. The choices for this question reflect more integrative considerations of your intentions as researcher coupled with the extent to which your research needs to connect to a wider range of stakeholder interests and with considerations of the research outcome(s) you envisage. Mode 1 knowledge building (Gibbons et al., 1994) refers to traditional academic and scientific knowledge, built up through disciplinary lines and usually with the intention of dissemination to and use by other researchers. Building up such knowledge is largely a cognitive exercise on the part of the researcher and immediate application of the knowledge may not be of specific interest. Mode 2 knowledge building (Gibbons, 2001; Gibbons et al., 1994; Nowotny, Scott, & Gibbons, 2001, 2003), on the other hand, refers to a more engaged form of participatory knowledge building that is transdisciplinary in focus,

256

9 Why Should I Think About Guiding Assumptions?

embedded within social and economic systems and involves a wide range of researchers and stakeholders. There is an explicit focus on application for Mode 2 knowledge, often coupled with a focus on participation and action. A rather different take on knowledge building has more recently emerged in the form of indigenous research methodologies (emerging from work especially in North American and Australian indigenous cultures). Knowledge building in this view can be seen as culturally privileged (Chilisa, 2012; Kovach, 2009; Wilson, 2001; Steinhauer, 2002). For the Indigenous research paradigm, knowledge exists and can be shared (under the right cultural circumstances); it is not owned by an individual. From an Indigenous perspective, knowledge is more about relationships (and being accountable for those relationships) than about objects and things, including relationships between researchers and Indigenous people (Wilson, 2001). Specifically, Wilson (2001) said. Our systems of knowledge are built on the relationships that we have, not just with people or objects, but the relationships we have with the cosmos, with ideas, concepts, and everything around us. For research it is important to think about our relationship with the ideas and concepts that we are explaining. Because this relationship is shared and mutual, ideas or knowledge cannot be owned or discovered (p. 177).

The Indigenous paradigm essentially embeds systems thinking as an implicit way of thinking about knowledge and knowledge sharing. The Feminist paradigm uses the same type of logic, except that knowledge is typically considered to be gender-privileged (where gender defines the cultural basis for privilege). In instances where the Feminist paradigm assumptions are used to guide research into sexuality, disability or any other way of identifying marginalised groups, that way of grouping people then defines the nature of cultural privilege. Note that in both the Indigenous and Feminist paradigms, the purpose of the research may be to contrast culturally-privileged knowledge held by the colonised or marginalised groups with the knowledge held by the non-Indigenous (i.e., colonising) or dominant majority groups, a focus that is very consistent with an explicitly critical intent. For the other named paradigm patterns of guiding assumptions above, the following choices may be considered typical in terms of Knowledge Building emphasis. The Positivist paradigm tends to be most effective and typical for Mode 1 knowledge building. The Interpretivist/Constructivist paradigm is often useful for Mode 1 knowledge building but can also work effectively for Mode 2 knowledge building. The Critical Social Science paradigm is typically oriented toward Mode 2 knowledge building. The Critical Realist paradigm is oriented toward Mode 1 knowledge building whereas the Participatory Inquiry paradigm is oriented toward Mode 2 knowledge building. The eighth question in Table 9.1, Assumption Diversity, is a more self-reflective higher-order one which can potentially have powerful implications for how convincing your research and what impact it has. The question focuses on the extent to which you plan to adhere to a single set of guiding assumptions throughout your research or to be open to employing a more diverse range of sets of guiding assumptions (a mindset known as ‘pluralism’; not to be confused with the

9.1 What Are Guiding Assumptions?

257

‘pluralist’ choice under Stance). This question is also intended to invoke systems thinking about your research intentions in general. The choices, moving from left to right in Table 9.1, reflect an increasing impact of systems thinking coupled with an increasing scope, and likely complexity, for your research. Generally speaking, as you consider the choices from left to right in the table, the research process will become more and more challenging for you as the researcher but also potentially more impactful for a wider range of research users. The Fixed choice is quite common in academic research and can often be detected in the types of articles that specific journals will tend to accept (for example, many journals in psychology, marketing, economics and management science have a strong preference toward publishing articles guided by the Positivist pattern of assumptions). The Fixed choice also tends to be more feasible for postgraduate research, particularly at the master’s level. The Varying choice signals your explicit intention to plan and use a different pattern of guiding assumptions for specific stages or phases of your research, often as we will see later, in the context of a sequential research configuration. The Flexible choice signals that you may start out with one specific pattern of guiding assumptions, but as events unfold, will implement different patterns of guiding assumptions, as appropriate to fit your emerging research needs (this choice cannot be fully planned from the outset and is a typical choice pattern for an adaptive researcher). The Creative choice is the most challenging choice to take on (e.g., least feasible, but not impossible, for postgraduate research) and signals an unwillingness to be locked into specific patterns of guiding assumptions; rather your intention would be to create and implement one or more new patterns of guiding assumptions (which may be hybrids of well-known paradigms, for example).

9.2

Exploding Some Misconceptions

It is at this point that we need to pause and explode some common misconceptions surrounding paradigms, guiding assumptions and their relationships to specific methodological choices downstream. Probably the most common misconception is the association between patterns of guiding assumptions and the type of data that researchers following those assumptions gather. Many researchers believe that data gathered under Positivist guiding assumptions must be quantitative in nature (numerical measurements) and that data gathered under Interpretivist/Constructivist guiding assumptions must be qualitative in nature (non-numerical, often textual data). This is a false set of equivalences. It is entirely possible to gather and make effective use of qualitative data under Positivist guiding assumptions. The data are simply ascribed different meanings and handled differently from the way that a researcher working under Interpretivist/Constructivist assumptions would handle them. Equally, a researcher working under Interpretivist/Constructivist assumptions can gather and make effective use of quantitative data. Again, such data would simply be ascribed different meanings and handled differently from the way a

258

9 Why Should I Think About Guiding Assumptions?

researcher working under Positivist guiding assumptions would deal with them. Thus, while there might be strong preferences for specific data types under specific sets of guiding assumptions, this does not mean that the non-preferred type of data is irrelevant or to be avoided in research conducted under that set of guiding assumptions. [Note that some research methods texts make the mistake of referring to approaches to gathering quantitative data as reflecting a ‘quantitative’ paradigm and approaches to gathering qualitative data as reflecting a ‘qualitative’ paradigm. Quantitative and qualitative refer simply to types of data, not to any specific pattern of guiding assumptions that underpin such data.] Another common misconception is that if you adopt one specific pattern of guiding assumptions, you should not/cannot then adopt another set of guiding assumptions within the same research project. This misconception has, in the past, led researchers to adopt a preferred pattern of guiding assumptions and to discount the value of other patterns of guiding assumptions (implicitly adopting the Fixed choice for the Assumption Diversity question). As a consequence, a kind of paradigm arrogance gets built up over time and this can (and has) become entrenched in some disciplines. The problem is that such researchers make their judgments of research conducted under other patterns of guiding assumptions using quality criteria from within their own preferred pattern of guiding assumptions – a cognitive process that is doomed to making erroneous judgments. Taking a systems thinking pluralist approach to social and behavioural research helps to overcome such paradigm arrogance by asking researchers to keep an open mind about the potential value of working under alternative patterns of guiding assumptions, even within the same research project. This is one reason why we considered Assumption Diversity in Table 9.1 to be an essential higher-order question you should explicitly address. The challenge is for you to do this in a way where logical inconsistencies don’t get introduced into your research processes. Managing to configure your research to be convincing while harnessing the potential benefits of using different patterns of guiding assumptions and at the same time as avoiding potential logical inconsistencies is a tough cognitive challenge (see Brocklesby, 1997 for a discussion of some of these challenges). There are a couple of strategies to consider: (1) use some type of sequential research configuration where different patterns of guiding assumptions are employed in different stages of the research (i.e., adopt the Varying choice for the Assumption Diversity question and/or (2) use a research team (less viable for most postgraduate research) where each member manages an aspect of the research under a specific pattern of guiding assumptions and the team then works together to integrate what they find (reflects one take on the Creative choice for the Assumption Diversity question, which can work even if a sequential set of research stages is not used). Another misconception, not unrelated to the previous one, is a clear manifestation of paradigm arrogance, namely that one pattern of guiding assumptions is better or to be preferred over all others. For a long time in social and behavioural research (particularly in disciplines like psychology, marketing and economics), Positivist guiding assumptions were strongly preferred as the way to do research. This is why the Positivist pattern of guiding assumptions became known as

9.2 Exploding Some Misconceptions

259

‘normative’, signalling that this is the traditional approach to doing social and behavioural science, i.e., ‘do research in the same way as they do in the natural sciences’. This misconception created self-perpetuating trends that could be observed in the ways graduate schools trained new researchers in specific disciplines, the types of projects postgraduate supervisors tended to supervise as well as the types of articles specific journals tended to publish. Some disciplines (e.g., education, sociology, management, information systems) have overcome, at least to some extent, this arrogance embracing a more pluralist and systems-thinking oriented outlook. Why did such a misconception emerge in the first place? Simply because looking at other patterns of guiding assumptions through the lens of one’s preferred pattern of guiding assumptions tends to lead one to judge the quality and character of the other patterns using quality criteria that were only ever designed to have meaning within one’s own preferred pattern. This is a complex way of saying that paradigm arrogance predisposes one to find other patterns of guiding assumptions wanting. Paradigm arrogance can manifest itself in other ways as well. For example, occasionally you will see a peer reviewer of a manuscript, submitted to a journal that holds strong views about the value of one pattern of guiding assumptions, use inappropriate quality criteria to judge the publication potential of that manuscript where the authors adopted a pattern of guiding assumptions different from the reviewer’s preferred view. Looking historically through specific research journals in a discipline (take, for example, the Journal of Applied Psychology, Journal of Educational Psychology, Journal of Marketing Research, Journal of Behavioral Decision Making or Management Decision), one can see strong trends in preferring to publish research conducted under a specific pattern of guiding assumptions (namely, Positivist). Awareness of the risk of paradigm arrogance is evident in some Australian universities when seeking examiners for a PhD thesis in that supervisors are advised to choose examiners that are open minded about paradigm guiding assumptions or that are known to be sympathetic to the guiding assumptions adopted in the PhD. Paradigm arrogance still exists today, but it is slowly becoming a less prevalent problem. More and more researchers are becoming aware that any pattern of guiding assumptions offers both advantages as well as disadvantages (and even blind spots) when adopted. No one pattern of guiding assumptions offers only advantages meaning that no one pattern of guiding assumptions can be promoted as being better in some essential way compared to others—each simply reflects a researcher’s particular pattern of answers to the pivotal questions. Importantly, by harnessing systems thinking when configuring your research, it may be possible to offset the disadvantages of one pattern of guiding assumptions employed in one part of your research with the advantages of a different pattern of guiding assumptions in another part of your research. Table 9.2 displays a side-by-side comparison of the benefits/advantages and costs/disadvantages of Positivist guiding assumptions and Interpretivist/Constructivist guiding assumptions. This table shows that some of disadvantages of one pattern of guiding assumptions can indeed be offset by some

260

9 Why Should I Think About Guiding Assumptions?

Table 9.2 Benefits/advantages and costs/disadvantages associated with the positivist and interpretivist/constructivist patterns of guiding assumptions with arrows signalling trade-offs POSITIVIST PATTERN OF GUIDING ASSUMPTIONS [typically reflecting quantitative preferences]

Benefits

Costs

INTERPRETIVIST/CONSTRUCTIVIST PATTERN OF GUIDING ASSUMPTIONS [typically reflecting qualitative preferences]

Cause-effect relationships can be teased out & substantiated (explicit theory testing) Precise control over measurements and contextual parameters is possible and necessary Standardisation of data gathering ensures data comparability across participants Data analysis is efficient & rule-driven using statistics Researcher remains separate from process and from participants (no researcher contamination of context) Explanation via theory, supported by empirical data, may enable prediction in future circumstances

In-depth contextualised understanding can be achieved Approaches are flexible, and methods can be adapted to contexts on the fly Data collection occurs in naturalistic & uncontrolled contexts Emphasises depth and thoroughness of data collection Data analysis is iterative, flexible and early analyses may provide feedback to guide future sampling & data collection Sample size is much less of an issue than who is in the sample (sample sufficiency is important) Unusual observations are incorporated via negative case analysis (inclusionary logic)

High degree of control imposes artificial constraints on context, reducing generalisation capacity Trade-offs between internal and external validity are inherent Results are often sample-dependent; construct validity & sample size affect how well statistical analysis can ‘perform’ Unusual observations are typically dismissed as outliers because they negatively impact on statistical analysis (exclusionary logic) The guiding theoretical or conceptual framework may have blind spots (e.g., omitted constructs and connections)

The research is almost always time-intensive (affects length of study) The research is almost always researcherintensive (affects depth of participation) Data analysis tends to be inefficient with relatively few rules for guidance; many approaches are possible It can be very difficult for the researcher to bracket his or her pre-conceptions during data collection & analysis unless steps are taken to reduce the risk The researcher can influence processes and participants and vice-versa The stories that emerge will tend not to be useful for making predictions

of the advantages of the other pattern of guiding assumptions—the trick is to properly configure your research to achieve this offset—something we have a lot more to say about in later chapters.

9.3

Make Your Guiding Assumptions Clear

It is critical in any high-quality research investigation that you provide clear arguments about the guiding assumptions you have adopted. This is especially important because it displays to anyone interested in learning from your research (call them ‘research users’) that you have thought carefully about what you are assuming, and that the decisions and choices you have made throughout your

9.3 Make Your Guiding Assumptions Clear

261

research process are consistent with those assumptions. This helps the research user to understand where you are coming from and helps them to decide which criteria for judging the quality of your research they should be relying upon. If you leave your assumptions hidden or undiscussed, you are depriving the research user of critical information they need to correctly understand the choices you have made. Ironically, in practice, you will find that a sizeable portion of the published literature you read is associated with authors who do not clearly unpack their guiding assumptions for the research user (perhaps because of length restrictions imposed by journals), and you are left to ferret this out for yourself using your knowledge of the discipline, the journal of publication, and any signals embedded in the strategic methodological choices the author has made. However, in our view, this is not good research practice. The better practice is for researchers to be explicit and transparent about their adopted pattern(s) of guiding assumptions in order to remove any ambiguity about where they are coming from. When beginning to consider assumptions to guide how to shape and scope a research problem of interest to you, it is good practice to let the problem and the research context(s) inform the choice of the most suitable and feasible pattern of guiding assumptions and associated practices and strategic methodological choices, and not the other way around. Too often, students (and academics as well!) will let their paradigm and methodological preferences, capabilities and comfortable ways of thinking determine which research problems are investigated and how. This amounts to forcing the research problem to fit a perspective, a particular way of thinking and specific method(s), thereby ruling out other potentially beneficial ways of thinking, other perspectives and/or other methods. For example, there can be a tendency for postgraduate students and academics who do not like quantitative data or methods (some students we have seen have an active ‘aversion’ to numbers) to automatically and uncritically gravitate toward a pattern of guiding assumptions associated with qualitative data gathering, even when such approaches do not really suit the research question they want to address. Conversely, we have seen postgraduate students and academics with a strong affinity for the assumptions associated with the scientific method (i.e., Positivist pattern of guiding assumptions as shown in Fig. 9.1) and its preference for quantitative data and statistical analysis, express a view that non-positivist patterns of guiding assumptions and their association with qualitative methods and data are ‘soft’, subjective and imprecise and have no place in their research endeavours even when their research question may be best addressed by them. The better strategy, in all cases, is to choose a pattern of guiding assumptions to fit your research problem. This can be a tough cognitive problem to get your head around, particularly if you have strong preferences or particular skills and capabilities, if your peer(s) and/or supervisor(s) have their own expressed preferences, and/or if the discipline you are working in has demonstrable preferences. However, for your research to truly be convincing, you must be clear about the guiding assumptions you are willing to adopt in order to have the best chance of addressing your research questions (Allard-Poesi & Maréchal, 2001, pp. 46–47). Thus, you have to be willing to question the status quo with respect to your guiding assumptions—another hallmark of systems thinking. Let your

262

9 Why Should I Think About Guiding Assumptions?

research problem, questions and contexts be your guide. If you have to develop new skills and capabilities or learn a new way of thinking, then it is better to do so than to force your problem to fit your current modes of thinking, skills and capabilities. Here is where systems thinking can help you in anticipating the need to develop and display new ways of thinking, new skills and new capabilities. After all, this is what postgraduate research is all about—developing and displaying new ways of thinking, new skills and new capabilities! For your particular research problem and questions, there may not be just one best-fitting perspective or method. If this happens, you will have to make a decision and actively defend it; first in your research proposal, then later on in your thesis, dissertation or portfolio. Alternatively, you may, through your reading of the literature, discover that your research problem has never been investigated under a particular pattern of guiding assumptions and that taking such an approach in your project would provide an original contribution to the literature or discipline. Here we see an important backward linkage that you may have to follow—you may have an initial idea about a research problem but the shaping of it and pattern(s) of assumptions appropriate to it may then be influenced by the literature that you read.

9.3.1

Arguments for Guiding Assumptions

There are several different and logically defensible ways that you can argue for the choice of guiding assumptions for your research. • Guiding assumptions contrast argument: You can construct the contrast argument by explicitly and defensibly contrasting the pattern of guiding assumptions you will be adopting against other potential patterns of guiding assumptions. You highlight the benefits of your preferred pattern of assumptions and show that those benefits are not achievable under other patterns of guiding assumptions. The contrast argument is often used when the choice is potentially contentious or is not obvious given the nature of your research problem, or when you have reviewed a body of research that has adopted a particular pattern of guiding assumptions and have concluded that a different approach is necessary. • Assertion of congruence of guiding assumptions with the research problem: The congruence argument involves a straightforward assertion of the pattern of guiding assumptions you will be adopting. Here, you choose not devote effort to contrasting your choice with other potential patterns of guiding assumptions. You could use the congruence argument when your choice of guiding assumptions is relatively uncontentious, is obvious/necessary given the nature of your research problem and questions, or if the majority of the research reviewed has adopted the same pattern of guiding assumptions and you wish your research to be consistent with that body of knowledge. Some postgraduate supervisors may steer you away from this type of argument in favour of the contrast argument because they believe part of demonstrating your research

9.3 Make Your Guiding Assumptions Clear

263

capabilities is being able to argue for your position against other potential positions. This is something you should discuss with your supervisor(s). • Previous research deficiency argument: You can mount a deficiency argument if you argue that previous research guided by a particular pattern of assumptions was flawed or deficient in some way, and, as a consequence, your research is being positioned to overcome or circumvent those flaws or deficiencies. This argument can be used to justify either a move to a new or different pattern of guiding assumptions (e.g. from Positivist to Interpretivist/Constructivist, essentially combining the deficiency argument strategy with a contrast strategy), or an improved effort at research within the same pattern of guiding assumptions (e.g. where previous interpretive efforts were flawed relative to the interpretive effort you will undertake). You could use this style of argument when clear gaps or issues emerge in the literature, and you want to set the stage for your study to rectify this. • Need for a pluralist approach: The pluralist argument is the most complex you can make and the most demanding in terms of the systems thinking required for getting it right. This argument commences, not from a basis of deficit in previous approaches, but from an assertion that more stands to be gained if a pluralist approach is adopted. By a pluralist approach, we mean that, for distinct aspects or phases of the research, different patterns of guiding assumptions are adopted with the ultimate goal of integrating the partial stories gleaned under each pattern. By doing this, some of the paradigm-specific weaknesses associated with one pattern of guiding assumptions may be compensated for by the strengths of another pattern (recall the lessons in Table 9.2). This sort of argument is similar to one often used by ‘mixed methods’ researchers. However, the integrated story is often difficult to achieve unless there is a way to bring learning under different guiding assumptions together into a single coherent story. An integrated story may also be difficult to achieve if you are not sufficiently broad-minded and flexible in your worldview to permit an even-handed approach to be taken, even in the face of perhaps contradictory guiding assumptions (so that paradigm arrogance does not come into the picture). As it turns out, the pluralist approach is one where the meta-criteria for research quality, to be discussed later, can be of great benefit. The following five examples, quoted from PhD theses Ray has supervised, illustrate these different types of arguments for a specific pattern of guiding assumptions. Example A Contrast argument arguing for adopting a Positivist pattern of guiding assumptions. “How then might one choose between paradigms when all have strengths and weaknesses, when all are partial? Clearly, a researcher’s own biases and experiences will play a role. This will be reflected in the research question addressed and the nature of this question will largely determine the appropriate research approach (Yin, 1994). The research question addressed in this

264

9 Why Should I Think About Guiding Assumptions?

thesis concerns universalistic, contingency and configurational theoretical approaches, each of which largely draws upon positivist assumptions (see Table 5.1). Furthermore, the research question addressed in this thesis is concerned with relationships between variables, a key focus of positivism, rather than the interpretivist search for patterns of meaning (Gephart, 1999). Finally, this thesis is concerned with testing the relationships suggested by the theoretical approaches, again a choice that indicates positivism because a positivist approach is generally suited to theory testing, while interpretivism is suited to theory generation (e.g. Parke, 1993). Therefore, this thesis can be considered to be embedded within the positivist paradigm.” (Harrison, 2003, pp. 96–97).

Example B Contrast argument for adopting an Interpretive/Constructivist pattern of guiding assumptions. “I argue that to understand group dynamics more fully, the multiple interactions that occur within and between a group and it are contextualised environment must be the focus of study. Thus, group-specific interactive observable behaviours will be revealed as emergent and self-organising dynamics, influenced by time. The presence of interplay between and within a group’s contextualised environment presents a group’s dynamics as a product of social construction that occurs through primary and secondary socialisation (Berger & Luckmann, 1966). Primary socialisation is the basic social structure that each individual is born into and learns to interpret and adapt to their world (Berger & Luckmann, 1966, p. 151). Primary socialisation occurs at two levels: the individual self and the cultural self. Secondary socialisation focuses on the acquisition of role-specific knowledge deemed necessary for an individual to complete tasks (Berger & Luckmann, 1966, p. 158). These interactions are not static but change dynamically over time, creating phenomena that cannot be easily understood through an objectivist epistemology and a positivist theoretical perspective, where a reductionist logic is applied to contextually simplify and isolate specific hypothesised constructs and their relationships (Crotty, 2010). Considering the complex and dynamic interactions that occur over time to form a group’s dynamics, the purpose of my research is to offer an in-depth understanding of those dynamic and complex interactions that occur within and between a group and its contextualised environment, over time. To gain this understanding required implementation of a constructionist epistemology that examined the ongoing social construction of group dynamics from multiple perspectives (Crotty, 2010).” (Wolodko, 2017, p. 3).

9.3 Make Your Guiding Assumptions Clear

265

Example C Congruence argument for adopting a Critical Realist pattern of guiding assumptions. “Central to the approach used in applying this conceptual model to farms was the philosophical basis I brought to the research, making this a crucial starting point for describing research design. This research sits broadly within the post-positivist paradigm (Clark, 1998). I approached it from a critical realist ontology. While I acknowledge an objective reality (Crotty, 1998), I consider how this reality can be seen or known is imperfect because of subjective experience (Boeree, 1999; Guba & Lincoln, 1994; Healy & Perry, 2000; Lincoln & Guba, 2000; Magee, 1985; Patomäki & Wight, 2000; Yeung, 1997). An application of the conceptual model to farms required the collection and analysis of farm-specific data. Due to the influence of subjectivity on perceptions of reality, I conducted this research from a modified dualist epistemological approach (Guba & Lincoln, 1994; Lincoln & Guba, 2000). Hence, I viewed the most reliable source of knowledge about the historical sequence of decisions on the farm to be the producer. This means my focus in gathering data was on eliciting information about producer experiences. Given this, the qualitative research methods I adopted centred on using the producer as the chief data source and undertaking procedures to monitor and safeguard against undue subjective influences of the researcher on the data. Thus, continuous care was taken to ensure both transparency of research process/ analysis/interpretation and authenticity of participant voice and meanings.” (Cowan, 2014, pp. 83–84).

Example D Deficiency argument for adopting an Interpretive/Constructivist set of guiding assumptions, involving method triangulation. “The research methodology adopted for this study works from a constructivist interpretive perspective using a case study approach. The approach draws on information gathered via a number of qualitative methods, including unstructured depth interviews, non-participant observation, and the analysis of relevant organisational documentation pertaining to the role of Vice-Chancellor. As will be explained, the interpretive methodologies adopted for this thesis were chosen to allow the researcher to get far closer to the people under investigation than has been the case in previous studies of Vice-Chancellors, particularly in the Australian context. The choice of methodology in this research has been determined by the conviction that getting close to the action of role enactment is an imperative. Further, a desire to enhance accuracy of interpretation required a multiple method approach to enhance the likelihood of richer data gathering and interpretation. A triangulated approach was regarded, therefore, as an

266

9 Why Should I Think About Guiding Assumptions?

appropriate strategy to maximise the likelihood of achieving this richness. As Stake argues, “however accuracy is construed, researchers don’t want to be inaccurate, caught without confirmation” (2005, p. 443). Previous research into university leadership and governance has tended to take a relatively “top-down” approach. This has been exemplified through the work undertaken by Marginson and Considine (2000) in particular. These authors also endeavoured to get close to the subject and obtained much valuable data from observation and interview. A criticism of their approach, however, was that the research was relatively top down in focus and tended to bypass input from a wider range of stakeholders in university governance and leadership, for example, student groups, general staff, and industrial relations groups (Gillies, 2000, p. 33). The lack of a 360° perspective does not satisfy an exploration of role in the sense defined by Katz and Kahn (1978) discussed previously. This research is focused on gaining an understanding of role enactment and its meaning from the perspective of the participants (Mullen, 1996). The research seeks to gain access to both public and “back of stage” (Goffman, 1956) performances in order to deepen our understanding of the Vice-Chancellor’s role in their university’s contexts. Very few researchers have been able to gain such deep level access to the professional lives of Vice-Chancellors. Thus, the methodology required for data gathering and analysis needed considerable openness and flexibility.” (McClenaghan, 2006, pp.112–113).

Example E Pluralist argument arguing for adopting different patterns of guiding assumptions at different stages of the research. “Moreover educational research on self- and collective efficacy beliefs are largely dominated by the positivist paradigm and very little has been done following the interpretivist paradigm (Labone, 2004; Pajares, 1992; Tschannen-Moran et al., 1998). The existing knowledge gap at the Bhutanese as well as at the international level provided the opportunity to employ a multi-paradigm perspective to gain more general and in-depth insights based on the perceptions of principals and teachers measured by their efficacy beliefs, importance, support system, and actions and impacts of GNH Education. … Consequently this study employed both the positivist and interpretive guiding assumptions, at different stages, to more deeply understand the efficacy beliefs and experiences of principals and teachers in implementing GNH Education. It is also the first study in the Bhutanese context to adapt tools for measuring self-efficacy of principals and teachers and collective efficacy of schools employing a combination of both the constructivist and positivistic paradigms. Future researchers in the Bhutanese education system can employ these efficacy tools to study a wide variety of key learning areas and programmes.” (Sherab, 2013, pp. 12, 15).

9.3 Make Your Guiding Assumptions Clear

9.4

267

How Do Guiding Assumptions Relate to Quantitative and Qualitative Data Preferences?

When speaking of quantitative methods, most people equate them with the Positivist paradigm and when speaking of qualitative methods, most people equate them with Interpretivist/Constructivist paradigms. However, as we argued before, this is a false equivalence (see also arguments in Bryman & Bell, 2015: 37–39) and it is important to show why here. Researchers following the pattern of guiding assumptions for almost any named paradigm may employ specific strategies to gather quantitative or qualitative data as appropriate to the research question at hand. Almost any data gathering strategy you can name can yield either quantitative or qualitative data. In this light, quantitative or qualitative data should be seen simply as types of information that you gather in order to build up particular kinds of inferences and draw particular kinds of conclusions. The key thing to remember is that it is not the type of data that matters so much as why you gather each type and what you do with that information to help flesh out a story about what has been learned. Different patterns of guiding assumptions will lead you to expect and do different things with different types of data. Table 9.3 tries to clarify this point by juxtaposing data type (quantitative or qualitative) against two common patterns of guiding assumptions, Positivist and Interpretivist/Constructivist. The table makes it clear that quantitative and qualitative data each have their own strengths and weaknesses, and each implies different considerations when planning research. Furthermore, guiding paradigm assumptions will influence how data of a particular type should be dealt with and made sense of. Thus, qualitative data gathered within a Positivist perspective are technically not the same kind of data and are not interpreted or treated in the same way as qualitative data gathered within an Interpretivist/Constructivist perspective. The same is true for quantitative data—guiding assumptions will direct what should be made of and done with such data.

9.5

Engaging Pluralist Logic—Moving Beyond ‘Mixed Methods’ Thinking

Should you gather both quantitative and qualitative data? While this is a question that postgraduate students often ask, we argue that this is not the right question. Our thinking with respect to research approaches in social and behavioural research has now progressed to the point where we should view them more systemically and holistically. In the past, attempts to achieve a more holistic and integrated perspective have tended toward concepts like triangulation (“the combination of methodologies in the study of the same phenomenon.”, Denzin, 1978, cited in Jick, 1979, p. 602; see also Denzin, 2012; Jick, 1979; Olson, 2004), bricolage (“the process of employing these [multidisciplinary] methodological strategies as they are

Permissible inferences?

Meaning of the data?

The number represents the positioning of an observation or response along a particular scale with known properties and potentially knowable precision (measurements will have a known amount of error associated with them; indexed by a reliability statistic; proportion of error associated with any measurement instrument equals 1.0 minus reliability) Interpretation of numbers critically depends upon the validity of the construct that the number is operationally defined

Numbers or perhaps circled words, ticked boxes or graphical marks Measurement on a specifically-designed scale or an instrument reading; some measures may be simple counts of categories observed (nominal scale measurement)

Initial form?

How obtained (typically)?

Quantitative data Positivist pattern

Characteristic

Not typically interpreted as a measurement; number may be taken to indicate prevalence or emphasis in a sample, strength

Quantifies the physical number of codes, themes, events, occurrences or observations identifiable in a data sample

Numbers tend to be treated as simple counts, rather than measurements Counting of themes, codes, occurrences, events and the like

Interpretivist/constructivist pattern

Interpretation of numbers depends upon the validity and reliability of the category classification system; inter-rater

Theme or idea that seems to best classify the response being considered. Preference for only one classification per segment of text (mutually exclusive category systems). Typically, this only yields measures at nominal or sometimes the ordinal measurement level

Written or recorded responses to open-ended questions or text documents

Written (sometimes oral) language

Qualitative Data Positivist Pattern

Interpretation of subjective meaning from participant’s perspective; the data are read as reflecting the voice of the (continued)

Recorded via field notes, self-report responses to questions, hard copy or electronic text files or stored on digital media during interviews or observations or in multi-media form from other sources Characterises a theme or idea (codes) that seems to capture a meaning or perspective the participant seemed to be reflecting. Multiple codings are possible for any segment of text; i.e., there may be multiple meanings at multiple levels

Written or oral language, texts, sounds, images

Interpretivist/Constructivist Pattern

Table 9.3 Quantitative and qualitative data and their characteristics within the positivist and interpretivist/constructivist patterns of guiding assumptions

268 9 Why Should I Think About Guiding Assumptions?

If assumptions of construct validity and reliability can be defended, quantitative measurements can afford: • Precision • Consistency • Participant comparability • Ready access to statistical analysis for assembling an efficient story about relationships and group differences

Measurement quality is heavily dependent upon the design of the measurement system itself and may create a false sense of ‘objectivity’ simply because

Dis-advantages?

to reflect as a measurement system as well as on the design of the measurement scale itself: nominal scale; ordinal scale; interval scale; or ratio scale (see Chap. 18 for more detail on measurement scales)

Quantitative data Positivist pattern

Advantages?

Characteristic

Table 9.3 (continued)

of feelings. Technically, these counts may be considered to fall along a ratio scale, where zero means nothing was coded or observed, and it makes conceptual sense to say that a theme coded 200 times was twice as prevalent as a theme coded 100 times Counts can provide a useful way to aggregate qualitative data across participants or occasions in order to tell a story at a higher level of analysis; such data are usually just treated descriptively using statistics, rather than for inferential statistical testing. Count data may help, as a form of triangulation, to reinforce a story about strength, prevalence or patterns of particular meanings or events Count data are usually considered as subordinate to qualitative data and, in order to actually produce the counts, one must assume that the things

Interpretivist/constructivist pattern

Qualitative data are treated as subordinate to quantitative data and are seldom dealt with in their original qualitative form in analyses. Such data are rarely

Can help overcome the contextual insensitivity and overly restrictive structural requirements of measurement scales by inviting participants to provide their own thoughts on particular issues. Such data are easily subjected to content analysis, which provides coding categories that can be quantified and incorporated into statistical analyses along with quantitative data

reliability is important to demonstrate

Qualitative Data Positivist Pattern

The fact that data may be read in multiple ways is also seen as a potential disadvantage in that different researchers may undertake different readings. (continued)

Data can be analysed in as close to their rich and raw form as possible, enabling analyses to remain close to the data that are being interpreted. Data may also be read and interpreted in multiple ways to help inform particular research stories. Contextual nuances can be identified which allows data meaning to be closely linked to data context

participant. Inferences are based on the researcher getting progressively closer to understanding the meaning of the data from the participant’s perspective. Multiple interpretations may be possible from the same data

Interpretivist/Constructivist Pattern

9.5 Engaging Pluralist Logic—Moving Beyond ‘Mixed Methods’ Thinking 269

Characteristic

numbers look ‘objective’ and ‘scientific’. The more the system is open to interpretation by different participants (as is the case with most self-report measurement systems like Likert-type scales), the more difficult it is to defend the comparability of responses. Also the high level of structure in measurements affords no opportunity for participants to express personal or alternative views—richness is removed, by design, from the data. Finally, measurement scales, by design, are context-free in interpretation, which removes the opportunity for contextual nuances to be tracked, unless contextual conditions are specifically built into the research configuration itself

Quantitative data Positivist pattern

Table 9.3 (continued)

being counted can actually be treated as having the same meaning—counts are thus insensitive to nuances in meaning

Interpretivist/constructivist pattern interrogated for their contextual and subjective meaning; interpretation is therefore mostly at the surface-level (i.e., what was objectively said rather than what might have been meant); other contextual nuances, such as emotive expression, motives and non-verbal content are usually ignored. Comparability across participants is generally assumed, as long as the data look the same

Qualitative Data Positivist Pattern

(continued)

Data recording must be in the form of low-inference descriptors (i.e., without pre-filtering or preliminary judgments or analysis) to minimise this possibility. It requires great care to ensure that all of what is desired to be represented in the data is actually captured in the data transcript (or other form) used for analysis

Interpretivist/Constructivist Pattern

270 9 Why Should I Think About Guiding Assumptions?

If the original data are captured as circled words (as for some Likert-type scales) or ticked boxes (e.g., for demographic variables), responses are transformed into appropriate corresponding numbers before analysis Experiments, quasi-experiments, systematic observations, questionnaires, structured interviews, case study, secondary data A multiple-item Likert-type scale for measuring a specific theoretical construct, such as organisational commitment, leadership style or job satisfaction, using a fixed-point scale with verbally-anchored scale points (e.g., 1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; 5 = strongly agree)

Final form for analysis?

Example

Which strategies can generate such data?

Quantitative data Positivist pattern

Characteristic

Table 9.3 (continued)

A matrix of counts of the number of utterances coded in particular thematic areas for interviewed participants occupying different organisational roles (e.g., non-managers versus line versus senior managers)

Participant observation, semi-structured interviews, case study, text analysis

Numbers as counts

Interpretivist/constructivist pattern

Open-ended questions on a questionnaire, inviting participant’s own thoughts about events or a construct like leadership; responses are coded, quantified and used in testing for relationships between categories of responses to the open-ended questions with quantitatively-measured demographic and/or attitudinal or performance measures

Questionnaires, structured interviews, content analysis, case study

Categories are quantified by nominal or ordinal scaling methods prior to analysis

Qualitative Data Positivist Pattern

Participant observation, ethnography, unstructured and semi-structured interview, text analysis, case study, questionnaires Extensive field notes gathered from participant observation, transcribed verbatim and stored as electronic text files for analysis

Raw data are transcribed, usually into a written text format, for analysis

Interpretivist/Constructivist Pattern

9.5 Engaging Pluralist Logic—Moving Beyond ‘Mixed Methods’ Thinking 271

272

9 Why Should I Think About Guiding Assumptions?

needed in the unfolding context of the research situation”, Kincheloe, 2005, p. 324; see also Kincheloe & Berry, 2004; Patton, 2011, Chap. 9), multimethodology (using “more than one methodology, or part thereof, possibly from different paradigms, within a single intervention”, Mingers & Brocklesby, 1997, p. 491; see also Bowers, 2011; Brocklesby, 1997; Flood & Romm, 1997; Hesse-Biber, 2012; Mingers & Gill, 1997) and, most commonly, mixed methods (“research in which the investigator collects, analyses, mixes, and draws inferences from both quantitative and qualitative data in a single study or a program of inquiry”, Journal of Mixed Methods, 2006, cited in Cameron, 2011, p. 96; see also Creswell & Plano Clark, 2018; Morgan, 2007: Morse & Niehaus, 2009; Plowright, 2011; Tashakkori & Teddlie, 2010). For a while, such concepts implied the idea of plurality and diversity in approaches and achieved a relatively wide-spread level of advocacy and currency in research methodology. They were viewed as the things to do if you wanted to carry out a more convincing piece of research. However, the central message of systemic integration and holism implied by each of these concepts was largely ‘hijacked’ by the on-going divide between quantitative and qualitative approaches. In fact, the logic of ‘mixed methods’ research appeared to be reduced to encompass only those research approaches that gathered both quantitative and qualitative types of data in some configuration (see Tashakkori & Teddlie, 2011). To us, that reduction has been counterproductive to achieving a truly holistic perspective, in that the binary divide between data types has usurped more systemic thinking about one’s research. [Part of our reasoning behind including Table 9.3 in this chapter was to show the fallacy of aligning one type of data with a specific pattern of guiding assumptions.] As a way forward, we advocate an explicitly pluralist logic where you argue for whatever configuration of research choices will feasibly allow you to achieve the most convincing outcomes given the research problem you are confronting, the research context(s) you are working within, the stakeholders you must deal with and the constraints and opportunities you encounter along the way. This logic shares much in common with ‘bricolage’ in the sense that Kincheloe (2005), intended it, but with a more open systemic sensitivity. It also shares logic in common with perspectives argued by Moran-Ellis et al. (2006), Midgley (2000) and Plowright (2011). For example, Plowright (2011), with his Frameworks for an Integrated Methodology (FraIM), put forward a more integrated perspective on social research that moved beyond the quantitative/qualitative mixed methods divide and echoes of some of his ideas can certainly be detected in our approach. However, Plowright did not situate his FraIM approach within a more systemic set of contextual considerations, was ambivalent about the usefulness of the philosophical underpinnings of social research (which give rise to patterns of guiding assumptions) and did not clearly link his approach to criteria for judging research quality, all of which we address in this book. Pluralist arguments could encompass the use of single and/or multiple patterns of guiding assumptions and perspectives, research frames (to be discussed in Chap. 11 ) and contexts, researchers, data sources, data gathering strategies, data types and/or analytical frameworks. These arguments do not typically imply binary choices; they

9.5 Engaging Pluralist Logic—Moving Beyond ‘Mixed Methods’ Thinking

273

address complex, interconnected and contextually-situated considerations and, in the end, the choices you make while pursuing your research journey should be those that will simultaneously best suit your research problem and make your journey realistically feasible to complete. Along the way, it will become clear that some of your early choices will help you to make, or at least narrow down, other choices downstream. For example, choosing a specific pattern of guiding assumptions to follow will make some data gathering strategies and quality criteria relevant while rendering others useless or meaningless. Choosing to include certain data sources in your research may narrow the range of possible data gathering strategies you can use. Knowing that you must meet certain stakeholder expectations may influence the type of research frame and data analytic framework you adopt. So, back to the initial question in this section “should you gather both quantitative and qualitative data?—the answer is, it depends! Should you use more than one pattern of guiding assumptions? It depends! Should you use more than one type of data source? It depends! Should you use more than one data gathering strategy? It depends! And on it goes. You must make all these decisions in context; there is no playbook, blueprint, schema or recipe that can claim to tell you what and how to orchestrate your research. This is the tough lesson about social and behavioural research—it is all about choices in pursuit of goal(s) (some of which may be yours; some may come from others) and it all depends. It is also the beauty of social and behavioural research—there is ample room for creative thinking, for inclusionary processes, for discovering new things, for stimulating change. What can be stated with some certainty is the more plurality you incorporate in your research choices, the broader your skill set must be, the more resources (including time and cognitive effort) you will typically require, the more diversity in supervisory input you will need and the more challenging it will be for you to bring everything together to convey a convincing story. However, if your choices are well-reasoned and feasible to implement within the constraints imposed by your postgraduate candidature, a more convincing story will tend to emerge. Using one or more forms of plurality, however you might label it, is becoming an increasingly desirable (but not mandatory) feature of social and behavioural research, including research done at the postgraduate level. When you employ pluralist logic, you make research configuration and implementation choices to harness methodological diversity in service of your research goals, in circumstances that demand it (of course, not all research circumstances will permit or demand a plurality of approaches). Pluralist logic can yield, but will never guarantee, greater capacity to convince a reader/examiner of the quality and value of your findings (as well as of your diversified set of skills in executing an independent research project!). However, realistically, it is a more difficult type of research to conduct and write up and involves a higher level of complexity in project planning. Recall that we flagged earlier in Table 9.2 that it may be possible to offset the costs of doing some aspect of your research under one pattern of guiding assumptions with the benefits of using another pattern of guiding assumptions, i.e., multi-paradigm triangulation (see also Bryman & Bell, 2015, p. 36). Such a pluralist strategy,

274

9 Why Should I Think About Guiding Assumptions?

particularly for a postgraduate student, would not necessarily be easy or feasible and might even be cognitively difficult to conduct (see Brocklesby, 1997; Bryman, 2007), but it is slowly becoming more commonplace in some form or another. Finally, there is no guarantee that research employing pluralist logic will converge on a single story, meaning that diverse, perhaps even contradictory, stories may have to be reconciled or, indeed, may not be reconcilable. Therefore, when thinking systemically and complexly about your research, you need to think about research and strategic methodological choices in a number of ways: • • • • • • • •

possible patterns of guiding assumptions; possible choices of research quality criteria to apply; possible choices of literature to draw upon; possible choices with respect to the positioning of your research from both your perspective and your participants’ perspectives; possible choices of research frames and configurations; possible choices of data gathering strategies; possible choices of types of data (quantitative or qualitative); and possible choices of data sources (humans, documents, artefacts, performances, multimedia recordings and so on).

The range of choices available in the above list is what reflects diversity. For our purposes, pluralist logic involves keeping all of these potential choices in play as you work your way toward a concrete research approach. Pluralism does not imply that multiple choices at any level are mandatory, only that they are considered. Mixed methods logic requires both types of data, quantitative and qualitative; pluralist logic goes beyond mixed methods thinking by considering all possible configurations: all quantitative data; all qualitative data; some data of each type. It is entirely feasible, that after due consideration, you may decide that your research problem can be convincingly addressed using a single pattern of guiding assumptions, a single research frame, a single data gathering strategy, a single type of data and one data source (what, in the past, would be construed as a ‘mono-method’ approach). Equally, you could decide to employ a single pattern of guiding assumptions, a single research frame, multiple data gathering strategies, multiple data types and multiple data sources, or could decide to employ multiple patterns of guiding assumptions, multiple research frames, multiple data gathering strategies, a single type of data (e.g., qualitative) and multiple data sources. Nothing is mandated in patterns of choices here—all potential patterns may be on the table for consideration—this is the major implication of pluralism and diversity with respect to systems thinking about social and behavioural research. In Chap. 12, we will explore a representational system that can help in configuring your research when engaging in pluralist thinking.

9.5 Engaging Pluralist Logic—Moving Beyond ‘Mixed Methods’ Thinking

Survey of employees

Multiple Data Gathering Strategies

Interview managers

Interview managers

Multiple Data Types

Video record behaviour

Interview subordinates

Read policy documents

Observe board meetings

Quantitative survey questions

Multiple Data Sources

275

Qualitative survey questions

Interpretive interviews of workers

Positivist questionnaire for consumers

Multiple Paradigms/Patterns of Guiding Assumptions Action research intervention to improve work processes

Fig. 9.2 Illustrations of potential pluralist logic in postgraduate research

Figure 9.2 illustrates some possibilities for pluralist logic that may be feasible under the constrained conditions of postgraduate level research. Note that some forms of pluralist logic may lead to hybridised combinations (for instance, the ‘multiple data gathering strategies’ example in Fig. 9.2 automatically implies the ‘multiple data sources’ logic as well). In the context of postgraduate thesis, dissertation or portfolio research, multiple researchers would generally not be an option, unless the Participatory Inquiry pattern of guiding assumptions was being implemented, and even then, there would only be one author, yourself, identified for the examinable research outcome. If you implement a pluralist logic for your research, you can expect your research timetable to be extended, in some cases substantially so. You can also expect that data analysis and writing up of results will be more complex and require a more diverse range of skills to manage. The outcomes from such analyses may yield stories that converge but may also yield contradictions and ambiguities that you will need to explain or resolve.

276

9.6

9 Why Should I Think About Guiding Assumptions?

Guiding Assumptions Are Associated with Paradigm-Specific Quality Criteria

Different patterns of guiding assumptions are accompanied by different sets of paradigm-specific quality criteria. Such criteria are intended to be used to judge the character and quality of research guided by the that pattern of (paradigm) assumptions, i.e., what constitutes ‘good’ research within that paradigm. Here, we will focus on two commonly used yet contrasting sets of quality criteria: one set associated with the Positivist pattern of guiding assumptions and the other set associated with Interpretivist/Constructivist and other non-positivist patterns of guiding assumptions. It is important to understand such criteria as you can use them to judge the quality of other’s work within the paradigm and others can and will judge your work in the same way.

9.6.1

Quality Criteria Associated with the Positivist Pattern of Guiding Assumptions

The pattern of guiding assumptions that defines the positivist paradigm has a long history in the context of the social and behavioural sciences and in the context of the natural sciences before this. In fact, Positivist assumptions for social and behavioural research constitute a straightforward importation from the natural sciences so that, under these guiding assumptions, we study human behaviour and systems in the same way that we study other natural phenomena. For such research, quantification of observations in the form of measurements is considered to be the most appropriate type of data to gather and analyse (qualitative data can also be used, but only after they are transformed into forms amenable to quantitative analysis, as signalled in Table 9.3). Accordingly, for research following the Positivist pattern of guiding assumptions to be considered of high quality, it must meet certain standards. In the social and behavioural sciences, Campbell and Stanley (1966) and later Cook and Campbell (1979) and Cook, Campbell, and Peraccchio (1990) crystallised these standards into a set of four essential types of validity, namely, construct validity, internal validity, external validity and statistical conclusion validity. When discussing these four types of validity below, we will refer to the first six pivotal questions shown in the guiding assumption choice patterns of Fig. 9.1. Construct validity Construct validity is linked to the quality of the processes you would use to translate theoretical ideas and concepts (called ‘constructs’) into concrete quantitative measurements (a process called ‘operational definition’). For example, job satisfaction is a construct that you may try to measure and quantify to test theories about its relationship to work performance (another construct to be measured).

9.6 Guiding Assumptions Are Associated with Paradigm-Specific Quality Criteria

277

Construct validity addresses the fundamental question: are you measuring the construct you claim to be measuring in your research? Implicit in the concept of construct validity is the notion of measurement reliability or consistency. That is, you cannot demonstrate or argue for the construct validity of any measurement system without also demonstrating or arguing for the reliability of those measurements; conversely, however, demonstrating or arguing measurement reliability says nothing about construct validity. Thus, construct validity concerns the appropriate inferences that can be drawn from measurement processes giving rise to the quantitative data used to test hypotheses. Reliability supports such inferences because it concerns how consistently you are measuring the construct of interest. Construct validity is an important quality criterion under the Positivist pattern of guiding assumptions because it focuses on how you (Perspective/Researcher) attempt to objectively quantify (Stance/Objectivist) and externally represent (Locus/Externalised reality) some aspect of human behaviour you are theorising about (Theory/Apriori). More about construct validity and reliability will be said in Chap. 18. Internal validity Internal validity refers to your capacity to appropriately claim that you have found specific and unambiguous relationships between constructs of interest. Causal relationships are especially, but not exclusively, of interest under the Positivist pattern of guiding assumptions. In order for a causal relationship to be claimed to exist between two or more constructs, all three of the following conditions must be met: (1) the causal construct must precede the effect construct in time (temporal priority of cause); (2) when the causal construct changes, the effect construct must be observed to change (covariation) and (3) alternative plausible cause constructs for the effect construct must be ruled out (alternative plausible causes are controlled). A weaker form of relationship, called an association or a correlation, may also be of interest, in which case, only condition 2 (covariation) must be met. Internal validity asks whether the configuration and execution of your research warrants the relational conclusions you are drawing, whether causal in nature or not. In short, internal validity concerns the appropriate meaning of the relationships you have found in quantitative data. Internal validity is arguably the most important quality criterion under Positivist assumptions because it focuses on how well you (Perspective/Researcher) objectively test (Stance/Objectivist) for the existence (Locus/Externalised reality) of theorised relationships between constructs (Theory/ Apriori) by controlling, where possible, for alternative plausible explanations for the relationships observed (Control/High or Partial). Internal validity is generally enhanced through your control over events and over the experiences of participants. Such control can be achieved through (1) the configuration of research conditions and experiences at the time of data gathering, (2) the use of procedures, prior to data gathering, for reducing or preventing the influence of unwanted causal factors and/ or (3) the use of statistical analyses to control for unwanted causal influences during the process of data analysis. More about this will be said in Chap. 14.

278

9 Why Should I Think About Guiding Assumptions?

External validity External validity refers to your capacity to appropriately generalise the conclusions about relationships substantiated in your research to other contexts, people, places and times. Especially important is the capacity for you to generalise what is learned about patterns and relationships from the sample of participants employed in your research to the population from which you drew the sample. If any biases occur in the sampling process and/or if issues arise associated with demand characteristics, external validity is reduced. Demand characteristics, in particular, may emerge with human participants in research. Demand characteristics refer to aspects of the research context (e.g., ‘Hawthorne effect’, where something different happening in the research context changes behaviours), research processes (e.g., ‘Guinea pig effect’, where something in the way you are gathering the data signals what you are testing for) or your own behaviours as researcher (e.g., ‘experimenter bias’, where you inadvertently reveal, through how you talk and/or behave, what you expect to find) that change in such a way as to render what you have measured to be idiosyncratic and only locally relevant. Sampling biases can be controlled for through use of an appropriate sampling strategy and demand characteristics can be explicitly controlled for by you, the researcher, through effective purpose- or treatment-masking procedures. However, external validity is about more than generalising from a sample to a population. It also concerns how well the research results can be generalised to other situations relevant to the members of the sample. Here we are talking about whether the experiences that members of the sample have while participating in your research bear any relationship or relevance to their lives in outside the research context. If those experiences are relevant only to the research context itself, then it is likely that external validity is lower. As we shall see down the track, this problem is especially relevant for laboratory-based research where participants may experience artificially-controlled environments and tasks that bear little or no relationship to anything they might experience outside of the laboratory. Brunswik (1952, 1956) drew attention to this problem when he developed the concept (and the process) of representative design as a way of trying to ensure that what is learned in highly-controlled laboratory research has some relevance to the larger life space of those who participate in the research as well as to those who do not participate in the research. In short, external validity concerns the meaningfulness of claims about the generality of the relationships found in quantitative data. External validity is an important quality criterion under positivist assumptions because it focuses on how well you (Perspective/Researcher) are able to generalise (Stance/Objectivist) the existence (Locus/Externalised reality) of the relationships established between constructs within your sample to the population from which your sample was drawn as well as to other contexts and situations in order to create opportunities for explaining and predicting the phenomena of interest at a more general ‘law-like’ level (Learning/Explanation & Prediction).

9.6 Guiding Assumptions Are Associated with Paradigm-Specific Quality Criteria

279

Statistical conclusion validity Statistical conclusion validity refers to whether you can draw appropriate conclusions from the statistical procedures used to analyse the data gathered in your research. The idea here is that using inappropriate statistical procedures to analyse data can lead to misleading and indefensible conclusions. In short, statistical conclusion validity concerns the appropriateness of the outcomes resulting from applying statistical procedures to quantitative data. This is an important quality criterion under Positivist assumptions because it focuses on how well you (Perspective/Researcher) use appropriate statistical procedures to objectively test (Stance/Objectivist) the existence (Locus/Externalised reality) of patterns and relationships (connections between constructs, some or all of which you may have theorised (Theory/Apriori)) by statistically controlling, where possible, for alternative possible explanations for the observations and relationships observed (Control/High or Partial) and by providing a logical basis for making statistical inferences from sample to population for purposes of explaining and predicting observations and relationships (Learning/Explanation & Prediction). Relationships amongst the validity criteria Figure 9.3 depicts the relationships between the four types of validity for Positivist research, conceptually mapped against the timeline for a research project. Construct validity is the first type of validity to focus on as it concerns the nature and quality of the measurements that give rise to the quantitative data you gather. Construct validity (supported by reliability of measurement) then underpins the achievement of internal validity (arguably the most important type of validity, hence its emphasis in Fig. 9.3) in association with the nature and quality of your research configuration (what is traditionally called ‘research design’ under Positivist assumptions). Statistical conclusion validity, through your choices of appropriate statistical analysis procedures, facilitates the achievement of both internal validity (as a pathway to establishing the covariation, i.e., correlation or association, between constructs) and external validity (facilitating generalisations to relevant

Timeline for the Research Construct Validity [supported by Reliability]

“Quality of measurements”?

supports

Internal Validity “Quality of research configuration (or ‘design’)”?

External Validity

Statistical Conclusion Validity

“Quality of generalisations”?

“Quality of analyses”?

Fig. 9.3 Relationships between quality (i.e., validity) criteria under the Positivist pattern of guiding assumptions

280

9 Why Should I Think About Guiding Assumptions?

populations) and, where appropriate, may assist in supporting claims about construct validity. Equally, construct validity and reliability can help support statistical conclusion validity through reduction of measurement errors. Finally, at the end of your research, external validity becomes relevant as you make your claims for the generality of your findings (which themselves may be supported by internal validity). The small opposing arrows, superimposed on the two-way link between internal and external validity, convey the idea that these two types of validity tend to work in opposition to each other. When you try to enhance one type of validity, the tendency will be for the other type to be lower. That is, pursuing high internal validity (typically, through the exercise of more control over context) will generally mean trading off or sacrificing some degree of external validity (usually because samples or participant experiences become less representative of ‘real-life’ populations and/or contexts). On the other hand, pursuing high external validity (typically, through obtaining larger and more representative samples, using more naturalistic contexts and representatively designing participant experiences) will generally mean trading off or sacrificing some degree of internal validity (as some control over context is usually lost).

9.6.2

Quality Criteria Associated with Interpretivist/ Constructivist and Other Non-positivist Patterns of Guiding Assumptions

Quality criteria for research conducted under Interpretivist/Constructivist guiding assumptions have emerged much more recently than validity criteria. Part of the reason is that, for a time, researchers working under Interpretivist/Constructivist and other non-positivist patterns of guiding assumptions had to try to adapt the Positivist validity criteria as standards to try and meet in order to get their research published (see, for example, Golafshani, 2003; Healy & Perry, 2000; LeCompte & Goetz, 1982). This was problematic for two reasons: (1) it privileged ‘validity’ as the gold standard for all research, meaning that validity criteria always had Positivist overtones associated with them, and (2) validity criteria ultimately proved to be ill-suited to the task of assessing the quality of Interpretivist/Constructivist research. As a consequence, new quality criteria evolved to provide a clearer, better-fitting and fairer approach to judging the quality of interpretivist/ constructivist research. Not all authors agree on these new criteria, so there remains some contention in the field. For most purposes, the following four quality criteria, crystallised by such authors as Fossey, Harvey, McDermott, and Davidson (2002), Golden-Biddle and Locke (1993) and Lincoln, Lynham, and Guba (2011), are appropriate for gauging the quality of research guided by Interpretivist/ Constructivist guiding assumptions: transparency, authenticity, sufficiency and transportability. When discussing these four quality criteria below, we will refer to

9.6 Guiding Assumptions Are Associated with Paradigm-Specific Quality Criteria

281

the first six pivotal questions shown in the guiding assumption choices patterns of Fig. 9.1. Transparency The criterion of transparency focuses on the openness with which you display all aspects of the research journey you have undertaken. This encompasses how you made choices about who you spoke to or observed or what you read, how the data were gathered, recorded, prepared and analysed, what you did and thought in the context of your data gathering activities (perhaps including how you felt) and how you might have impacted on the context during the course of your research. One of the key guiding assumptions in Interpretivist/Constructivist patterns is subjectivity where you seek to understand the perspectives/world views of the participants in your research rather than imposing your own perspective (as is done under objectivist Positivist guiding assumptions). Accordingly, transparency also must address how you have managed your own preconceptions, beliefs and ideas to minimise your influence on data source choices, contextual choices and data gathering and analysis processes as you pursued the perspectives of others. One of the best tactics you can use to enhance transparency under Interpretivist/ Constructivist guiding assumptions is to maintain a complete research journal that can also double as an additional data source for your research (recall the illustrations in Chap. 3). Transparency is an important quality criterion under Interpretivist/Constructivist patterns of guiding assumptions because it focuses on the actions you take, as researcher, to gain access to, record and interpret the meanings and the perspectives (Locus/Internalised ‘realities’) of the participants and/or other data sources (Perspective/Participants) as well as to manage your own beliefs, preconceptions and emerging ideas to keep you from influencing the perspectives you are gaining access to (Stance/Subjectivist) and allow deeper understandings and insights to emerge (Learning/Deep understanding). Transparency helps others, who might read or use your research, to understand exactly what you did every step of the way. Authenticity (or Trustworthiness) Authenticity (sometimes referred to as trustworthiness or genuineness) concerns how well the data you have gathered have captured/conveyed the genuine and authentic perspectives of your research participants, without taint from your own perspective. Without authenticity, you cannot claim to have understood your participants’ points of view or gained insights into their world views. Authenticity must be reflected not only in your data gathering and recording processes but also throughout your data analysis and reporting processes. To enhance authenticity, you must seek, as far as possible, to keep faith with and appropriately and accurately reflect the genuine words, voices and views of your participants and keep your analyses as close to the data as possible. Authenticity is arguably the most important quality criterion under Interpretivist/Constructivist patterns of guiding assumptions because it focuses whether or not you actually gain access to the genuine meanings and perspectives (Locus/Internalised ‘realities’) held by your

282

9 Why Should I Think About Guiding Assumptions?

participants and/or reflected in other data sources (Perspective/Participants) in order to allow deeper understandings and insights to emerge, untainted by your potential biases and preconceptions (Stance/Subjectivist and Learning/Deep understanding). Sufficiency (or Completeness) Sufficiency concerns whether you have tapped into an appropriately diverse set of voices and views in the research context to an appropriate depth to achieve the understandings pursued. The goal for sufficiency is for you to build on a diversity of perspectives to achieve a well-rounded and convincing interpretation of meanings and views within the research context. This includes seeking out non-mainstream, minority, marginalised, disenfranchised or disempowered perspectives, where necessary, to tell a complete story. Sufficiency concerns not only who you talked to or observed or what is read, but also why and about what and to what depth. It focuses on sampling, of course, but also has wider concerns as well, such as which topics of conversation are followed and the degree or depth of engagement you display with each of your data sources. Sufficiency is an important quality criterion under Interpretivist/Constructivist patterns of guiding assumptions because it focuses whether or not you gain access to genuine voices, meanings and perspectives (Locus/Internalised ‘realities’) from a sufficiently diverse set of relevant data sources (Perspective/Participants) in order to develop and convey well-rounded and deeper understandings and insights (Stance/Subjectivist and Learning/Deep understanding). Transportability Transportability, under an Interpretivist/Constructivist pattern of guiding assumptions, is the only quality criterion that may be considered optional. Transportability focuses on where, for whom and why the meanings and interpretations you have surfaced in your research may also be meaningful in other contexts for other people or groups at other times and in other circumstances. Often, in Interpretivist/ Constructivist research, you may not seek transportability—rather, you seek to tell the story relevant only to the research context at hand and you must keep faith with this intention throughout your processes of assembling and presenting that research story. If transportability is your intention, this intention should be clearly reflected in your research configuration (using such tactics as multiple case studies or multi-site data gathering). Often, a case for transportability can be made if you make a deliberate attempt to illuminate convergences and divergences (i.e., compare and contrast) between different contexts being researched (such as comparing the stories and interpretations from two or more different companies or schools). Transportability is an important quality criterion under an interpretivist/ constructivist pattern of guiding assumptions only when and where you intend it to be relevant, because it focuses whether or not you are able to sensibly argue that what you have learned from the voices, meanings and perspectives of the participants and/or other data sources (Perspective/Participants and Locus/Internalised

9.6 Guiding Assumptions Are Associated with Paradigm-Specific Quality Criteria

283

Timeline for the Research “Openness /completeness of research story”? if intended, enhances the relevance of

Transparency dynamic interplay

Authenticity

Sufficiency

“Coverage of relevant stories/voices”?

Transportability

may or may not be intended?

“Stories/interpretations meaningful for others”?

“Trustworthiness of data/stories and researcher’s representations of such”?

Fig. 9.4 Relationships between quality criteria under patterns of interpretivist/constructivist guiding assumptions

‘realities’) within the specific research context is relevant for and provides potentially meaningful insights into other contexts (Learning/Deep understanding). Relationships amongst Interpretivist/Constructivist quality criteria You may have already detected that there are synergies between the various quality criteria for research guided by interpretivist/constructivist assumptions. Figure 9.4 tries to capture these synergies. Throughout the entire research process, authenticity (arguably the most important quality criterion, hence its emphasis in Fig. 9.4) and sufficiency feed mutually off each other (seeking sufficient diversity in authentic perspectives) and both may be further enhanced by close attention to transparency. Equally, transparency can be enhanced through clarity in conveying what you have done to enhance authenticity and sufficiency. Transportability, if intended, will consist of arguments made toward the end of your research, but your intention also needs to be reflected earlier on in choices of research configuration, data gathering and sampling strategies, which will in turn make sufficiency a more relevant criterion to meet.

9.7

Paradigm-Independent Meta-Criteria for Judging Research Quality

Thus far, we have looked at quality criteria specific to certain patterns of guiding assumptions. The operative word here is ‘specific’. Quality criteria under Positivist assumptions (i.e., validity criteria) have little meaning under Interpretivist/ Constructivist patterns of guiding assumptions (despite historical attempts to force them to) and vice versa. This makes it difficult, if not impossible, to compare the quality of research conducted under different patterns of guiding assumptions,

284

9 Why Should I Think About Guiding Assumptions?

because no quality criterion is shared with all patterns (highlighting what philosophers call the problem of paradigm ‘incommensurability’). This problem is further exacerbated if you include practice-based research in the mix, which require a focus on the practical utility of research outcomes (see, e.g., Bleijenbergh, Korziluis, & Verschuren, 2011). A number of researchers have attempted to identify higher-order more inclusive research quality criteria (e.g., Bryman, Becker, & Sempik, 2008; Caracelli & Riggin, 1994; Healy & Perry, 2000; Krathwohl, 1985; Le Compte & Goetz, 1982; O’Cathain, 2010; Sale & Brazil, 2004), but none has really taken research quality criteria to the next level as complex systems thinking requires. Additionally, the frameworks that have emerged have tended to be very cumbersome where the same two tactics appear to be in play: either additional conceptual torque is placed on Positivist or ‘traditional scientific’ quality criteria or unique quality criteria are devised for research conducted under different guiding assumptions or different data types. For example, O’Cathain (2010) proposed a comprehensive framework of quality criteria that encompassed 8 different domains and 44 different criteria. Patton (2002, pp. 544–545) identified no fewer than five alternative sets of quality criteria that could be relevant to qualitative inquiry: traditional scientific research criteria, social construction and constructivist criteria; artistic and evocative criteria; critical change criteria; and evaluation standards and principles. Accordingly, we still need to deal with the following two questions: (1) how can you evaluate the quality of a social or behavioural research outcome independently of the pattern of guiding assumptions that underpin that research; and (2) how can you compare the quality of research investigations that are guided by very different patterns of guiding assumptions? To address both questions, Cooksey (2001, 2006, 2008) developed a simpler logical system of higher-order criteria for judging the quality, coherence and value of any type of social and behavioural research, irrespective of paradigmatic assumptions and alignments; i.e., quality criteria that can be applied across all patterns of guiding assumptions. These higher-order criteria are referred to as meta-criteria in order to clearly signal that they are intended to operate above and beyond any localised paradigm-specific quality criteria. It is important to note that meta-criteria do not replace paradigm-specific quality criteria; they subsume them under a wider and more systemic umbrella. Additionally, as we will show later on, meta-criteria are equally useful for both evaluating research quality and for planning quality research. The focal point of the meta-criteria is the assumption that, in the end, research is fundamentally a ‘storytelling’ activity (Daft, 1983) and that research quality is reflected in the impacts of that story on its readers/users. The research act itself focuses on conceiving, constructing and displaying all the necessary components of that story. This logic is similar to Lincoln’s (1995) arguments for the ‘inquiry community’ as the ‘arbiter of research quality’ in qualitative and interpretive research but extends those arguments to research conducted under any particular set of guiding assumptions. Quality is inherent in the configuring and execution of the research itself in that you, as researcher, can do things as part of your research to improve quality. This is where paradigm-specific quality criteria exert their major

9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality

285

influence. However, and more importantly, quality is vested externally in the value of your research as seen by relevant others. Thus, inferences about the quality of your research may be drawn by those who review or examine it, read it, fund it, build upon it, change practices based on it, supervise it, use it, apply it, provide input to it, critique it, learn from it, take ownership of it and share knowledge arising from it with others. In the end, it does not matter if you think your research is of high quality, it is the judgments of quality made by others that matter. Thus, you need to be oriented toward convincing those others of the quality and value of your research. Figure 9.5 displays a three-level mind map of the system of meta-criteria, liberally adapted from Cooksey (2008). There are 12 meta-criteria in the system. The 12 meta-criteria are grouped into three higher-order domains, which then inform judgments about overall research quality, captured in the concept of convincingness. As well, there are interconnections and dynamic interplay between specific meta-criteria as well as between the three domains of meta-criteria, which means that focusing on quality in one area may enhance or have implications for quality in another area. In discussing the system of meta-criteria, we will work from the centre of the mind map outward. Throughout the discussions below, we will implicate you as the postgraduate researcher planning/evaluating your own research activities. However, later, we will show that these meta-criteria can be used by you as well as by others to evaluate the quality of another researcher’s work.

9.7.1

Convincingness

Convincingness constitutes the central quality judgment and addresses the most important question regarding research quality: is the research story, as an integrated whole, convincing with respect to the arguments being made? That research should be convincing to a relevant audience is not a new idea since it begs the question: convincing to whom? Golden-Biddle and Locke (1993) talked about research as being convincing and that being convincing has various dimensions. However, their focus was solely on research conducted under Interpretivist/ Constructivist patterns of guiding assumptions. Aguinis et al. (2010), Duncan and Harrop (2006) both presented ‘consumer-centric’ perspectives on research quality but offered no meta-criteria for judging quality that would work independently of pattern of guiding assumptions. Cooksey’s (2008) intention, from a systems thinking perspective, was to identify dimensions of convincingness that could be universally meaningful for any social or behaviour research endeavour, irrespective of the particular pattern(s) of guiding assumptions that had been or were being adopted. Convincingness is influenced by and, in most cases, is explicitly linked to, a specific presentation of the research story (e.g., an outcome such as a proposal, report, thesis, dissertation or portfolio, conference paper or presentation, published

286

9 Why Should I Think About Guiding Assumptions?

Fig. 9.5 Three-level mind map of the 12 meta-criteria and their inter-relationships in pursuit of convincingness (adapted from Cooksey, 2008, Fig. 9.1)

article). Telling a convincing research story comprises attending to those aspects that add power, weight and value to that story in ways that effectively reach one or more specific audiences while at the same time managing or accounting for any emergent issues or concerns having the potential to confuse or devalue that story. It also requires that you know who potentially needs to be convinced by your research.

9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality

9.7.2

287

Three Domains of Meta-Criteria

Convincingness is informed by answers to three principal questions about research quality, where each question implicates a specific domain of relevant meta-criteria (refer to Fig. 9.5): 1. How well has your research been contextualised? This question defines the Contextualisation domain of meta-criteria, which establishes the boundary conditions, research goals, questions and/or hypotheses, opportunities, constraints and guiding assumptions surrounding and underpinning the research, the research context(s), and your positioning/status as researcher and the positioning/status of all relevant data sources. Contextualisation meta-criteria also help to situate your research within relevant larger systemic contexts, including the extant literature, organisations, institutions and society as well as your own context (including guiding assumptions and paradigm choices), the context(s) relevant to research participants and other data sources and the context(s) relevant to potential research users (anticipating possible research outcomes). Thus, these meta-criteria capture the dynamics and specifics associated with your contextualised positioning of an instance of research. 2. How appropriately have your research processes and learning been realised? This question defines the Realisation domain of meta-criteria, which focuses on the methodological steps taken, configurations of data gathering strategies adopted, and procedures and technologies employed to access, assemble, analyse and establish/construct patterns, relationships, meanings, interpretations and extensions using the information/data collected during your research processes. In short, Realisation concerns how well you have moved from the contextualised conceptualisation and planning stages of the research to the point where what you have learned in your research can be shared. It is in the context of the Realisation meta-criteria that we also explicitly look for congruence between your choices of and ‘localised’ quality criteria associated with the pattern(s) of guiding paradigm assumptions you adopt. Thus, Realisation meta-criteria capture the dynamics and specifics associated with your contextualised execution of an instance of research. 3. How appropriately and carefully has your research been explicated? This question defines the Explication domain of meta-criteria, which focuses on how and how well the story of your research, in the context of a specifically-produced research outcome, is configured and presented, how well you have or will handle unexpected findings, what limitations and caveats apply to your research and what that story might mean for other researchers, for theory, for methodology, for practitioners and/or for other relevant stakeholder groups. The research outcome may be anticipatory in intent (as with a research proposal) or retrospective (as in a thesis or journal article). The focus here is on meeting the needs and expectations of anticipated research users and audiences through the sharing of specific research outcomes. Thus, Explication meta-criteria capture the dynamics and specifics associated with the

288

9 Why Should I Think About Guiding Assumptions?

contextualised emergent stories from an instance of research and their associated implications and, where relevant, applications. This meta-criterion domain is most closely related to and predictive of research impact. The higher-order Contextualisation, Realisation and Explication domains of meta-criteria dynamically interconnect, cross-influence and, often, trade-off with each other to build a convincing case for your research and its findings, conclusions and implications. The 12 meta-criteria to be discussed in more detail below share important dependencies with each other as they work together to inform judgments about the Convincingness of research for given purposes. These meta-criteria should not be expected to be of equal importance/relevance in all research investigations. This means that an important aspect of judging Convincingness is reflecting on and applying an appropriate balance in emphasis among the meta-criteria, given the contexts, intentions and emergent textures of your research. Contextualisation Meta-Criteria Figure 9.6 presents an expansion of the Contextualisation domain of the three-level mindmap in Fig. 9.5. Each meta-criterion in this domain now has branches that provide additional details associated with each of the relevant meta-criteria. Juxtapositioning with Other Research This meta-criterion focuses on the role(s) that the extant literature plays in your research, something we will explore more fully in Chap. 13. A convincing story will coherently and critically argue for how and where your research is situated relative to previous relevant research. This meta-criterion forms an important aspect of research contextualisation because it is often the starting point for you to build the case for your research. Juxtapositioning requires you to engage in critical thinking and evaluative reflection on past research and the system of meta-criteria can actually help in this regard. Often, what contributes to Convincingness is the emergence, through Juxtapositioning with Other Research, of a niche for your research, an innovative turn or twist, theoretical development or extension, new contextual application or a logical or methodological gap where you can see that research is needed. Researcher Positioning This meta-criterion reflects the importance of understanding where/how/why you are situated in the research context. A research story that displays your cultural status as well as constraints encountered, and opportunities pursued, reinforces a more open and convincing account. This meta-criterion demands that you critically reflect on your own role(s) in the research context. Some roles may need to be negotiated with gatekeepers who control access to research contexts or resources and, where relevant, with others involved in the research, such as postgraduate supervisors, research team members, research participants, institutional or organisational partners and/or contracting or funding organisations. Researcher positioning also concerns how clearly and logically you argue for the goals of your

Fig. 9.6 Expansion of the contextualisation domain of the three-level mindmap showing its four meta-criteria with expanded branch details (adapted and expanded from Cooksey, 2008, Fig. 9.1)

9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality 289

290

9 Why Should I Think About Guiding Assumptions?

research, adopted pattern(s) of guiding assumptions, and the development of your research questions/hypotheses to be investigated. In the context of a professional doctorate, researcher positioning may need to include discussion of your researcher role vis-à-vis any organisational/institutional managerial roles you hold in the research context (e.g., you would need to clarify any tensions between demands as a researcher and demands as an organisational member with responsibilities and how this tension may influence your researcher role). There is an important dynamic here with the Juxtapositioning with Other Research meta-criterion when building the case for your research goals and research questions/hypotheses using the literature is one important basis for your arguments. Positioning of Participants and Other Data Sources This meta-criterion reflects the importance of knowing where/how/why the participants and any other data sources (e.g., documents, recordings, images, videos, performances) are situated and relevant with respect to the research context. A research story that explores participants’ contexts, expectations, concerns and vested interests in the research being conducted and/or the contexts, credibility and positioning of relevant documents and other handiworks of human activity also reinforces a more open and convincing account. This meta-criterion requires you to critically reflect on any power dynamics/relationships that may exist between you and participants in the research context and the implications these might have for research quality and value. This is especially important if you are conducting research in an organisation you work for (as many professional doctorate postgraduates do). Working with this meta-criterion explicitly acknowledges that participants also have differing roles, backgrounds and expectations about the research, thus recognising that they have voices to be heard. This creates an important dynamic or dependency with Contextual Sensitivity (discussed below) concerning the choices of organisations, participants and other data sources for inclusion in your research along with the ethical and social responsibility considerations that such choices may evoke. Contextual Sensitivity This meta-criterion captures the extent to which the implementation, findings and conclusions of research are appropriately sensitive to locally-relevant contextual choices and considerations. Openly reflecting such sensitivity will enrich both the contextualisation and the realisation of the research, may facilitate relationships with key potential gatekeepers and stakeholders and may enhance the prospects for meeting Extensional Reasoning and Value for Learning meta-criteria expectations (discussed below). Contextual Sensitivity should be taken to include any personal, ethical, political and organisational considerations associated with configuring any data gathering protocols, tasks, interventions or treatments and identifying/choosing data sources in specific contexts. As well, Contextual Sensitivity means being responsive to contextual nuances for data gathering, such as physical and social environment, time and place, as well as data interpretations. Even where such contextual considerations are beyond your control as researcher, through external

9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality

291

imposition (such as where a school principal rules that you may not treat students differently within a classroom cohort), a convincing research story should be open about the perceived potential impact of such considerations. Realisation Meta-Criteria Figure 9.7 presents an expansion of the Realisation domain of the three-level mindmap in Fig. 9.5. Each meta-criterion in this domain now has branches that provide additional details associated with each of the relevant meta-criteria. Internal Coherence This meta-criterion focuses closely on how well your research works as an integrated set of activities designed to address your specific research goals, questions, hunches or suspicions, anticipations or hypotheses/predictions. One important consideration is how well your entire research process works as a logical sequence of choices and actions to both inform and support your inferences, interpretations, conclusions and implications. Congruence between pattern of guiding assumptions and the data gathering configurations and strategies you have implemented is another important consideration under this meta-criterion. That is, this is one meta-criterion where paradigm-specific quality criteria have an explicitly recognised role to play. Under the Positivist pattern of guiding assumptions, Internal Coherence would depend, in part, upon meeting the expectations of construct and internal validity. Under Interpretivist/Constructivist patterns of guiding assumptions, Internal Coherence would depend, in part, on transparency, authenticity and, in certain cases, sufficiency. Against this meta-criterion, a convincing study should provide, from a range of possible alternative accounts, the strongest warrant for the conclusions drawn, given the quality of the positioning, configuration and execution of your research. There are some critical inter-dependencies between Internal Coherence and Analytical Integrity (discussed below) through showing congruence between your analytical choices and the positioning, configuration and execution features of your research, Presentational Character (discussed below) through informing the story you tell about the positioning, configuration and execution of the study and Contextual Sensitivity, where influences in or emerging from the research context may add to or detract from the intended positioning, configuration and execution of your research. Analytical Integrity Here, we focus on the quality and implementation of your choices of analytical technologies, pathways and approaches, consistent with your choice of pattern of guiding assumptions, for making sense of the data that you gather. For example, under the Positivist pattern of guiding assumptions where quantitative data are gathered, statistical conclusion validity becomes relevant. Under Interpretivist/ Constructivist patterns of guiding assumptions, keeping interpretations closely connected to the data and to the voices of participants, while managing your own preconceptions becomes relevant. Convincing research will show how and why you made specific analytical choices and interpretations, argue for their appropriateness/

Fig. 9.7 Expansion of the Realisation domain of the three-level mindmap showing its three meta-criteria with expanded branch details (adapted and expanded from Cooksey, 2008, Fig. 9.1)

292 9 Why Should I Think About Guiding Assumptions?

9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality

293

suitability to the task at hand and sensitise what you have learned against the quality and character of the available data and the context(s) in which they were gathered (interaction with Contextual Sensitivity). This meta-criterion will dynamically interact with Presentational Character (discussed below) in that analytical choices, particularly those related to displays of data patterns and relationships, should be sensitive to audience expectations and competencies with attention to how you should communicate findings, meaning and interpretations most clearly and Extensional Reasoning (discussed below) in that your analytical choices may inhibit or enhance the reach of the stories emerging from your quantitative and/or qualitative data analyses. Extensional Reasoning The final Realisation meta-criterion focuses on your intentions and reasoning for extending the findings from localised research context(s) to other contexts of potential interest, including other participants, groups, organisations, cultures, times, situations and places. Here, you establish the defensible boundaries and extent of reach for meanings and implications of your research findings, against the background of the contexts, constraints, configuration and execution of your research. This is another meta-criterion where paradigm-specific quality criteria have an explicitly recognised role to play. For example, under the Positivist pattern of guiding assumptions, external validity and representative design of tasks and conditions becomes relevant. Under Interpretivist/Constructivist patterns of guiding assumptions, sufficiency and, where appropriate, transportability, become relevant. Extensional Reasoning shares an important inter-dependence with the Value for Learning meta-criterion (discussed below) in that the boundaries and reach of conclusions help to shape where value for learning from your research might be realised. Extensional Reasoning also establishes the nature and role of any forecasts, explanations and speculations that you may produce to help make sense of your data patterns, explain why things happened the way they did and argue for use and applications by others in the future. Explaining why things happened the way they did creates important inter-dependencies with the meta-criteria for Handling of Unexpected Outcomes (discussed below) in terms of how you handled data anomalies during your analysis processes and Acknowledgement of Limitations (discussed below) in terms of what limits and caveats apply to the extensions and implications of your conclusions. Explication Meta-Criteria Figure 9.8 presents an expansion of the Explication domain of the three-level mindmap in Fig. 9.5. Each meta-criterion in this domain now has branches that provide additional details associated with each of the relevant meta-criteria. Value for Learning The Value for Learning meta-criterion captures the extent to which you make the implications of your findings/conclusions logical and transparent and closely connected to your goals and questions/hypotheses. Convincing research will clearly

Fig. 9.8 Expansion of the Realisation domain of the three-level mindmap showing its five meta-criteria with expanded branch details (adapted and expanded from Cooksey, 2008, Fig. 9.1)

294 9 Why Should I Think About Guiding Assumptions?

9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality

295

indicate where its important contributions are intended to be. Value for Learning needs to focus on relevant context(s) for relevant research readers/users, highlighting the nature of the learning and where and how it is situated with respect to academics, other researchers, professionals, managers, organisations, communities, markets, potential/actual innovation adopters and/or practitioners. In short, Value for Learning is all about the emergent ‘take-home’ messages from your research story. Thus, it answers the ‘so what’ question for the research reader/user. Value for learning may be displayed for theory, methodology, practice, action, policy, education and development, management and many other points of focus and may convey important lessons for development, innovation, improvement and change in social and organisational contexts. In action learning research, for example, Value for Learning may stimulate shifts in the positioning of both yourself, as researcher, and your participants (creating dynamics with Researcher Positioning and Positioning of Participants and Other Data Sources) as well as reflecting Contextual Sensitivity to relevant cultures, communities, institutions or organisations with respect to how change unfolds. Fertilisation of Ideas The Fertilisation of Ideas meta-criterion is explicitly forward-looking in intent. It involves an element of futures thinking. Fertilisation of Ideas explicitly captures the notion that convincing research should stimulate interest in and possible guidance for future research and signal, where relevant, any potential innovations, evaluations and/or follow-up applications or interventions by practitioners that might flow from your research (in this way, it shares a dynamic with Value for Learning). Convincing research should also leverage possible or actual blind alleys and dead ends you encountered in ways that may assist future researchers to avoid or circumvent the traps your current research may have fallen into. Thus, we can see that there is a dynamic interplay with the Acknowledgement of Limitations meta-criterion (discussed below). Furthermore, convincing research poses unanswered questions arising from your research which can point to directions for future research. This meta-criterion has a built-in delayed feedback loop linked to whether your research story is read (or heard) and cited/used/applied/adopted by future relevant others and whether it has an influence on some future relevant contexts. Meeting expectations against this meta-criterion can therefore provide you with opportunities to learn about the ‘impact’ of your research. Handling of Unexpected Outcomes For your research to be convincing, you must openly discuss and actively deal with any unintended or unanticipated, i.e., surprising, outcomes arising from your research. Your discussion should logically set out how you handled such outcomes without being defensive or apologetic. These outcomes may range from anomalies in the data (e.g., quantitative data that do not meet appropriate statistical assumptions; qualitative stories that contradict previous accounts or reveal a surprising or very unique perspective) to unanticipated constraints imposed on you that affect data quality or quantity (e.g., an organisation withdraws its participation from your

296

9 Why Should I Think About Guiding Assumptions?

research or influences data gathering activities in ways you did not anticipate; poor questionnaire response rates). How you should deal with such outcomes will very much depend upon your adopted pattern(s) of guiding assumptions and the methodologies and analytical strategies you employed (creating dynamics with all three Realisation meta-criteria). Handling of Unexpected Outcomes can have a particularly powerful positive or negative influence on Convincingness depending on whether you attempt to logically and openly account for such outcomes or instead attempt to explain them away, ignore them or gloss over them. Handling of Unexpected Outcomes shares important dynamics with the Acknowledgement of Limitations meta-criterion discussed below (e.g., recognising where limitations occurred in your research processes or where contextual insensitivity may have produced unexpected outcomes) and the Fertilisation of Ideas meta-criterion (e.g., using the unexpected outcomes as springboards for identifying future directions and areas for research). Acknowledgement of Limitations The Acknowledgement of Limitations meta-criterion reflects the expectation that convincing research should incorporate your critical reflections on the limitations, constraints and difficulties you encountered through your research processes. Many limitations tend to be dynamically created when you make trade-off decisions, often on the fly, to move from what might be ideally desirable to implement in the conduct of your research to what is feasible to implement, given the constraints you encounter. Acknowledgement of Limitations shares an important dynamic with Handling of Unexpected Outcomes in that limitations that you could not effectively manage or circumvent may produce some of the unexpected outcomes that you need to address. Like Handling of Unexpected Outcomes, Acknowledgement of Limitations can have a very powerful positive or negative influence on Convincingness depending on whether you actively engage in critical thinking about and reflection on your own research by openly discussing its limitations or instead avoid critical thinking by glossing over, ignoring or discounting them. Effective Acknowledgement of Limitations can enhance Convincingness by providing a backdrop for clearer and more appropriately bounded arguments for Value for Learning and Extensional Reasoning as well as helping to make a stronger case for Fertilisation of Ideas. Presentational Character The final Explication meta-criterion is Presentational Character, which focuses on who needs to be convinced by your research and how you might best convince them. Here, we explicitly recognise the central role played by audience(s) or intended users in helping you to produce a quality research outcome. Conveying a story that targets the wrong audience, that is pitched at the wrong level and/or that is poorly crafted, lacking logical coherence and clarity in writing, are some of the quickest ways to damage the Convincingness of your research investigation (and possibly your own credibility). This means that Convincingness can be strongly influenced by the strategic choices you make when writing, delivering or displaying

9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality

297

your story, above and beyond any inherent strengths your research processes might possess and how compelling your investigation was to pursue. Deficiencies in Presentational Character can have a powerful perceptual as well as cognitive impact on research readers/users, which may override strong positions on Contextualisation and Realisation meta-criteria. Even minor conceptual or mechanical difficulties (e.g., using the wrong report format or writing/formatting style, inappropriate modes of expression, poor vocabulary choices, inappropriate representation for a theoretical or conceptual framework, errors in statistical reporting, tables or figures; grammatical errors; lack of proof-reading evidenced by many typographical errors) may negatively influence Convincingness, even where you have correctly targeted research readers/users (e.g., examiners for a postgraduate thesis, academic readership/reviewers/editors for a specific journal, research colleagues, managers, employers, practitioners, even laypeople, such as members of a community or workforce). On the other hand, a research process that is weaker against one or more Contextualisation or Realisation meta-criteria may recover some impact potential and standing with research readers/users through strong Presentational Character. There would, however, be an important dynamic to recognise here: this ‘quasi-recovery’ is likely only to happen if your story is also strong with respect to Acknowledgement of Limitations, Value for Learning and Handling of Unexpected Outcomes. Presentational Character is where you need to pay attention to the appropriate balance in emphasis/weighting on various meta-criteria, depending upon intended research reader/user audience(s) and the purposes and anticipated uses of your research.

9.7.3

Working with the Meta-Criteria

How can these meta-criteria help to enhance the meaningfulness, quality and cumulative nature of your research? Firstly, the meta-criteria reinforce the idea that research processes need to be seen in a more realistic as well as holistic light as a contextualised, constrained and very human activity, subject to numerous influences. They also promote systems thinking to help you combat the tendency to reify specific research paradigms and approaches (i.e., negating paradigm arrogance). Secondly, the meta-criteria can facilitate your identification or recognition of (1) critical decision points (i.e., places that emerge during your research journey where a choice you made altered the pathway for your research in a productive or positively adaptive way or counter-productive negative way) and (2) constraints within your research context and the wider contexts in which it is situated, that had the potential to impact on research quality in various ways. By anticipating and making decisions on how to best handle emergent constraints and take action at critical decision points, research quality can be enhanced. Remember, though, that convincing research also needs to meet the expectations of any relevant paradigm-specific quality criteria in addition to the expectations of the various

298

9 Why Should I Think About Guiding Assumptions?

meta-criteria and that all relevant paradigm-specific quality criteria and all the meta-criteria may not be of equal importance for a particular instance of research. Research Planning Researchers, supervisors and postgraduate research students (e.g., masters, professional doctorate, PhD) can employ the meta-criteria mindmaps as planning guides to provide an annotated record of guiding assumptions held, choices made, configuration and procedural logic employed, contextual qualifications and related matters and to help spot potential gaps in Convincingness that emerge during the conduct of your research. As a planning tool, the mindmaps can assist you in building the case for the quality, coherence and value of your research from its initial stages to finished convincing outcome. They can also prove useful for anticipating and recording key aspects of the research journey in your research journal as discussed in Chap. 3. You can enhance your planning focus on Convincingness through a process of mental simulation focusing on your research processes—asking yourself ‘what if’ questions and trying to anticipate the implications of potential decisions you might make. This process can also highlight the necessity for devising ‘Plan B’s’ for various aspects of your research processes, in anticipation of the emergence of critical constraints. Critical decision points could include: • • • • • • • • • •

choice of research context(s); funding source(s) to pursue; choice of pattern(s) of guiding paradigm assumptions; choice of approaches—using singular or pluralist logic; selection and configuration of data gathering strategies (coupled with the previous point); decisions on how best to manage stakeholder input, political and ethical dilemmas as they arise; choice of data sources; sampling choices—contingent upon accessibility and research goals; choice of data-oriented technological pathways and approaches, including choices of software support for data gathering and analyses; and/or for postgraduate researchers, choice of award/degree program (e.g., masters, PhD, professional doctorate), department/school and supervisor(s) to pursue.

The meta-criteria offer a more systemic and flexible landscape within which to energise your creative harnessing of a plurality of research paradigms and processes, research configurations, research entry and exit points, data gathering strategies and method mixtures, and positionings with respect to theory, methods and practice, where your goal is to enhance the Convincingness of your research. Finally, as shown in the three-level and four-level mindmaps in Figs. 9.5, 9.6, 9.7 and 9.8, the meta-criteria can assist in (1) directly informing appropriate evaluations of research outcomes and products, thereby circumventing problems that can emerge where paradigm-inappropriate criteria are applied to judge the quality of

9.7 Paradigm-Independent Meta-Criteria for Judging Research Quality

For Research Planning

Contextualisation sation Realisation eria Meta-criteria Meta-criteria

299

For Research Evaluation

An cipa ng how others would evaluate your research

Contextualisation sation Realisation eria Meta-criteria Meta-criteria

Learning from evalua ng other’s research outcomes

Explication Meta-criteria

Explication Meta-criteria

Fig. 9.9 Using the meta-criteria for both research planning and research evaluation

specific research endeavours and (2) reinforcing the importance of not only considering paradigm-specific research quality criteria but also looking beyond these localised assumption-specific criteria using the meta-criteria when planning and/or evaluating research processes and outcomes. When focusing on Convincingness in research planning, the Contextualisation and Realisation meta-criteria domains take centre stage in anticipation of how Explication might best unfold as symbolically represented in the Venn diagrams on the left side of Fig. 9.9. How best to meet Explication meta-criteria entails anticipating how relevant research readers/users might evaluate the study you are planning and adapting your research configuration in accordance with what you can foresee being raised as potential issues by those readers/users (realising, of course, that you will never be able to anticipate everything). Part of preparing yourself for meeting Explication meta-criteria expectations involves (1) being very clear about research configuration and procedural choices and trade-offs you have made to enhance the chances that your research will be feasible and realistic and (2) knowing the strengths and limitations of your planned research so that you can critically reflect on this knowledge in any research outcome you produce. Research Evaluation Postgraduate researchers in the context of conducting their literature review can employ the meta-criteria as guides for judging the quality of a specific research story (journal article, consultancy report, conference paper, manuscript for a journal, media report, someone else’s thesis, dissertation or portfolio). If you employ meta-criteria for judging research quality, you can avoid (or at least recognise) paradigm confusion, blindness, bias or arrogance in your evaluations while ensuring that the criteria you draw upon to make your judgments remain transparent, accountable and comparable across different types of social and behavioural research. When judging Convincingness in the evaluation of others’ research, the Contextualisation and Realisation meta-criteria domains form the reflective

300

9 Why Should I Think About Guiding Assumptions?

backdrop against which the Explication domain is contextualised and judged, as symbolically represented in the Venn diagrams on the right side of Fig. 9.9. Furthermore, the meta-criteria can facilitate higher-order ‘learning’ by helping you to maintain an awareness of the importance of diversity in paradigms, choices and approaches to social and behavioural research and of the need to employ criteria sensitive to that diversity, reflecting what Flood and Romm (1996) called ‘triple loop learning’. When you are reading someone else’s research (i.e., their explication; for example, a published article you are reading for your literature review or a colleague’s research report) with a critical eye (something you should always do), you are looking to be convinced by attending to (1) how well that researcher has contextualised and realised their research processes to get to the conclusions they have drawn; (2) evidence of the researcher’s awareness of and responsiveness to emergent issues and constraints during their research; and (3) evidence of the researcher ‘s critical insights into what they have done and learned. Research proposals can also be evaluated using the meta-criteria but from a prospective (‘what the researcher plans to do’) rather than a retrospective (‘what the researcher actually did’) mindset. You can then use this evaluative learning in your own research planning process.

9.8

Key Recommendations

Some important things to remember about patterns of guiding assumptions, their association with quality criteria and use of the meta-criteria to move beyond the boundaries of specific patterns of guiding assumptions, are: • Choosing a pattern of guiding assumptions to work under means that you are making explicit choices about your research orientation and intentions and those choices will influence many of your research activities downstream. We showed though that it is a mistake to assume that patterns of guiding assumptions necessarily implicate or demand a specific type of data (quantitative and/or qualitative). There are preferences, to be sure, but any type of data can potential play a role under any pattern of guiding assumptions. • You must be clear as to the type of knowledge building you are pursuing as this can influence how your research plays out, who relevant stakeholders are and how they should be included in your research processes. Mode 1 knowledge building invokes a narrower set of stakeholders, mostly of an academic orientation, and their role in your research will largely be passive. Mode 2 knowledge building invokes a wider array of potential stakeholders and many of these may play an active role in shaping your research. Culturally-privileged knowledge building is rather different in focus and intent as it seeks to understand knowledge from within a specific cultural context and in recognition of the cultural history that preceded your research. This type of knowledge building is especially relevant to the Indigenous and Feminist patterns of guiding assumptions.

9.8 Key Recommendations

301

• Different patterns of guiding assumptions reflect different perspectives on researcher and participant positioning and relationships (more about this in Chap. 10). • There are different types of arguments you can use to defend your choice of guiding assumptions, including building contrasts with other paradigms, asserting the utility of your chosen paradigm and overcoming deficiencies imposed by the guiding assumptions reflected in previous research. • If you employ pluralist logic, you can consider using multiple sets of guiding assumptions but will need to develop clear views on the relationships between them at different stages of your research. Pluralist logic also extends to choices about positioning and contextualisation (see Chap. 10), research frames, problems and questions/hypotheses (see Chap. 11), research configurations (see Chap. 12), data gathering strategies (see Chap. 14), and data sources (see Chap. 19). This will influence how you will shape your conclusions and the story that conveys them to a reader/examiner. Pluralist logic will add complexity to your research and impose demands on you to develop a more diverse set of research skills. This can detract from feasibility given constraints you might face and, therefore, pluralism should not be seen as a requirement. However, pluralist logic, if used effectively, can greatly enhance convincingness. • Different patterns of guiding assumptions are associated with different sets of criteria for judging research quality that need to be applied and your research decisions will need to be sensitive to the particular quality criteria that are relevant for your adopted set of guiding assumptions. For the Positivist pattern of guiding assumptions, the four validity criteria are: construct validity, internal validity, external validity and statistical conclusion validity. For Interpretivist/ Constructivist and other non-positivist patterns of guiding assumptions, the four quality criteria are: transparency, authenticity, sufficiency and, where appropriate, transportability. However, such paradigm-specific quality criteria are generally not transferrable or applicable across different patterns of guiding assumptions. • In order to comparatively assess research quality independently of patterns of guiding assumptions, you should use the meta-criteria. The three domains of meta-criteria, Contextualisation, Realisation and Explication, can help you to judge the Convincingness of any research outcome. Furthermore, you can use the meta-criteria prospectively in planning/proposing research as well as retrospectively in evaluating not only your own research processes and outcomes but also those of other researchers.

References Aguinis, H., Werner, S., Abbott, J. A., Angert, C., Park, J. H., & Kohlhausen, D. (2010). Customer-centric science: Reporting significant research results with rigor, relevance, and practical impact in mind. Organizational Research Methods, 13(3), 515–539.

302

9 Why Should I Think About Guiding Assumptions?

Allard-Poesi, F., & Maréchal, C. (2001). Constructing the research problem. In R. A. Thietart (Ed.), Doping management research: A comprehensive guide (pp. 41–50). London: Sage Publications. Bleijenbergh, I., Korzilius, H., & Verschuren, P. (2011). Methodological criteria for the internal validity and utility of practice-oriented research. Quality & Quantity, 45(1), 145–156. Bowers, T. D. (2011). Towards a framework for multiparadigm multimethodologies. Systems Research and Behavioral Science, 28(5), 537–552. Brocklesby, J. (1997). Becoming multimethodology literate: An assessment of the cognitive difficulties of working across paradigms. In J. Mingers & A. Gill (Eds.), Multimethodology: The theory and practice of combining management science methodologies (pp. 189–216). Chichester, UK: John Wiley & Sons. Brunswik, E. (1952). The conceptual framework of psychology. Chicago: University of Chicago Press. Brunswik, E. (1956). Perception and the representative design of psychological experiments (2nd ed.). Berkeley, CA: University of California Press. Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8–22. Bryman, A., Becker, S., & Sempik, J. (2008). Quality criteria for quantitative, qualitative and mixed methods research: A view from social policy. International Journal of Social Research Methodology, 11(4), 261–276. Bryman, A., & Bell, E. (2015). Business research methods (4th ed.). Oxford, UK: Oxford University Press. Cameron, R. (2011). Mixed methods research: The five Ps framework. Electronic Journal of Business Research Methods, 9(2), 96–108. Campbell, D., & Stanley, J. (1966). Experimental and quasi-experimental designs for research. Chicago: Rand-McNally. Caracelli, V. J., & Riggins, L. J. C. (1994). Mixed-method evaluation: Developing quality criteria through concept mapping. Evaluation Practice, 15(2), 139–152. Charmaz, K. (2014). Constructing grounded theory (2nd ed.). London: Sage Publications. Chilisa, B. (2012). Indigenous research methodologies. Los Angeles: Sage Publications. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin. Cook, T. D., Campbell, D. T., & Peracchio, L. (1990). Quasi-experimentation. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 4, pp. 491–576). Palo Alto, CA: Consulting Psychologists Press, Inc. Cooksey, R. W. (2001). What is complexity science? A contextually-grounded tapestry of systemic dynamism, paradigm diversity, theoretical eclecticism, and organizational learning. Emergence: A Journal of Complexity Issues in Organizations and Management, 3(1), 77–103. Cooksey, R. W. (2006). Evaluating research quality: Meta-criteria for management and organisational research. In CD-ROM Proceedings of the 2006 Annual December Conference of the Australian and New Zealand Academy of Management. Yeppoon, Qld. Cooksey, R. W. (2008). Paradigm-independent meta-criteria for social & behavioural research.In CD-ROM Proceedings of the UNE Postgraduate Research Conference. Armidale, NSW. Cooksey, R. W. (2011). Yours, mine or ours: What counts as innovation? Journal of Agricultural Education and Extension, 17(3), 283–295. Cowan, L. K. (2014). Path dependence: A prism for framing constraints on adaptation in Australian dairy farms. Unpublished PhD thesis, UNE Business School, University of New England, Armidale, NSW. Creswell, J. W., & Plano Clark, L. (2018). Designing and conducting mixed methods research (3rd ed.). Los Angeles: Sage Publications. Crotty, M. (1998). The foundations of social research. St. Leonards, NSW: Allen & Unwin. Daft, R. L. (1983). Learning the craft of organizational research. Academy of Management Review, 8(4), 539–546. Denzin, N. K. (2012). Triangulation 2.0. Journal of Mixed Methods Research, 6(2), 80–88.

References

303

Duncan, S., & Harrop, A. (2006). A user perspective on research quality. International Journal of Social Research Methodology, 9(2), 159–174. Flood, R. L., & Romm, N. R. A. (1996). Diversity management: Triple loop learning. Chichester, UK: Wiley. Flood, R. L., & Romm, N. R. A. (1997). From metatheory to “multimethodology”. In J. Mingers & A. Gill (Eds.), Multimethodology: The theory and practice of combining management science methodologies (pp. 291–322). Chichester, UK : Wiley. Fossey, E., Harvey, C., McDermott, F., & Davidson, L. (2002). Understanding and evaluating qualitative research. Australian and New Zealand Journal of Psychiatry, 36(6), 717–732. Gibbons, M. (2001). Mode 2 society and the emergence of context-sensitive science. Science and Social Policy, 27(3), 159–163. Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., & Trow, M. (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. Thousand Oaks, CA: Sage Publications. Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The Qualitative Report, 8(4), 597–607. Golden-Biddle, K., & Locke, K. (1993). Appealing work: An investigation of how ethnographic texts convince. Organization Science, 4(4), 595–616. Harrison, J. L. (2003). Information scope in small service firms: A comparison of universalistic, contingency and configurational theoretical approaches. Unpublished PhD thesis, New England Business School, University of New England, Armidale, NSW. Healy, M., & Perry, C. (2000). Comprehensive criteria to judge validity and reliability of qualitative research within the realism paradigm. Qualitative Marketing Research: An International Journal, 3(3), 118–126. Heron, J., & Reason, P. (1997). A participatory inquiry paradigm. Qualitative Inquiry, 3(3), 274– 294. Hesse-Biber, S. (2012). Weaving a multimethodology and mixed methods praxis into randomized control trials to enhance credibility. Qualitative Inquiry, 18(10), 876–889. Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24(4), 602–611. Kincheloe, J. L. (2005). On to the next level: Continuing the conceptualization of the bricolage. Qualitative Inquiry, 11(3), 323–350. Kincheloe, J., & Berry, K. (2004). Rigour & complexity in educational research: Conceptualizing the bricolage. London: Open University Press. Kovach, M. (2009). Indigenous methodologies: Characteristics, conversations, and contexts. Toronto, Canada: University of Toronto Press. Krathwohl, D. R. (1985). Social and behavioral science research: A new framework for conceptualizing, implementing, and evaluating research studies. San Francisco: Jossey-Bass. Kuhn, T. S. (1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press. LeCompte, M., & Goetz, J. (1982). Problems of reliability and validity in ethnographic research. Review of Educational Research, 52(1), 31–60. Lincoln, Y. S. (1995). Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry, 1(3), 275–289. Lincoln, Y. S., Lynham, S. A., & Guba, E. G. (2018). Paradigmatic controversies, contradictions, and emerging confluences, revisited. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (5th ed., pp. 108–150). Los Angeles: Sage Publications. Mingers, J. (2014). Systems thinking, critical realism and Philosophy: A confluence of ideas. London: Routledge. Mingers, J., & Brocklesby, J. (1997). Multimethodology: Towards a framework for mixing methodologies. Omega, 25(5), 489–509. Mingers, J., & Gill, A. (Eds.). (1997). Multimethodology: The theory and practice of combining management science methodologies. Chichester, UK: Wiley.

304

9 Why Should I Think About Guiding Assumptions?

McClenaghan, P. B. (2006). The role of Vice-Chancellor in Australian higher education: A role theory analysis. Unpublished PhD thesis, New England Business School, University of New England, Armidale, NSW. Midgley, G. (2000). Systemic intervention: Philosophy, methodology, and practice. New York: Kluwer Academic/Plenum Publishers. Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48–76. Moran-Ellis, J., Alexander, V. D., Cronin, A., Dickinson, M., Fielding, J., Sleney, J., et al. (2006). Triangulation and integration: Processes, claims and implications. Qualitative Research, 6(1), 45–59. Morse, J. M., & Niehaus, L. (2009). Mixed methods design: Principles and procedures. Walnut Creek, CA: Left Coast Press. Neuman, W. L. (2013). Social research methods: Qualitative and quantitative approaches (7th ed.). Boston: Pearson Education. Nowotny, H., Scott, P., & Gibbons, M. (2001). Re-thinking science: Knowledge and the public in an age of uncertainty. Cambridge: Polity. Nowotny, H., Scott, P., & Gibbons, M. (2003). Introduction: 'Mode 2’ revisited: The new production of knowledge. Minerva, 41(3), 179–194. O’Cathain, A. (2010). Assessign the quality of mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social & behavioral research (2nd ed., pp. 531–555). Thousand Oaks, CA: Sage Publications. Olsen, W. (2004). Triangulation in social research: Qualitative and quantitative methods can really be mixed. Developments in Sociology, 20, 103–118. Plowright, D. (2011). Using mixed methods: Frameworks for an integrated methodology. Los Angeles: Sage Publications. Sale, J. E. M., & Brazil, K. (2004). A strategy to identify critical appraisal criteria for primary mixed-method studies. Quality & Quantity, 38, 351–365. Sayer, A. (2000). Realism and social science. Los Angeles: Sage Publications. Sherab, K. (2013). Gross national happiness education in Bhutanese schools: Understanding the experiences and efficacy beliefs of principals and teachers. Unpublished PhD thesis, School of Education, University of New England, Armidale, NSW. Steinhauer, E. (2002). Thoughts on an indigenous research methodology. Canadian Journal of Native Education, 26(2), 69–81. Tashakkori, A., & Teddlie, C. (Eds.). (2010). Sage handbook of mixed methods in social & behavioral research (2nd ed.). Thousand Oaks, CA: Sage Publications. Teddlie, C., & Tashakkori, A. (2011). Mixed methods research: Contemporary issues in an emerging field. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (pp. 285–299). Thousand Oaks, CA: Sage Publications. Wilson, S. (2001). What is an indigenous research methodology? Canadian Journal of Native Education, 25(2), 175–179. Wolodko, K. (2017). The emergence of group dynamics from contextualised social processes: A complexity-oriented grounded-theory approach. Unpublished PhD thesis, UNE Business School, University of New England, Armidale, NSW.

Chapter 10

How Should I Contextualise and Position My Study?

Social and behavioural research is always constrained in some way or another. Given the multiple layers and complexities associated with people and organisations embedded within social and behavioural systems (whom you might wish to access as data sources) as well as within the social and behavioural systems that you yourself inhabit as a researcher and as a person, it is not hard to see that you will have to make some fairly tough choices to help you focus and configure your research so that it is doable, while still being as convincing as it can be. Figure 10.1 depicts, using an elaborated Venn diagram, these domains of potential contextual influences on your research project. The Research, Researcher, Participants and Other Data Sources and Research Sponsor/Reader/User contexts are all embedded within a larger set of contextual influences and dynamics and overlap when your specific research project brings them together. The quadruple intersection area of the four Venn diagram circles, overlaying the area of larger contextual influences, is the region from which your specific research project emerges and evolves and, as we will see in this chapter, where your contextualising and positioning strategies are shaped. Influences may take the form of constraints which can emerge from any or all of these contexts, which adds to the complexities that you must wrestle with in deciding how to best overcome, circumvent or adapt to them. Of course, influences may also take the form of new opportunities, which can emerge from any or all of these contexts, and you must decide whether or not you wish to or are able to pursue an emergent opportunity. Constraints operate to limit perhaps even hamstring your research whereas opportunities operate to expand/extend your research. A crucial part of your research journey is thus recognising, balancing and managing the most relevant and impactful of these contextual influences, dealing effectively with constraints and being opportunistic where the need arises—all in pursuit of carrying out a convincing research project. This will be an important skill to maintain and implement in your research well beyond your postgraduate years.

© Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_10

305

306

10 How Should I Contextualise and Position My Study? Possible Influences from Larger Contexts in which your Research is Embedded

Government & legal requirements/expectations; professional development, support, expectations & relationships; ethical standards & codes of conduct; societal & environmental needs & expectations; funding body expectations; environmental & physical conditions; markets; competitors; clients, customers, global, national & cultural influences, differences & power relationships; downstream stakeholders in the research …

Research Context(s) Stakeholders in your research; gatekeepers controlling access to data sources; controllability of context, if needed; nature of research task(s), if needed; institutional power structures, management, culture(s) & expectations; physical setting(s); events, rituals & symbols …

Researcher Context(s) Your background; beliefs, hopes & fears; personality; prior experiences; interests; training; research skills & preferences; personal needs & expectations; choice of patterns of guiding assumptions; networks & relationships w/ peers, colleagues &/or supervisors; social, personal & familial contexts; institutional programs, allegiances & obligations …

Research Sponsor/ Reader/User Context(s) who needs to be convinced and their expectations; decision makers; potential/actual adopters/users of products/ innovation/other research outcomes; participants in/impacted by change production & consequences; gatekeepers controlling access to research outlets …

Participants & Other Data Sources Context(s) Background; beliefs, hopes & fears; personality; prior experiences; interests; incentives to participate or not; needs & expectations; networks & relationships w/ others; status & power relationships; social, personal & familial contexts; institutional allegiances & obligations …

Fig. 10.1 The confluence of potential contextual influences on your research

10.1

Why Is Contextualisation Important?

“Contextualising your study” means creating the landscape within which the meanings that you intend readers or users to construct from your research become apparent. Contextualisation involves clarifying your assumptions, stating your intentions and goals and drawing boundaries around your research and its context (s). For example, if you appropriately contextualise your study, it should be very clear as to (1) where you, as researcher, well as your data sources, as participants, are coming from (what can be called ‘positioning’ arguments) as well as larger contextual considerations (e.g., ethics, stakeholders); (2) what assumptions you are making in order to make your research ‘do-able’; (3) what you will and will not be addressing in your research (e.g., research goals, questions and/or hypotheses), including how you are or are not building on the work of others and your choice of research frame(s) (to be discussed in more detail in Chap. 11); (4) what contribution you see your research making to theory, methodology, practice or other areas of potential impact, and, importantly, (5) what your research context(s) are and their implications for what you can do and what you may learn. Appropriately contextualising your study will help readers or users of your research to form appropriate expectations and subsequently draw appropriate inferences and conclusions. This is a major reason why the four Contextualisation meta-criteria are so important to address. Only you, as the researcher, can appropriately contextualise your study, but your supervisor(s) can certainly be of great assistance. If you don’t clearly attend to this,

10.1

Why Is Contextualisation Important?

307

it will leave the door open for others to make unjustified inferences about your work, based on their own perceptions and reading. Such judgments may not fit well with your intentions and may lead to some adverse consequences (e.g., your research outcomes or conclusions are misconstrued; your approach or arguments are misinterpreted or critiqued using inappropriate criteria). This is precisely what you don’t want to happen because it can lead to awkward or unreasonable requests for modifications. The more explicit you make and defend the contextualisation of your research and reflect this in consistent decisions throughout your research process, the more convincing your outcomes and conclusions will be. Readers will know where you are coming from and will have a clearer idea of what constraints and opportunities this created for your research. In short, you want to leave nothing to chance! In Chap. 22, we will explore how you can make the best arguments possible when you are writing up your research, and appropriate contextualisation strategies will be indispensable in that regard so that you can tell a clear, compelling and convincing story (see, for example, Fisher, 2010, pp. 293–328). However, what you should realise here is that positioning will help you to map the terrain from which those arguments will eventually emerge. In other words, understanding and clarifying the positioning of your research via various strategies can help you formulate a clear idea of what you are trying to achieve, who you want to ‘speak’ to and what constraints and points of focus you have chosen to live with. This type of thinking, aligned with the process of shaping your research problem, can help even during the early planning stages of your research (Thomas & Brubaker, 2008, pp. 55–69). It bears some relationship to a process that some researchers have referred to as ‘finding your voice’ (Lewis & Habeshaw, 1997, pp. 79–80), or establishing your ‘stance’ (Creswell & Plano Clark, 2018) in the research. Contextualisation also involves elements tied to clarifying and explicating the guiding assumptions and paradigm commitments you are willing to make in your research (a process previously discussed in detail in Chap. 9). There are several ways you can potentially contextualise your research. Strategies for contextualisation are not mutually exclusive and you may feel that several strategies need to be melded together to best suit how your research evolves. It is critical to realise that contextualisation is not just something postgraduate students have to worry about. Professional academics and researchers must worry about this as well. Appropriate contextualisation enhances publication prospects, and lack of contextualisation greatly reduces publication chances—something that many journal editors will tell you. In fact, one of the most common reasons for rejecting a submission to a journal or for asking for major revisions and re-submission of a thesis is that the author(s) have not provided sufficient argumentation to contextualise their study and what they have learned from their research.

308

10.2

10 How Should I Contextualise and Position My Study?

What Kinds of Contextualisation Strategies Can I Employ in My Research?

One important type of contextualisation concerns the intent of your research and there are a number of strategies you could employ. • Exploring relationships: This can mean you will be looking at different kinds of social, managerial or collegial relationships between/among participants in the research. This could also include relationships with the public, with consumers or with other organisations or institutions. Different patterns of guiding assumptions will shape the nature and specificity of the relationships studied. A positivist approach will be interested in testing specific hypotheses about the nature of relationships, through the clear specification of social and relational constructs. An interpretivist/constructivist approach will be more interested in the meanings inherent in various relationships and in how they have evolved. A critical approach will be more interested in the power dynamics that can be surfaced in various relationships. In Chap. 11, we will see that this contextualisation is consistent with the exploratory research frame. • Extending existing theory: Here, your research may be providing a new twist or take on a pre-existing theory and/or incorporating new or different constructs or relationship connections. This could also include extending theory through integration of two or more theoretical perspectives. This contextualisation strategy will primarily be associated with the positivist pattern of guiding assumptions. • Developing a new theory: This contextualisation strategy involves developing a new theory. However, you should understand that this type of contextualisation is intimately linked with your paradigm assumptions and consequent methodological choices. In a positivist research investigation, a new theory is proposed and developed through leveraging what you learn from a coherent literature review and is set out for explicit testing before any data are actually collected. Here, theorising is apriori, driving your research agenda and the methodological choices you need to appropriately test deductions from that theory. It is also possible for a new theory to be advanced at the conclusion of a positivist investigation as a way of inductively pulling research findings together in a coherent way to provide guidance for future research. In an interpretivist/ constructivist approach, such as grounded theory, theory is developed by working iteratively through the data as they are collected. Thus, theorising is emergent, dynamic and heavily dependent upon methodological choices for its shape and sufficiency. Furthermore, early emergent forms of a theory are then evaluated and elaborated through more focused data collection in later stages of the research. • Replicating a test of theory in a new context: This contextualisation strategy focuses on showing whether a theory and its associated constructs and relationships transcend or generalise across specific types of boundaries (e.g., social,

10.2

What Kinds of Contextualisation Strategies Can I Employ …

309

cultural, organisational, industrial, political). A good deal of cross-cultural research implicitly requires such contextualisation. This contextualisation will likely be associated with the positivist pattern of guiding assumptions. However, you have to be careful as to how you shape a replication in a postgraduate research project, since most universities will apply a criterion related to the originality of the contribution made by your research during the examination process. Exact replications of someone else’s research are to be avoided in postgraduate research. Partial replications, if coupled with other aspect or phases of a study that involve elaboration using pluralist logic, extension of ideas, different types of data, or novel enhancements to the theory, may be acceptable. The previous three contextualisation strategies are consistent with the explanatory research frame, to be discussed in Chap. 11. • Exploring boundaries: This strategy of contextualisation brings to the surface an intention to test or examine the generality, transportability or uniqueness of a phenomenon, perspective or impact. It looks to establish the contextual reach of some type of understanding or explanation. In positivist research, this goes to the heart of external validity. However, it extends beyond simple considerations of sample size and sample representativeness, to involve considerations of the relationship between the settings, tasks and circumstances directly observed in the research and those not directly observed but about which something is to be said. This aspect of external validity has been called representative design by Brunswik (1956; see also Hammond & Stewart, 2001) and it has become an increasingly important characteristic for positivist research to possess. To illustrate, consider an operations management researcher using an inventory management simulation task to study how managers cope with specific problems of inventory control under different experimental conditions. It would be important, in such research, to establish that the experimental conditions as well as the simulation task had variables, relationships, content, features and dynamics that were representative of the ‘real world’ in which inventory managers must operate, if your intention was to generalise what is found using the simulation task to actual behaviours in organisational environments. In interpretivist/constructivist research, the contextualisation strategy of exploring boundaries may or may not be a viable or desirable strategy to follow. This is because the interpretivist/constructivist patterns of guiding assumptions tend to favour the contextual situatedness and potential uniqueness of every individual’s or group’s perspective rather than the achievement of understandings or theories that can be meaningfully transported across various types of boundaries. However, if your goal is to explore not only what is unique in particular contexts, but also what emerges in common across them, then it may make sense to invoke this contextualisation strategy and, in terms of paradigm-specific quality criteria, this will make transportability a necessary focus for your research. • Describing phenomena in context: Often, one intention of research is to provide a description of some process, state of affairs, events, perspectives, situations, groups or organisations in the context in which it is being observed. This may be especially relevant for research underpinned by an interpretivist/constructivist

310

•

•

•

•

10 How Should I Contextualise and Position My Study?

pattern of guiding assumptions. In such research, the description will tend to be qualitative, very rich, highly contextualised and subjective (i.e., anchored in the perspectives of the research participants). Such a description is often undertaken as a precursor to interpretation and theorising. We will see, in Chap. 11, that this contextualisation is consistent with the descriptive research frame. It is possible that certain types of positivist research can also be positioned as descriptive. In particular, cross-sectional questionnaires often have a descriptive intent. What is less well-understood is that such questionnaires should be regarded as embedded within specific contexts and those contexts will ultimately serve to delimit the boundaries and applicability of the description. Critiquing phenomena in context: In research guided by the critical social science, Indigenous or feminist patterns of guiding assumptions, the intent of the study may be to critique a state of affairs, situation, policy, group or organisation with respect to power relationships, constraints and impediments with a view toward overcoming them and thereby achieve improvements or a better relational balance. This contextualisation strategy has very explicit ties to other stakeholders who have a vested interest in the problem or decision being investigated, and those stakeholder interests will need to be addressed as an expected part of the strategy. Such stakeholders will often be those who currently do not enjoy power, favour or equity in a current situation (creating an intention that purists in critical research might call ‘emancipation of the oppressed’ classes or people). Evaluating phenomena in context: In certain types of applied research, a product, individuals, groups, organisations, interventions, programs or policies may be evaluated in terms of effectiveness, efficiency, acceptance, durability, compliance and the like. This contextualisation strategy is explicitly oriented toward gathering evidence to support evaluative judgements, perhaps with a view toward making concrete recommendations about adoption and implementation. Such research may be guided by positivist, interpretivist/constructivist or critical social science pattern of assumptions. Thus, arguments for this position must necessarily be derived from and be consistent with your chosen pattern of guiding assumptions. This contextualisation strategy is consistent with the evaluation research frame to be discussed in Chap. 11. Understanding perspectives in context: This contextualisation strategy is almost always associated with interpretivist/constructivist, critical social science, Indigenous or feminist patterns of guiding assumptions. The broadest view of context may be taken here, including cross-cultural or multi-cultural contexts. However, the central principle is that you cannot look for commonalities across contexts until you first understand meanings within each context. Thus, the stories sought here are highly situated and this contextualisation will lead the reader to expect to achieve a deep contextual understanding and perhaps even a contextualised theoretical account. This contextualisation strategy will often be associated with ‘describing phenomena in context’ contextualisation strategy. Exploring/filling a perceived gap in knowledge: Often, when you conduct a comprehensive and critical literature review, you will find a gap or an

10.2

•

•

•

•

What Kinds of Contextualisation Strategies Can I Employ …

311

unexplored avenue. This gives rise to a contextualisation strategy where you explicitly set out to conduct research to fill the gap or explore that avenue. Gaps or unexplored avenues may be theoretical, methodological, conceptual, contextual, paradigm-based or application-based. This is frequently one contextualisation strategy adopted in postgraduate research simply because it helps you to identify, argue for, and focus in on a potential research problem or question. Creating/validating an instrument or measuring process: This contextualisation strategy may be closely associated with ‘developing or testing a new theory’ or with 'replicating a test of theory in a new context'. It is almost always linked to the positivist pattern of guiding assumptions and is often a preliminary step in a larger study, rather than being an end in itself (which it certainly can be as well). Setting out this contextualisation strategy will lead a reader to expect a research investigation that has at least two sequential stages: instrument/measurement development and validation, followed by hypothesis and/or model testing. More rarely but still quite acceptable, the research may be concerned only with instrument/measurement development and validation. Solving a problem and/or informing a decision in a specific context: This is another explicitly applied research contextualisation strategy where your research is intended to provide solutions and/or recommendations in a specific context. It may emerge in conjunction with the ‘evaluating phenomena in context’ contextualisation strategy. This is a common contextualisation strategy in action research, for example, where your intention may be to formulate and perhaps even evaluate solutions and potential decision choices. This contextualisation strategy has very explicit ties to other stakeholders who have a vested interest in the problem or decision being investigated, and those stakeholder interests will need to be addressed as an expected part of the strategy. Improving a process or outcome: This contextualisation strategy starts from the premise that processes or outcomes need to be improved and that the research is intended to facilitate the achievement of that improvement as well as providing the necessary evidence to demonstrate that improvement has occurred. The focus could be on improving a change program or process, product or process quality, employee or managerial performance, training quality and the like. This strategy is often underpinned by the critical social science pattern of guiding assumptions and is associated with an action research approach. In many cases, this contextualisation strategy may be linked to an explicit expectation of participant involvement in the research process itself. If this is the case, then the participatory inquiry pattern of guiding assumptions may be relevant. Facilitating learning and reflection: Sometimes, you may conduct research in order to demonstrate how theory might be translated into action in a group or an organisation, i.e., a strong practice/practitioner focus. This suggests a contextualisation strategy directly linked to stimulating organisation learning and critical thinking. An expectation of this strategy would be for the research to provide concrete and constructive feedback to a group or organisation in order to facilitate learning and change. Another expectation could be for your research to facilitate the actual learning and reflection process, which would align this

312

10 How Should I Contextualise and Position My Study?

strategy very closely with the ‘improving a process’ or ‘outcome contextualisation’ strategies. This contextualisation strategy as well as the previous two are all consistent with the action research frame to be discussed in Chap. 11. • Combining/synthesising/hybridising positioning strategies: Various combinations of the above types of contextualisation are certainly possible in the context of a single, postgraduate research effort. In some cases, contextualisation strategies will be brought together by virtue of the configuration and conduct of your research. This will be especially if your research employs pluralist logic, the critical realist pattern of guiding assumptions, and/or applied problem-solving research frames such as action research. In other cases, combining or hybridising contextualisation strategies may occur in order to meet the needs and expectations of various stakeholders in the research, thereby allowing your research to potentially speak to multiple audiences. Note that the above list of strategies is not intended to be exhaustive of the possible ways to contextualise the intent of your research, but it does cover a range of potential strategies.

10.3

What Might Your Contextualisation Strategies Be Linked To?

Contextualisation strategies can be adopted from the outset or can emerge as your project evolves. Each strategy will have a specific point of focus that emerges in the context of your research which can often be conceptualised as specific queries that you should address as your research unfolds. To clarify, contextualisation could potentially occur, either explicitly or implicitly, with respect to: • The previous literature in the area: Where does your research sit in the context of previous research? What is your research contributing to the literature and/or to what is currently known (or unknown) in the area? • Previous events: Where does your study sit with respect to specific and relevant previous events (including historical events) in the research context? • Performance criteria, including regulations and policies: Where does your research sit in the context of ways of assessing and controlling organisations in the larger environmental context in which the organisation is situated? • Market/consumer/client needs: Where does your research sit in the context of the markets or clientele that organisations or institutions must relate to? • Other researchers: Where does your research sit with respect to other researchers (including your supervisor(s))? • Participants and other data sources: Where do your research participants and other data sources sit with respect to their relationship to you as well as to the research context more generally? Here, other data sources should be taken to

10.3

• • • •

•

• • • •

•

What Might Your Contextualisation Strategies Be Linked To?

313

refer to any non-human source of information, such as, databases, documents, media stories and audio and video recordings. Your career or educational progression: Will the research facilitate you realising important career or educational goals or desired outcomes? Disciplinary, multi- or cross-disciplinary needs and expectations: Are you committed to one discipline, or are you trying to work beyond the boundaries of a single discipline, using a pluralist or transdisciplinary logic? Your own connection to the research context: What is your particular relationship with the context of the research, and is this relationship changing or evolving? Expectations of yourself: How are you situated with respect to having required skills, gaining access, collecting and analysing data, writing up the results, managing deadlines and milestones and managing relationships with your supervisor(s) (things this book is explicitly designed to help you develop)? Expectations of your supervisor(s): How might your supervisor(s) benefit from your research (may be linked to the intention to publish)? Furthermore, what do your supervisor(s) expect in terms of the role they play in your research and your competencies to carry out that research? Other, perhaps competing, disciplinary, methodological, professional or theoretical perspectives or traditions: Is your research appropriately sensitive to, or have meaning for, other perspectives? Decision makers: How might your research be relevant to people who need to make specific decisions? Possible futures: How might your research be relevant to organisational investment, choices and strategies for the future? How might your research be relevant to further research in the area? Needs of a professional organisation: Does your research appropriately meet the expectations of a relevant professional body? [This may encompass, for example, ethical issues, how research outcomes are disseminated and how research outcomes might be relevant to professional practice.] Expectations of a specific journal or conference outlet: Does your research, as you present it, meet the expectations of an intended target journal or conference? Do you clearly understand these expectations and are you willing to conform to them (extends to how you respond to feedback from reviewers and editors)?

10.4

Researcher Positioning

From a systems perspective, an essential facet of the contextualisation of social and behavioural research is associated with you, as the researcher. It represents an explicit acknowledgement that you are human and that your humanity can influence what you believe in and do. It is important enough that one of the Contextualisation meta-criteria domains is devoted to evaluating it. Researcher positioning serves

314

10 How Should I Contextualise and Position My Study?

several important functions, not least of which is ensuring that you understand where you are situated with respect to the research you are conducting; this contributes to your ‘big picture’, i.e., the ‘forest and the trees’. Other relevant functions encompass intrapersonal (yourself), interpersonal (other people you relate to), professional (your professional identity, networks and practices) and contextual (your motivation, experience and roles) domains. Your research journal forms a critical and enduring platform for you to record your deliberations in each of these domains and set out any potential implications for your research. In the intrapersonal domain, researcher positioning requires critical self-awareness, in terms of knowing your preferences, capabilities and skills, capacities, interests, needs, goals and expectations. It establishes where you are coming from and what you are trying to achieve. It involves insights into the duality of and tensions between your roles of ‘you as a person’ and ‘you as researcher’. On the ‘you as a person’ side, considerations emerge that concern your basic personality and motivations (Type A vs. Type B personality, cognitive style, empathy, dogmatism, authoritarianism, curiosity, ability to work with others, motivation to persist; motivation to succeed, and so on); intellectual (e.g., in mathematics, reasoning, critical and systems thinking, logic, language, reading, writing, oral language skills) and creative abilities (e.g., intuitive thinking, visualisation, capacity for lateral thinking); family, social and life history (e.g., who and what your parents/ siblings/friends do/did; family support for study, development and professional activities); preferences and interests (e.g., likes and dislikes; burning questions; types of rewards); life goals (e.g., career aspirations, family, reputation); educational history (e.g., schools/university attended; majors and minors in university study; opportunities to pursue interests) and so on. These often provide a backdrop against which some aspects of the ‘you as a researcher’ might emerge (e.g., preferences for numbers/mathematics or language, interests in social issues, long-term career goals, past training in research methods at university). On the ‘you as a researcher’ side, considerations emerge such as your preferred research approaches and patterns of guiding assumptions (which may be shaped by university training, supervisors and early research experiences); opportunities of interest; relevance/ sufficiency of prior research training and skill sets and access to personal and organisational resources and technology to support your research efforts. A critical aspect of researcher positioning, on the ‘you as a researcher’ front, is coming to a clear understanding of and making conscious choices about the pattern(s) of guiding assumptions that would be most useful to adopt for your research project. You should be able to clearly unpack your reasoning about adopted guiding assumptions and to clearly convey that reasoning to the target audience(s) for the intended research outcomes. In the interpersonal domain, researcher positioning involves considerations of the relationships you have with significant others in your personal and professional lives and how one or more of these might impact on your research. Potential relationships to consider might include: your family (e.g., expectations for your work-life balance, time away from home); friends (e.g., sounding boards for problems in general; distraction/relief from stress); colleagues and peers (e.g.,

10.4

Researcher Positioning

315

sounding boards for research-related issues; access to professional networks; other members of a research team or collaborative research effort); supervisors and supervisory committees (e.g., for a postgraduate student, the people mostly closely related to and influential on your research); examiners (e.g., people who will evaluate your postgraduate research outcome, who may or may not be personally known to you); reviewers and journal editors (e.g., gatekeepers who pass judgment on the worthiness of a research outcome for publication, who may or may not be personally known to you); bosses and/or subordinates (e.g., people/organisation a researcher works for; people who work for you); people/gatekeepers who may control valued resources or who might have useful networks to tap into (e.g., people who work with funding organisations; gatekeepers who control access to potential data sources); people/groups/organisations who have an interest in and/or who stand to gain or lose by the research you are doing (e.g., key stakeholders, about which more will be said below); and so on. It is in this domain that you begin to unpack just who the most influential people/organisations are on your research, how those influences might or do play out and how you might respond or adapt to such influences. Ownership of your research and the exercise of power over the shape and conduct of that research are very important considerations in this domain. It is important to note that positioning with respect to the ‘you as a researcher’ may not be fully fleshed out prior to your research commencing. In fact, such positioning may continue to evolve over time as research events transpire, constraints are confronted, and opportunities are taken. In the professional domain, researcher positioning involves understanding what the expectations of relevant professions, governments and society hold for you as a researcher. Such positioning encompasses your obligations to behave ethically, often formalised in professional standards or codes for research conduct, and which may include institutionally- or nationally-mandated policies/guidelines for research conduct and may overlap ethical guidelines for the conduct of research with human participants (see discussion with respect to positioning of data sources below). Table 10.1 provides some links to a sample of codes of conduct and professional standards in Australia and other countries. These codes of professional conduct typically have sections that detail obligations to deal in good faith with all research participants as well as with other researchers involved in the project and with gatekeepers; deal in good faith with all data gathered, recorded, stored and analysed; obligations to avoid engaging in fraudulent research behaviour (e.g., falsifying data, plagiarism of others’ work, undisclosed deception of participants, undisclosed use of information about people gathered for purposes other than your research project or the use of any such information without the consent of the persons concerned) and to avoid subjecting participants to harm, either during the research process itself or downstream as a consequence of their research participation. Note that the distinction between a professional code of conduct and ethical guidelines is often blurred. This is because the difference is really one of perspective: a code of conduct tends to focus on your obligations/behaviours as researcher; ethical guidelines tend to focus on research participants and their rights and protections (obviously, though, it is you, as the researcher, who is obliged to

316

10 How Should I Contextualise and Position My Study?

ensure that those rights and protections are always safeguarded). Thus, where professional standards/codes of conduct and ethical guidelines overlap lies largely in the relationship between researcher and research participants. Other aspects of the professional domain in researcher positioning encompass the relationships you have with various professional organisations and networks. The term ‘professional’ here is intended to encompass all aspects of your professional identity, be it research-based, practitioner-based, service-based, academic and/or industry-based). Professional networks may be physical, as with peers and colleagues gathered at a conference or virtual with colleagues connected via a web-based service such as LinkedIn (https://www.linkedin.com/), Kudos (https:// www.growkudos.com/about) or ResearchGate (http://researchgate.net/). In either case, shared interests and identities are what bring a professional network together and such relationships can facilitate and support a professional’s research-oriented activities and may even lead to the emergence of a research team. Professional and academic societies and academies (e.g., the U.S. Academy of Management (AOM), European Academy of Management (EURAM), American Nurses Association (ANA), Australian and New Zealand Marketing Academy (ANZMAC), American Psychological Association (APA), Australian and New Zealand Academy of Management (ANZAM), European Group for Organizational Studies (EGOS), Cultural Studies Association of Australasia (CSAA), Australian Association for Research in Education (AARE), American Educational Research Association (AERA), Social Research Association (SRA) in the U.K.) are also sources of professional relationships and/or accreditation for a practitioner/researcher and may help provide a professional identity for you (including, in some cases, offering a code of conduct for members). Many such organisations run annual conferences for their members where professional networks and special interest groups can be created, explored and extended and where knowledge-sharing is explicitly and regularly encouraged. Finally, you have professional relationships with your employing organisation and co-workers. Any of these relationships can potentially influence the shape of a specific research project you undertake, perhaps through access to expertise, access to networks that can link you to desired data sources, support for data analysis, collaboration or access to outlets for research outcomes. The contextual domain of researcher positioning focuses on the intersection between you as a person and the research activities you are undertaking. Your motivation for undertaking research, for example, is relevant here. You may be motivated to conduct research because you are pursuing a qualification or degree that requires it, because you have a professional or personal interest in or curiosity about a specific issue, problem and/or innovation, because research is a formal expectation of your job role (as it is for university academics seeking publication, market researchers) and/or because you are honouring a commitment or obligation you have made (such as through the receipt of research grant funding, a research consultancy for some organisation or industry or a research team or collaboration you are part of). The level of experience you have in conducting research in specific contexts is also relevant here. If you are an apprentice or novice researcher (e.g., postgraduate student), then this may influence the entire landscape of your research

10.4

Researcher Positioning

317

Table 10.1 Descriptions and links to a sample of professional standards/codes for research conduct available in different countries Country

Originating institution

Description

Website link

Australia

Code of ethics

https://www.psychology.org. au/about/ethics/ http://www.amsrs.com.au/ professional-standards/amsrscode-of-professional-behaviour https://www.nhmrc.gov.au/ guidelines-publications/r39

New Zealand

Australian Psychological Society (APS) Australian Market and Social Research Society (AMSRS) Australian Government/ National Health and Medical Research Council (NHMRC) Royal Society of New Zealand

United Kingdom United Kingdom

Social Research Association (SRA) British Psychological Society (BPS)

Canada

United States

Marketing Research and Intelligence Association (MRIA) American Psychological Association (APA)

United States

Academy of Management (AOM)

United States European Union

American Sociological Association (ASA) European Science Foundation (ESF)

European Union

European Commission

Ethics for researchers

European Union

World Medical Association (WMA)

European Union

European Society for Opinion and Marketing Research (ESOMAR)

Declaration of Helsinki—ethical principles Codes and guidelines

Australia

Australia

Professional standards Code for responsible conduct of research Code of professional standards and ethics Ethical Guidelines Ethics and standards Code of conduct for members Ethical principles of psychologists and code of conduct Code of ethics for members Code of ethics Code of conduct for research integrity

http://www.royalsociety.org. nz/organisation/about/code/

http://the-sra.org.uk/wpcontent/uploads/ethics03.pdf http://www.bps.org.uk/whatwe-do/ethics-standards/ethicsstandards http://mria-arim.ca/about-mria/ standards/code-of-conduct-formembers http://www.apa.org/ethics/ code/

https://aom.org/uploadedFiles/ About_AOM/Governance/ AOM_Code_of_Ethics.pdf http://www.asanet.org/ membership/code-ethics http://www.esf.org/fileadmin/ Public_documents/ Publications/Code_Conduct_ ResearchIntegrity.pdf http://ec.europa.eu/research/ participants/data/ref/fp7/89888/ ethics-for-researchers_en.pdf http://www.wma.net/en/ 30publications/10policies/b3/ https://www.esomar.org/ knowledge-and-standards/ codes-and-guidelines.php

318

10 How Should I Contextualise and Position My Study?

endeavour relative to when you have experience doing research and/or being a practitioner in specific contexts (such as schools, public service or private organisations). This landscape may identify additional key stakeholders for your research (e.g., supervisors, bosses, customers, clients) and it may influence the role you undertake in the research context (e.g., educational researcher who is also a teacher; policing researcher who is also a sworn or unsworn police officer; an Indigenous or non-Indigenous researcher doing research in a specific Indigenous context). This latter consideration regarding the role of the researcher can emerge as critically important for some research frames as we will see in Chap. 11. If you are conducting research within your own organisation and/or cultural context, we say that you are an ‘insider’. This implies that you possess more detailed knowledge about the research context and should be able to leverage that insider knowledge in ways that will benefit your research project as well as identifying personal relationships that will require special attention to manage through the research project (e.g., interviewing co-workers or supervisors or subordinates). If you are not conducting research within your own organisation and/or cultural context, we say that you are an ‘outsider’. This implies that you will have a greater sense of detachment (approximating objectivity) with respect to the research context but, at the same time, will have to work harder to understand that context in ways so as to approximate what an ‘insider’ would already know. There are advantages and disadvantages associated with both the ‘insider’ and ‘outsider’ research roles. For the ‘insider’ role, your more intimate knowledge of the context may provide insights into who valuable data sources might be and how access to them might be negotiated and how to contextualise data that you gather (i.e., contextual sensitivity advantages are conferred). However, the ‘insider’ role may be accompanied by participant suspicions about what agenda you are really running (e.g., ‘are you simply doing research for management to use against us?’). Furthermore, there are risks that the power relationships you are involved in (e.g., between supervisor and subordinates) could potentially be abused or misused (if you are the supervisor) or could deny you access to certain types of knowledge (if you are a subordinate). The ‘outsider’ role comes with the potential benefits of heightened objectivity and perceived neutrality (e.g., perceived to be less likely to be running someone else’s agenda). However, drawbacks include lesser capacity to appropriately contextualise what you have learned in the research because of the lack of insider knowledge (i.e., you must work harder to achieve contextual sensitivity) and tougher negotiations to pass gatekeepers for access to desired data sources. It might also mean that your research may need to be reconfigured to try and gain some relevant insider knowledge. If you are conducting research as part of a consultancy arrangement with the management of an organisation, things get a bit more complicated in that you would still be considered an ‘outsider’, but not necessarily as someone who would adopt a neutral stance (which could make gaining access to desired sources of data even harder). You might ask, ‘what about experimental research in a laboratory’? Technically, here, your researcher role would be as an ‘outsider’ with respect to the contexts that any of your research participants normally inhabit but as an ‘insider’ with respect to the experimental

10.4

Researcher Positioning

319

tasks, measurement instruments and conditions you create and manipulate in order to provide experiences for those participants. Brunswik’s (1956) concept of ‘representative design’ outlines a process whereby you can build a meaningful bridge between the ‘insider’ and ‘outsider’ perspectives of a laboratory researcher through designing experiences that have meaning for, are relevant to or analogues/ simulations of experiences that research participants would have outside of the laboratory in their everyday lives. This then enhances your prospects of making convincing generalisations beyond the boundaries of the laboratory experience(s). Communicating relevant aspects of researcher positioning is often a weak point in many research outcomes, especially published research. Interestingly, different patterns of guiding assumptions take rather different stances with respect to how transparently a researcher exposes their positioning. For interpretivist/constructivist, critical social science, Indigenous and feminist patterns of guiding assumptions, transparency in researcher positioning as it impacts on your role in the research context, gaining access to and gathering data from data sources, analysing those data and forming interpretations and conclusions is critical to demonstrate. In contrast, positivist guiding assumptions downplay the need for you to expose/ explore your positioning, instead depending, often uncritically, upon the presumption of objectivity to ‘negate’ any arguments about influences arising from researcher positioning. However, irrespective of one’s adopted pattern of guiding assumptions, it is the case that absence of or thinness in critical details about researcher positioning may have an adverse impact on how convincing your research can be. For example, not making clear that you are an employee of an organisation in which you are doing your research risks raising questions about conflicts between your role as a researcher and your role as an employee. Such conflicts might influence what data sources or information you have access to, what physical or financial resources you can call upon and could create problems with data quality arising from power differentials between you and participants (especially if you are in a managerial role). As another example, having a vested interest in a theory you have created and tested or that you strongly believe in can lead to an imbalance in effort to critically discount or dismiss contrary evidence while simultaneously reinforcing supporting evidence, even to the point of downplaying flaws in your own research (more effort expended) vis-à-vis modifying your theory to accommodate the contrary evidence (less effort expended). Decision researchers refer to this imbalance as the ‘confirmation bias’ (see, for example, Kaptchuk, 2003), where people tend to focus on and seek out evidence that confirms rather than disputes one’s perspective; a commonly revealed bias that does not reflect an objective stance. Active avoidance of this problem requires transparent acknowledgement of the risks associated with dogged adherence to your own perspective. Interpretivist/constructivist researchers refer to this active avoidance process as ‘managing or bracketing your preconceptions’. Positivist researchers use control over experimenter bias (e.g., single- or double-blind studies, see, for example, Kaptchuk, 2001) and other procedures designed to reinforce objectivity to try to combat this problem. The greater the awareness of your own researcher positioning, the greater the likelihood you will actively work to avoid such biases.

320

10 How Should I Contextualise and Position My Study?

Another important benefit of awareness of your own positioning is that key areas requiring further development may be identified. Such insights can add learning value to your research which can circumvent potential downstream problems that may emerge. For example, skill deficits in handling certain types of data or data analytic procedures/software support systems may surface and awareness of this can motivate you to seek developmental training or to cultivate a source of expert advice (a human resource). Inexperience with a specific data gathering strategy may motivate you to plan a trialing phase in your research where you obtain practice in implementing the strategy. Awareness of your lack of access to certain necessary resources, including desired data sources, may motivate you to seek out alternative resources or help you to make a conscious decision to modify your research goals so as to be achievable without the necessity for those resources (one of many trade-off decisions you may have to make during your research journey).

10.5

Positioning with Other’s Research

It is essential that social and behavioural researchers appropriately acknowledge and build upon what previous researchers have already learned or not learned. Thus, you need to contextualise and position your own emerging work with other work in relevant discipline(s), field(s), context(s) and samples. This is so essential that we devote the Contextualisation/Juxtapositioning with Other Research meta-criterion to its evaluation. What we are talking about here is exploring the knowledge assembled and disseminated by others and using that knowledge to build a rationale for your own research. Positioning your research with other’s research involves trying to establish where your research fits into the landscape of what has come before. Such positioning might involve, amongst other things, building or testing a theory, creating knowledge to fill a perceived gap, doing something different so as to overcome limitations you see in previous research, seeing how previous knowledge applies in a new context, working to provide evidence for practical decision making or change, or trying out a new methodological approach. Positioning with other’s research requires you to demonstrate an understanding of relevant research that has been previously conducted and then logically argue for how your research will fit into or extend that landscape. Positioning arguments might adopt a: • cumulative stance—adding value to or taking the next step in contributing to an existing body of knowledge; • a critical stance—correcting, redirecting or circumventing previous ambiguities or deficiencies; • an application stance—showing how research may translate into action or utility, • an innovation stance—seeking to construct or do something new and different that can be used by others,

10.5

Positioning with Other’s Research

321

• an evaluative stance—seeking to provide evidence to inform judgments of worth or value, • a greenfield stance—pushing out into new frontiers that have not been well-researched; and/or • an extension stance—doing research in new contexts to see if existing knowledge translates or has meaning. Positioning with other’s research has a bi-directional dynamic relationship with choice of research frame (to be discussed in Chap. 11) where positioning may influence choice of frame and choice of frame may influence positioning. Carrying out and presenting a coherent, well-organised and critically executed literature review forms the essential foundation for positioning with other’s research. While we will provide a detailed discussion of literature reviewing in Chap. 13, it is important to make some salient points with respect to its role in research contextualisation and positioning. Reviewing literature helps you to build a logical basis/argument for the research you want to do or are reporting on. All the literature you review should be harnessed in support of this argument. Accordingly, unless the intent of your research is to do a systematic, perhaps historical, review of the literature in a particular area, your review of literature should generally not be exhaustive. Instead, it should be focused and pointed covering the essential aspects of the problem you are trying to shape/formulate. The goal is to lead you, and when the literature review is written up, the research reader/user, toward your emerging research problem and its associated research questions/hypotheses. It is easy to get lost in a review of literature as the volume of information that could be pursued and drawn upon is generally massive. However, it may be useful to view the review of literature as a funnel that starts broad at the beginning and progressively narrows so that, at the end, the research questions/hypotheses that emerge are entirely expected and provide a firm and logical focus for you to follow. It is important to realise that you can use a literature review to serve several useful purposes: • it can provide you with contextual insights into what other authors have done and why, including some insight into their various contextualisations, positionings and research frames; • it can help you to achieve new learning insights into essential concepts, theories and patterns of relationships; • it can provide you with methodological insights into how other researchers accomplished their research goals, including the patterns of guiding assumptions they have adopted in pursuit of those goals; • it can provide you with practical/professional insights showing how learning might have meaning or be useful outside the specific research contexts; and • it can help you to develop new insights into future directions for research by suggesting new pathways to follow or avoid. In order to realise these various purposes, your review of literature must be critical rather than simply descriptive. You should be aware that there is an

322

10 How Should I Contextualise and Position My Study?

approximate hierarchy of credibility in terms of the convincingness reflected in research outcomes from various types of sources. The hierarchy, generally ordered from most to least convincing, is as follows. • Peer-reviewed journal articles. Typically, the most convincing literature is peer-reviewed research published in academic journals. These outcomes constitute primary sources of research information that have been formally vetted by expert research peers and judged for their quality and contribution value. However, even within this category, there is a rough hierarchy of credibility, where some journals have a reputation of publishing what is perceived to be very high quality research (the so-called A* and A journals in many journal ranking systems, see, for example, http://www.harzing.com/resources/journalquality-list for journals ranked in the Economics, Finance, Accounting, Management, and Marketing disciplines) whereas others have poorer reputations. Journal ranking systems attempt to rank published research outlets with respect to some combination of citation rates, impact factors and expert perceptions. However, it is important to realise that these ranking systems are not infallible and that different systems may provide different rankings of the same journal. Online and open access journals as well as journals where authors pay a submission and publication fee to have their research published, have been generally perceived as publishing research of lesser quality, although this perception is evolving as publishing systems and the vetting of submitted content become more sophisticated and transparent. An emerging problem in the world of online research publishing is the growth in the number of so-called predatory publishers and journals, where making money without regard to the quality of what is published dominates (see, for example, https://www.enago.com/ academy/avoid-paying-predatory-publishers-journals-for-publication/). You should also be aware that even articles published in peer reviewed journals may not be all that convincing as publication practices are very diverse as well as being highly selective about what is considered publishable in the context of the journal. You need to be aware that many journals are not paradigm-blind, preferring to publish, or with a pattern of publishing, research predominantly produced under specific patterns of guiding assumptions (e.g., positivist quantitative research only or interpretivist qualitative research, e.g., specific marketing, psychological, sociological, medical/nursing and educational journals). This trend may reflect editorial policies and preferences as well as larger discipline preferences/expectations. Other journals encourage a more diverse range of submissions. In either case, using the meta-criteria to evaluate what you read can be a valuable practice, placing different contributions on a more level playing field. • Research books. Following closely behind peer-reviewed journal articles in convincingness are research books. A research book may constitute a primary source of information if it focuses on synthesising research done by a specific researcher or team of researchers (see, for example, Beach, 1998; Gigerenzer, Todd, & The ABC Research Group, 1999) or a secondary source of information

10.5

Positioning with Other’s Research

323

if it brings together research from a range of researchers, perhaps even a range of disciplines (see, for example, Cooksey, 1996; Hammond & Stewart, 2001; Hattie, 2009). Research books will have typically been professionally vetted as a holistic concept in terms of the value they would add to the field. Edited research books are generally produced by well-respected researchers in a particular discipline and often have their individual chapter contributions from different authors specifically invited and peer-reviewed (e.g., Allen, Maguire & McKelvey, 2011; Schuelka & Maxwell, 2106; Sinclair, 2014; Tashakkori & Teddlie, 2010). An edited research book may also be a systematically assembled compendium of previously published research in an area (e.g., Lichtenstein & Slovic, 2006). The value of research books is that they may help you to organise your thinking about specific research areas and may also provide theoretical, methodological and practical guidance. Certain publishers (for example, Sage, Oxford University Press, Cambridge University Press, John Wiley & Sons, Edward Elgar) routinely engage internationally-known researchers in a specific field to assemble a handbook in a specific research area and the word ‘handbook’ will generally feature in the book title. • Grey literature. Next most convincing are primary sources of research information generally referred to as grey literature. “Grey literature can include government reports, committee reports, academic papers, theses, bibliographies, conference papers and abstracts, discussion papers, newsletters, PowerPoint presentations, conference proceedings, program evaluation reports, standards/ best practice documents, technical specifications and standards, and working papers” (Alberani et al., 1990, cited in Benzies, Premji, Hayden, & Serrett, 2006). However, they are much less likely to have been peer-reviewed for quality and contribution. Some of these, such as working papers, conference papers and Powerpoint presentations, might be interim research outcomes from an on-going research project at a university or research centre. Postgraduate theses, dissertations or portfolios will have typically been vetted by examiners or a supervising committee, but this vetting is from the point of view of judging the convincingness of the outcome with respect to achieving some credential, such as a PhD or professional doctorate, rather than suitability for publication. Other types of grey literature documents and reports may include: – company annual reports (for example, see https://www.microsoft.com/ investor/reports/ar15/index.html); – statistical/financial/economic/educational databases (e.g., https://www.imf. org/external/pubs/ft/weo/2016/01/weodata/index.aspx; – information produced by the Australian Bureau of Statistics, see http://www. abs.gov.au/); – periodic and final reports to granting/funding bodies (e.g., https://www.mq. edu.au/__data/assets/pdf_file/0009/68994/DP0558372FinalReport.pdf); – reports submitted to meet the expectations of government regulations (such as equity and access reports for the education sector in Australia; for example, see the reports available at https://www.education.gov.au/accessand-participation); and

324

10 How Should I Contextualise and Position My Study?

– position and consultation papers (e.g., see examples at https://www. universitiesaustralia.edu.au/news/submissions-and-reports#.V7po15h97id). Note that grey literature may be produced with political, social or other goals (e.g., meeting educational requirements, providing data for other researchers, providing policy advice, enhancing public image, providing information for potential investors) in mind and you must be careful to identify and understand what these other agendas might be and how they might have influenced the content and quality of the outcome as part of their critical evaluation. What this means is that you need to contextualise such research yourself. • Academic textbooks. Next in order of convincingness are academic textbooks. These books constitute secondary sources of research information, reporting on and interpreting research from primary sources, primarily for teaching purposes. These secondary sources may look attractive in terms of bringing a lot of research literature together in a straightforward manner. However, what must be realised is that such textbooks organise their understanding of selected research literature in specific ways to enhance their pedagogical goals and those goals may not intersect a research quality or contribution argument. Note that you should avoid over-relying on secondary sources, such as textbooks, as these sources have already filtered and interpreted what was reported in a primary source and such filtering and interpretation may or may not be in accord with your research goals and what you want to learn. • Trade/professional/practice-oriented publications. Next in order of convincingness, at least from an academic research perspective, are trade or professional publications. These also constitute secondary sources of research information, often mixing information from primary sources with anecdotal information. In some cases, the authors attempt to build a useful, potentially marketable, conceptual framework or set of tools that can be used by managers or other professionals to produce change or some other valued result (for example, Hammond, 1996; Senge, 1994). In other cases, trade and professional publications may report on the state of play with respect to some issue of concern to the trade or profession, perhaps with the goal of highlighting an important dilemma or policy issue (see, for example, magazines like Human Capital, the Professional Educator, Professional Nursing Today, Psychology Today, The Economist). These types of publications focus on industry/professional practices, often with the intent of influencing those practices. Such literature can be quite useful to professional doctorate students as ways of learning about communities of practice and the meta-criteria can help you to evaluate their quality. • Media reports and internet materials. Generally, the least convincing areas of literature, for research purposes, include media reports and internet materials. However, within this category of literature, these sources can themselves vary from more credible, such as Wikipedia and some newspaper/magazine/media stories, academically-oriented multimedia feeds (such as Ted talks: see https:// www.ted.com/ and online newsfeeds) to less credible (such as blogs, Facebook/ Twitter postings, popular magazine articles, YouTube feeds and manifestos),

10.5

Positioning with Other’s Research

325

which are considered to be more in the domain of popular literature or as expressions of public opinion. Such sources of literature may help you to contextualise/position some aspects of your research but are generally far too anecdotal and opinionated to be considered as providing credible research evidence. It is important that you read the literature with the intent to learn and critically evaluate. You don’t just review literature for its own sake; your goal should always be to address the question ‘what can I take from this {item of literature} that will help me to better contextualise and position my own research in some useful way?’ Equally, each item of literature must not just be read and then later simply summarised, it must be critically assessed with respect to its convincingness (i.e., identifying positive points which add to convincingness as well as negative points which detract from convincingness) in order to add value to the development arguments you are making for your own research investigation (the meta-criteria as well as paradigm-specific quality criteria will be of great help in such critical evaluation). When reading literature, you should aim to learn from a number of different angles and each angle may help you to establish where your research is situated with respect to what has been done before. Thus, read critically and focus on: • Content and context. Understand what authors did and judge whether or not they did it well (convincingness judgments). What intentions, contexts and positionings were evidenced in their article? Who were they speaking to in their writing? What were they trying to learn and why? • Theories. Examine any theoretical positions and propositions the authors explored and perhaps served as the focus for their research. • Methodologies. How did the researchers obtain the evidence they used to draw their conclusions and set out implications? This should include understanding the research frame and pattern(s) of guiding assumptions that the authors adopted/reflected. Try to glean what things seemed to work well and what things did not. Are there practices/strategies you can take on board that might assist in your own research approach? • Applications/practices/innovations. Did the authors develop, implement or learn anything new and novel, relative to what had been done before? Did they signal or evaluate practical applications of what they learned or show how what others had previously developed or learned worked in a new context? Did the authors signal potential for any innovation stemming from their research? • Leverage value. How can you best leverage what has been claimed, shown or learned from the authors to contextualise and position your own research? Here, you should look for things like convergences and/or divergences between what the authors have learned and what others have learned, gaps or unanswered questions that remain after reading what the authors have produced, blind alleys, limitations or obstacles the authors encountered during their journey and any directions for further research signalled by the authors (value for learning,

326

10 How Should I Contextualise and Position My Study?

fertilisation of ideas, handling of unexpected outcomes and acknowledgement of limitations are highly relevant meta-criteria for gauging leverage value). • Springboard potential. You can also use what you read as a springboard to other relevant literature. Here, you can look at how the authors themselves have positioned their own research with the research of others and search their reference list for possible new sources of literature to investigate. Back-referencing is a useful way to use secondary source of information, such as an academic textbook, to identify particular primary sources to pursue and will be discussed more fully in Chap. 13. Finding one or more meta-analyses relevant to your research area can also provide springboard potential for your literature review. One useful benefit of a meta-analysis is that its goal is to analyse patterns of findings in a sample of research outcomes, thus assisting in the critical review of that body of literature. However, you should realise that a meta-analysis is only effective for summarising quantitative research guided by the positivist pattern of guiding assumptions. Thus, meta-analysis is not paradigm-blind, it is paradigm-dependent. For qualitative research, approaches such as meta-ethnography (e.g., Campbell et al., 2012; Doyle, 2003) or meta-synthesis (e.g., Walsh & Downe, 2005) attempt to bring together/synthesise the stories that emerge from qualitative research and finding one of these articles could be useful to draw upon. Doing a literature review for research purposes can be characterised using the metaphor of a funnel. The search is generally very wide ranging when you start (at the top of the funnel), where you are just scratching the surface of what is important to read, and it is easy to get distracted/diverted by tangential ideas. As you progress, the search/review becomes much narrower and more focused where you dig deeper into specific aspects of the most relevant literature that provide specific foundations for your research (at the bottom of the funnel). The key is to appropriately manage the funnelling process to add value, logic and coherence to your arguments, always with the goal being to identify and work through the literature that is most relevant for developing the positioning logic for your own research. You should not prematurely focus your review so tightly that you neglect to consider possible learning and approaches taken to your subject matter/issue from other disciplines—this is one value of taking a multidisciplinary approach to research and such pluralist logic can be applied to a review of literature as well. You should use your research journal to record all summaries, thoughts, critiques, quotes and ideas that emerge as you read the literature. In your literature review, it is important not to ignore grey (unpublished; not widely available; not generally peer-reviewed) or professional/practice-based/trade literature or the popular media as these can provide contextual as well as theoretical and/or methodological input into your research arguments. However, you must take what they say/argue for in context as they will vary greatly in credibility and convincingness. Generally, but not always, literature that has undergone formal peer-review is more convincing, but even the best peer-reviewed journals do not publish perfect papers. Remember that, in the end, convincingness is a critical

10.5

Positioning with Other’s Research

327

cognitive judgment made by the research user (in this case, you, as the reader), who will be reading/learning from/applying that research for specific purposes. It is also possible to use secondary data to support your research/conceptual framework development, help contextualise your research and for juxtapositioning your research against the work of others. [Of course, it is also possible to use one or more secondary databases as the main data sources for your research (e.g., DataStream and other financial data sources, such as the databases maintained by Global Financial Data and the International Monetary Fund’s World Economic Outlook database); company/government-maintained secondary data sources (e.g., various census-level databases maintained by the Australian Bureau of Statistics, various tertiary-education related databases maintained by Universities Australia, labour-related databases maintained by the International Labor Organization). We discuss this possibility further in the Chap. 14.]

10.6

Positioning of Participants and Other Data Sources

When planning for and carrying out your research, one critical yet often neglected facet of contextualisation concerns understanding the positioning of intended participants and other data sources. This is so critical that it is tied directly to another Contextualisation meta-criteria. We differentiate here between human participants and non-human (yet, in most cases, human-produced handiworks) data sources (e.g., webpages, blogs, YouTube videos, documents, films, newspapers, images, social media postings, cartoons, artefacts, secondary database, music), but positioning is important to understand in either case.

10.6.1 Positioning of Human Participants Positioning with respect to human participants encompasses considerations associated with the fact that they have their own lives and aspects of those lives may intersect your research journey in a variety of important ways, including how they are to be treated. In short, positioning of participants entails understanding where they come from, what they expect and what their participation might mean to them. Research participants may be relatively passive (as in research conducted under the positivist pattern of guiding assumptions, where they are still referred to by the distancing label ‘subjects’ in some quarters) or active, even participatory or collaborative (as in research conducted under interpretivist/constructivist, critical realist, critical social science, Indigenous or participatory inquiry patterns of guiding assumptions). However, irrespective of the influence of guiding assumptions, understanding the positioning of participants is critical because it may influence your sampling choices, access to and quality of information gathered, how participants expect to be treated, the character of interpretations and conclusions

328

10 How Should I Contextualise and Position My Study?

and the extensional reach of those interpretations and conclusions beyond your research context. Positioning of participants addresses questions such as: • Who is participating, from where and why are they relevant to your research? This encompasses considerations such as cultural and ethnic background, work and life experiences relevant to the research or lack thereof, as well as other relevant demographic characteristics. For example, in positivist research, this helps you to define the population of interest for sampling and subsequent generalisation inferences. If you are guided by non-positivist assumptions, answering this question helps you decide who you wish to connect with for data gathering in your research and what they will or can contribute to your research. • What role(s) do participants currently play and what are their prior experiences/roles within the research context? This may help to identify majority, minority, marginalised and/or silenced voices within that context that you may wish to connect with. Different roles in the research context (e.g., head teacher, CEO, manager, line worker, new employee, union official, shopper or consumer, community member and so on) may have implications for what information you gain access to and can potentially influence how well-rounded or representative a story you will be able to tell. Participants who are new to a research context would likely have differing perspectives from those with longer associations and a richer set of experiences with the context, so experience does matter. • What are participants’ relationships to you? It is possible that there is no relationship between you, as researcher, and participants (as is quite often the case in positivist research). However, in naturalistic or field research, you may share a personal, professional, organisational (e.g., manager/subordinate) or collegial relationship with at least some participants. In this case, there are potential power dynamics between yourself and participants that might need to be acknowledged and managed (as when you do research in your own organisation). • What are the ethical expectations regarding how participants and any information/knowledge/insights they provide are to be treated/protected/ safeguarded? This highlights the power of ethical constraints over your research activities by creating obligations that, in dealing with human participants, you cannot ignore. This also acknowledges that participants are themselves important stakeholders in your research. We will see that there are different sets of expectations depending upon whether participants are from Indigenous or non-Indigenous backgrounds and contexts. • Who controls access to participants and what do they expect in return for granting access? Here, the role of gatekeepers that you must negotiate with to gain access to participants emerges as an important consideration. It also implies that those gatekeepers are key stakeholders in the research. In certain industries (primary and secondary schools or other public sector organisations, for example), there may be several layers of gatekeepers that you must work your way through before gaining access to desired participants (e.g., state

10.6

Positioning of Participants and Other Data Sources

329

government department, school management, teachers and parents in the case of research involving students in public schools as research participants). Even then, at the last stage, ethical principles demand that every participant is their own gatekeeper in that, even if all other gatekeeper layers are passed, the participant has final say with respect to whether they will participate and, even if they do agree to participate, retain the right to end their participation at any time. It is important to understand that the term ‘participants’ as we are using it here refers both to potential participants (those you wish to approach and connect with) as well as to participants who have already given their permission (you have an ethical obligation to seek this) to participate or are already involved in participating. The issue of ethical expectations with respect to research participants is so important that while we provide a helicopter overview here, it is worth a more detailed separate discussion (see Chap. 15 of this book; also check out Bryman & Bell, 2015, Chap. 6; Cohen, Manion, & Morrison, 2011, Chap. 5; Mauthner, Jessop, Miller, & Birch, 2012). These expectations relate closely to codes of conduct for researchers but focus on the participant’s rights. Ethical expectations evolve over time are often codified in ethical guidelines. Ethical guidelines may be produced by government bodies, professional associations, funding bodies or special interest groups. Table 10.2 lists a sampling of ethical guidelines from different institutions in different countries along with a relevant website link. Those guidelines listed in the upper unshaded area of the table cover human participants in general and typically make statements about the following rights. • Participants have the right to have their informed consent sought and obtained for research participation. This means that potential participants must be fully informed as to what they can expect to happen to them during their participation, potential benefits they might realise, any risks they might face and what each of their rights are as participants. Participants would generally be asked to sign a statement affirming that they understand this information when they agree to participate. Researcher contact details must be provided so that the participant may ask any questions that arise. Any promises made must be kept. Research based on observable recordings of individuals obtained for other purposes (such as using camera recordings of shoppers in a store, done to deter shoplifting, to study consumer behaviour in context) or covert observations (as when a researcher goes undercover to study a social group like a gang) is ethically problematic because those shoppers/gang members have not given their informed consent to participate in that research. For certain types of research (psychological or sociological research, for example, conducted especially under the positivist pattern of guiding assumptions), the right to informed consent may conflict with the need for you not to signal what you are looking for (because that knowledge risks changing the very behaviours you are studying, something psychologists call demand characteristics). Historically, a researcher’s right to information/data superseded a participant’s right to informed consent, but nowadays, the participant’s right to informed consent trumps the researcher’s

330

10 How Should I Contextualise and Position My Study?

Table 10.2 Descriptions and links to a sample of ethical guidelines for research available in different countries

10.6

Positioning of Participants and Other Data Sources

331

right to control what participants know about the research. The upshot is that you need to plan additional steps to offset the possibility that the knowledge conveyed by obtaining informed consent influences the data being gathered (an example of a research journey trade-off being imposed on you by societal concerns). • Participants have the right to expect that their privacy and identity will be protected by you and that any information they might provide will be appropriately safeguarded from access by others. This means that participants have the right to privacy with respect to all aspects of their participation. Their identity must not be linked in any way to any information they might provide (anonymity). Assurance of confidentiality may not always be sufficient, because it begs the question ‘confidential to whom?’. You are generally obliged to keep any information provided by participants in a secured location, usually for a

332

10 How Should I Contextualise and Position My Study?

specified period of time. If you are conducting longitudinal research over time and need to track data gathered on separate occasions from each participant over that time period, this needs to be signalled during the informed consent process and each participant’s identity and contact details (for tracking purposes) must be kept separate from their actual data records through the use of an anonymising identifier, such as an identification number or pseudonym. • Participants have the right to be protected from exposure to unnecessary risks, harm and/or deception, but if such risks are incurred you have the obligation to rectify any adverse consequences. Participants should expect not to be deceived (i.e., not be lied to) with respect to what will happen to them during their participation and what you are actually interested in. Participants should also expect not to be placed at risk of harm or embarrassment nor should they be harmed in any way (physically, socially or psychologically) during the course of their participation. For certain types of research where deception is absolutely necessary in order to study the phenomena of interest (this includes saying you are investigating one kind of behaviour when you are really studying some other kind of behaviour in psychological experiments; using placebo (i.e., fake) drugs in medical drug trials), you must take all reasonable steps to ameliorate any damage done/disadvantages created via the deception. Use of deception generally means that true informed consent is difficult to obtain as many participants would not agree to be deceived. Researchers conducting medical drug trials often get around this constraint by signalling to potential participants that there is a chance they may receive a placebo treatment as part of their participation and that, if they do, they will be given effective treatment after the trial is concluded. • Participants have the right withdraw their participation in the research at any time without consequence. Once a participant has given their informed consent to participate, this in no way commits them to go the distance with their participation. They retain to right to withdraw at any time for any reason. What would concern you most about this is where their withdrawal is due to something that happens to the participant during the course of your research (recall the previous right regarding exposure to unnecessary risks and harm). The right to withdraw can create an unfortunate but unavoidable feedback effect on your research in that withdrawal of a participant means loss of access to any data they might provide (as well as negating the use of any data they have already provided), which effectively reduces or alters your sample. We can see from the above discussion of participant’s ethical rights that these rights may conflict with your goals and intentions. Such is the life of the modern-day social/behavioural scientist—ethical rights protecting participants supersede your rights to good data. Ethical dilemmas surface all the time in social and behavioural research and these dilemmas need to be resolved within the localised context of your research. However, consistency in expectations is important, especially in western democratic societies and this has given rise to sets of ethical guidelines for researchers that apply in different countries, as

10.6

Positioning of Participants and Other Data Sources

333

demonstrated in Table 10.2. In many cases, ethical guidelines are adopted by research organisations (including universities and funding bodies, such as the Australian Research Council, the National Science Foundation in the U.S. and the Social Sciences and Humanities Research Council in Canada) and compliance is often overseen by some type of ethics committee or governing body. This means that an outside group has the power to influence what you can and cannot do in research involving human participants, often requiring that the proposed research be fully vetted and approved by the overseeing body before the research can commence. Virtually all ethics committees for universities work in this fashion. Such outside research vetting bodies, by virtue of their power, become de facto stakeholders in the research and may actually be held accountable if breaches in research ethics occur within their purview.

10.6.2 Positioning of Indigenous Participants In several countries, including Australia, New Zealand, Canada and the U.S., there are distinct ethical principles and guidelines for research involving Indigenous, Native or First Nation peoples (see the shaded rows in Table 10.2). Such guidelines exist because of differences in worldviews, beliefs about the nature of knowledge and relationships between the physical, mental and spiritual worlds, and the role of culture and tradition as it bears on learning and knowledge relative to non-Indigenous cultures (see, for example, the excellent discussions in Chilisa, 2012, Chap. 2; Kovach, 2009, Chap. 8). Critical to these codes of ethics is the fostering of relationships and building of trust. Some codes (e.g., codes adopted by the Australian and Canadian Governments) generally modify and/or augment codes of ethics that apply to non-Indigenous people. For example, the NHMRC document, entitled Ethical Conduct in Research with Aboriginal and Torres Strait Islander Peoples and Communities: Guidelines for Researchers and Stakeholders, built their guidelines around six distinct values: Spirit and Integrity; Cultural Continuity, Equity, Reciprocity; Respect; and Responsibility (NHMRC, 2018, p. 3). These values are continually being reaffirmed through time from the past to the future and Spirit and Integrity forms the integrating central value weaving together the remaining five values. The code shows how each value might be demonstrated in/reflected by research and each is linked to specific ethical requirements for researchers from the National Statement on Ethical Conduct in Human Research 2007 (updated 2018) (NHMRC, ARC & UA, 2018). In contrast, some codes of ethics for research on Indigenous peoples are created by Indigenous people themselves (e.g., Aboriginal and Torres Strait Islanders in Australia, Māori in New Zealand, Inuit in Canada) and these codes take a distinctly different approach, seeking instead to construct a specific ‘decolonised’ ethical framework for research involving Indigenous people within that country and establish Indigenous people as a collective stakeholder in research. For example, the Guidelines for Ethical Research in Australian Indigenous Studies, produced by

334

10 How Should I Contextualise and Position My Study?

the Australian Institute of Aboriginal and Torres Strait Islander Studies (AISTSIS), details the following 14 principles for ethical research involving Indigenous people: • Principle 1—Recognition of the diversity and uniqueness of peoples as well as of individuals is essential. • Principle 2—The rights of Indigenous peoples to self-determination must be recognised. • Principle 3—The rights of Indigenous people to their intangible heritage must be recognised. • Principle 4—Rights in the traditional knowledge and traditional cultural expressions of Indigenous peoples must be respected, protected and maintained. • Principle 5—Indigenous knowledge, practices and innovations must be respected, protected and maintained. • Principle 6—Consultation, negotiation and free, prior and informed consent are the foundations for research with or about Indigenous peoples. • Principle 7—Responsibility for consultation and negotiation is ongoing. • Principle 8—Consultation and negotiation should achieve mutual understanding about the proposed research. • Principle 9—Negotiation should result in a formal agreement for the conduct of a research project. • Principle 10—Indigenous people have the right to full participation appropriate to their skills and experiences in research projects and processes. • Principle 11—Indigenous people involved in research, or who may be affected by research, should benefit from, and not be disadvantaged by, the research project. • Principle 12—Research outcomes should include specific results that respond to the needs and interests of Indigenous people. • Principle 13—Plans should be agreed for managing use of, and access to, research results. • Principle 14—Research projects should include appropriate mechanisms and procedures for reporting on ethical aspects of the research and complying with these guidelines. (AIATSIS, 2012, p. 2) The ethical dimension of positioning of human participants is probably the most challenging aspect for researchers to grapple with. It raises dilemmas that need to be resolved, creates tensions between what you would ideally like to do and what is ethically appropriate to do and reinforces the need to plan, modify and extend your research activities so that ethical expectations can be met. This means that participants are always the most immediate stakeholders in the research. Doing research involving Indigenous people magnifies the complexity and sensitivity of the issues you must deal with; issues created by the historical oppression of Indigenous peoples by non-Indigenous people. The complexities multiply even more if you are a non-Indigenous person seeking to do research with Indigenous people or in an Indigenous community. It is the ethical dimension that truly brings home the point

10.6

Positioning of Participants and Other Data Sources

335

that you, as a social or behavioural researcher, do not have power over every aspect of your research journey; some of your power must be deliberately sacrificed to meet the demands of larger more systemic concerns.

10.6.3 Positioning of Non-human Data Sources The positioning of non-human data sources looks somewhat different but is no less important to address. If you wish to access or draw upon a non-human data source for your research, you need to understand: • the status of the data source (e.g., non-fiction or fiction); • whether the data source is formal (aligned with some institution or group of interest) or informal; • whether it has a regulatory or policy focus; • whether it is sacred and/or of high cultural value or symbolic value/associated with a specific ritual; • its relevance to the research context; • who created it and why; • who owns it; • who controls access to it, why and how (points to relevant gatekeepers); • when and where it was created, in what context and for what purpose(s); • what its intended audience is/was; and • the nature of its contents (e.g., first hand reflection/recording of events, experiences, images, artworks, creative ideas, performances or thoughts/ observations; second-hand reflection/recording of other’s events, experiences, images, creative ideas or thoughts/observations, financial or economic data). Some non-human data sources, such as secondary organisational, institutional, economic or financial databases, policy documents, social media postings, have dynamic content, which means it is important to understand who maintains the data source, how frequently content is updated, whether or not that content has been filtered or vetted in some way and by whom, and how the quality/accuracy/ currency/consistency of content is monitored and ensured. Without a clear understanding of the positioning of non-human data sources, coupled with a clear reflection of that understanding in any research outcome, it is much more difficult to be convincing about how relevant or meaningful any content from that data source is for your intended purposes. As an added complexity, if you want to access certain institutional or organisational data sources (e.g., internal reports, strategy documents, long-range forecasts) that have been flagged as commercial-in-confidence, this places ethical constraints upon your use of the information provided by or within those data sources. You may, in fact, either be denied access to those documents, even if they would be valuable for your research or constrained as to how you use such information and whether you can summarise or otherwise

336

10 How Should I Contextualise and Position My Study?

represent the information in any research outcome you produce. Gatekeepers may require you to sign a non-disclosure agreement, which will basically mean that you cannot use the information provided by the data source in any way, even as notes in your research journal. In short, such data sources may simply not be feasible to include in your research and you would have to be transparent about any gaps in your knowledge or learning that this constraint might create for you.

10.7

Focusing in on Issues Relevant to Your Specific Research Context(s)

The more general contextualisation and positioning activities associated with you, as researcher, other research and participants and other data sources all converge while making choices about the specific context(s) in which your research is to be conducted. Various contextualisation strategies and positionings may influence the choice of and considerations associated with specific research context(s) and choice of specific research context(s) may shape various positionings in a kind of ‘which-comes-first-chicken-or-egg’ dynamic. This dynamic continues throughout the duration of your research and cannot be fully anticipated and planned for prior to research commencement, because constraints, surprises and opportunities emerge over time. The best you can hope to do is to try to anticipate and plan for as much as possible, while acknowledging the uncertainties associated with those activities and preparing yourself to be flexible and adaptable, if unforeseen circumstances arise. Accordingly, there are a range of considerations that arise in association with your interest in specific research context(s); the nature and dynamic implications of each of these considerations is discussed below. • Research goals, objectives, questions/hypotheses. Your research goals, objectives, questions/hypotheses typically emerge from the confluence of contextualisation and positioning strategies and directly implicate your chosen research context(s). You must form a clear idea of what you want/intend to learn and in what context(s) and, associated with that, what objectives you want to achieve, questions you want to address and/or hypotheses you want to test against the background of your adopted pattern(s) of guiding assumptions. This will then facilitate your downstream decision making regarding research configuration, data gathering and analysis and creation of research outcomes. You will need to be able to convey your research goals, objectives, questions/hypotheses to relevant research readers/users via relevant research outcomes (the first instance of which, for most postgraduates, is your research proposal). Much more will be said about research goals, objectives, questions/hypotheses in Chap. 11. • People. This involves considerations of (1) who is relevant in the research context (which can help you to identify not only potential participants, but also potential stakeholders and gatekeepers); (2) when do people become relevant, including pertinent historical details about relevant people in the context;

10.7

Focusing in on Issues Relevant to Your Specific Research …

337

(3) where are relevant people located, which encompasses cultural considerations as well; and (4) why are specific people relevant to/suitable for participation in your research. Choice of research context may raise positioning issues to consider with respect to potential participants, gatekeepers and stakeholders. For example, if your research context is a naturalistic setting, such as a farm, classroom, emergency room, council meeting or office, then addressing who, when, where and why issues may lead to identification of key people to connect with for gathering data, to negotiate with for gaining access to people and/or to ensure that any learning that emerges from your research benefits rather than harms others not directly involved in that research. Even if your research context is a laboratory setting, people-oriented considerations still emerge. Here, undergraduate or postgraduate students may constitute the potential participants to enlist, which may mean they may have a vested interest in gaining course credit or remuneration in exchange for their participation and may constrain the extent to which you might wish to extend the learning found in your research beyond the boundaries of the laboratory. • Controllability. This involves considerations about the controllability/ malleability of the specific research context, i.e., the choice between doing your research in a laboratory or in the field under naturalistic circumstances. This consideration depends heavily upon your positioning with respect to adopted pattern of guiding assumptions. Laboratory-based, highly controlled research is most closely associated with the positivist pattern of guiding assumptions, whereas naturalistic field research is more closely associated with interpretivist/constructivist or other non-positivist patterns of guiding assumptions. However, this is not a one-to-one relationship, as positivist research (even experimentation) can be done in the field in naturalistic settings, yielding what is referred to as a quasi-experiment. If research is done in a laboratory under controlled conditions, the design of experimental laboratory tasks to be undertaken by or conditions to be experienced by participant becomes an important consideration, i.e., the representative design of tasks and conditions (Brunswik, 1952, 1956). If this consideration is ignored, your capacity to generalise beyond the laboratory context is greatly constrained. For example, if participants in a laboratory experiment are asked to do a task they have never done in their lives before or to participate under environmental conditions they have not experienced before, then generalisations based on those tasks and conditions are much harder to make convincing. Brunswik (1956) suggested that this problem could be easily solved by focusing as much effort on sampling the life spaces of potential participants to design tasks and conditions as on sampling of people/ participants from some larger population (the latter being a statistical concern in terms of achieving a representative sample). • Prior relationships. If there are pre-existing relationships between you and people in the research context (as when you do research in your employing organisation or in one you worked in/for previously; or a context where you have contacts/friends; or where you have a contractual/paid arrangement with management to conduct research, as an outsider consultant), the potential

338

10 How Should I Contextualise and Position My Study?

implications of those relationships for your research must be explored and managed. Prior relationships can also influence participant positioning, especially if people in those relationships are gatekeepers controlling access to other people or to documents or if you wish to involve people you have a known relationship with as participants (e.g., when a manager interviews one of her own subordinates). • Key stakeholders and gatekeepers. Every research context has potential stakeholders associated with it who stand to gain or lose depending upon how your research journey plays out and what outcomes are generated and for whom. Even laboratory research has key stakeholders, including the institution who governs/owns the laboratory resources and possible downstream people and contexts about which generalising conclusions and implications might be drawn. Participants should always be considered as key stakeholders who are to be safeguarded by ethical principles and practices. Where non-human data sources are accessed, the authors/maintainers of the documents become important stakeholders to consider as well as any people, groups and/or organisations to which such documents belong. Other potentially relevant stakeholders in your research might include: organisations who fund your research (perhaps via a grant help by your supervisor); regulatory and policy-making authorities for whom your research might have implications; editors of books and journals who publish and disseminate your research; public sector departments like a police department, department of correctional services or local city council: tribal elders in Indigenous communities; CEOs/principals, middle managers and supervisors in organisations and institutions; parents/guardians (if children are to be participants); downstream customers, clients, organisations and even society (including Indigenous people as a collective) at large; potential adopters of an innovation on which your research is focused and so on. In short, you need to take the widest possible view of potential stakeholders in order to properly position yourself and participants and other data sources as well as the research context(s). These considerations can also help you to shape your research outcomes to meet certain stakeholder expectations. Gatekeepers are a specific category of stakeholder who control access to desired participants and/or other data sources. You must devote considerable effort to passing the gatekeepers to gain access. However, gatekeepers, as part of these negotiations, may require something in return for allowing access as flagged earlier and you must consider carefully what you promise in exchange for access and ensure that such promises are kept (not only preserves your own integrity but also helps to maintain good will when other researchers seek access in the future). Finally, part of what gatekeepers may demand in exchange for granting you access is at least partial, if not full control, over how and where the research outcomes are disseminated. If your research outcomes touch on commercial-in-confidence issues, gatekeepers have a legitimate right to impose restrictions on the dissemination of your research. In some cases, you may be barred from publishing, at least for some time period. In many universities, PhD or professional doctorate thesis, dissertation or portfolio documents which report on field research with specific

10.7

Focusing in on Issues Relevant to Your Specific Research …

339

businesses, organisations or communities may be embargoed for a specific period at the request of one or more gatekeepers. In this discussion, we see that gatekeepers can potentially impose bi-directional influence on the research: your access to an organisation and/or relevant participants and other data sources as well as on the outward dissemination of your research. • Constraints and expectations. Here, considerations revolve around localised emerging constraints within the research context itself and expectations that arise from key stakeholders or gatekeepers in the context. For example, gatekeepers within an organisation may require things from you in return for granting research access, such as regular updates, a report to the organisation, the right to view/vet any report before public release (which can create an ethical dilemma for you if your research findings are unfavourable to the organisation), recommendations for change, insights into who said what during data gathering (creates another ethical dilemma for you, vis-à-vis prioritising protection of privacy/identity of participants over what the gatekeepers in the research context are demanding). An emergent constraint from a research context could be something as drastic as the organisation withdrawing its permission for your access once your research has commenced or the organisation going into bankruptcy during your research (which means your sampling strategy has to change) or something less drastic (but still requiring some adaptation on your part), such as when a school states that students in the same classroom should all be treated in the same way, meaning that those students in the same class cannot be assigned to different experimental conditions, even if that is what you wish to do. • Accessible/available resources. The research context may or may not offer the physical space (e.g., a room for conducting interviews, an office for you to work in), financial or in-kind support (e.g., for research activities such as photocopying, support for travel expenses or for additional research personnel), technological support (e.g., internet access for the researcher, space on the organisation’s website for questionnaire distribution, access to email for soliciting/securing participants; permission to digitally, photographically or video record aspects of the research context), and/or personnel support (e.g., persons to liaise with and organise meetings with gatekeepers, schedule participants for data gathering sessions). Where certain necessary resources are not available, you will need to adapt your research activities so that the resource constraint has minimal impact. • Larger contextual background issues. Laboratory and naturalistic research contexts are all embedded within a larger set of contextual settings (e.g., a behavioural laboratory may be owned and resourced by a university; an organisation may be part of a larger public sector government context; a local council is embedded with a larger community that they serve; a focal group under study is part of a larger organisation or institution; organisations may have to abide by regulatory/governance requirements; organisations have competitors, markets/consumers/clients that they can be influenced by/answer to/deal with). Consequently, there may be constraints and opportunities that emerge

340

10 How Should I Contextualise and Position My Study?

from these larger contexts in which your research context is embedded. For example, an organisation or institution you are researching may be currently experiencing or has very recently experienced a major historical event such as increased pressure from new competitors in the market, imposition of new regulatory or legal requirements, being taken over or absorbed by another company, or suffering a reduction or scoring an increase or in government funding and such an event may alter conditions of access or expectations of you as the researcher and may even fundamentally change the context so that it is no longer suitable for your research. Adaptation will almost always be imperative in such circumstances, not least putting in place data gathering strategies that will provide you with insights into these events and their impact, so you can use that knowledge to qualify any research findings and perhaps redirect or reshape some data gathering activities (adaptations that can be evaluated using the contextual sensitivity meta-criterion). A facility in which you wish to conduct your research, if it is owned by a larger organisation, may be subject to scheduling (to manage competing pressures from other researchers) and resource-based (e.g., fee/costs for use; limitations on time of use) constraints that you will have to manage. Even if you are conducting research only using data gathered from secondary economic or financial databases (i.e., no human participants involved at all), there will be considerations to manage such as knowing who owns the databases and controls access to them so that access can be negotiated or paid for. Those database owners may have their own expectations about what can/should be done with the data they collect and store and how that source of data should be referenced/cited in any research outcome that will need to be managed.

10.7.1 Political Contextual Influences in and/or on Your Specific Research Context(s) Politics comes into play in research in the relationships between you, as the researcher, and key stakeholders and gatekeepers with respect to specific research contexts you are interested in. Politics is about the exercise of power to achieve specific goals and is a dynamic that emerges when contextualising and positioning your research. From your perspective, you have certain needs which require that you connect or negotiate with key stakeholders and gatekeepers, so your goals can be achieved. This means you try to exercise power over key stakeholders and gatekeepers to gain access to specific research contexts, to resources within those contexts, to participants and other data sources as well as to background information about those research contexts. Such power, from your side, needs to be of a persuasive rather than coercive nature, oriented toward achieving your desired ends without creating bad blood, negative impressions or antipathy in the process. The political process usually involves negotiations or exchanges of some description. For example:

10.7

Focusing in on Issues Relevant to Your Specific Research …

341

• promising ethical protection for participants in exchange for their agreement to participate and hopefully to stay the course with their participation; • promising a tangible outcome, such as payment or other incentive (like a chance to win a holiday), to participants in exchange for their agreement to participate and hopefully to stay the course with their participation; • promising a report on your research results or a set of recommendations for action to key gatekeepers in exchange for access to a research context and perhaps to potential participants and other data sources; this could also be in exchange for allowing you to publish research outcomes; • highlighting the potential benefits might be realised for potential participants (and possibly the wider public, as with, for example, medical research) in exchange for their agreement to participate and hopefully to stay the course with their participation; • highlighting the potential benefits might be realised (such as an economic or human resource benefit for a company; improved student outcomes for a school) benefit in exchange for allowing access to a research context and perhaps to potential participants and other data sources; or • providing a well-developed research proposal suggesting positive outcomes and benefits from a specific project to a funding organisation or another relevant organisation or institution in exchange for a research grant, institutional funding or a consultancy contract. From the perspectives of stakeholders and gatekeepers, they have organisational/ institutional interests that they wish to safeguard as well as their own goals to achieve. However, power leveraged by some stakeholders and gatekeepers may be more coercive than persuasive in nature. In some cases, gatekeeper power is leveraged in terms of conditions to be met before you are permitted to pursue your goals, and, in some instances, this may create ethical dilemmas for you. Stakeholder power may also be leveraged in terms of concrete expectations of you in the conduct of your research or dissemination of its findings and such expectations may be anchored in the knowledge that the stakeholder will be providing resources (financial, physical, technological) to support your research or is safeguarding larger contextual interests (such as national security). Below are some examples. • Gatekeepers who control access to research contexts and/or potential participants or other data sources may demand a report on the research findings from you or, more seriously, access to who said what to you during your data gathering activities (thereby creating a severe ethical dilemma if you want to promise anonymity to potential participants). • An individual, organisation or institution may require you to focus your research on a specific innovation or on a specific problem or certain aspects of a problem as a condition of providing access, typically though a consultative arrangement (which may have financial implications as well that could exert additional leverage on you).

342

10 How Should I Contextualise and Position My Study?

• A funding organisation (e.g., Australian Research Council, National Science Foundation) may demand that you (or, more likely, your supervisor if the funding was granted to them) provide it with periodic progress reports as well as a final report on your research project as a condition of providing and/or continuing funding and, in addition, may expect proper public acknowledgement of their support in any research outcome. • An individual, organisation or institution may expect to dictate or control the dissemination and distribution of research benefits and outcomes, especially where they have provided resources to support your research and, in certain cases, this could lead to difficulties in allocating and managing intellectual property rights. • An individual, organisation or institution may expect enhancement of their reputation or public stance, a political advantage or a marketing advantage for an innovation as an outcome of the research, creating pressure on you to deliver such outcomes. • A supervisor of your thesis, dissertation or portfolio may demand, as an expectation of their provision of supervisory input, that they be listed as an author (in some cases and, in certain disciplines, as first author) on any research publications that flow from your research project. • An individual, organisation or institution may demand that any commercial-in-confidence information uncovered during your research be suppressed in any research outcomes, so that their reputation, market or political advantage and/or customer relationships are not threatened and may, in addition, demand that their identity and/or participants’ identities be hidden or anonymised. • An individual, organisation or institution may request or demand changes to your research process or to the content of disseminated research outcomes (so-called ‘whitewashing’) in exchange for providing access or funding support (e.g., drug trials funded by pharmaceutical companies may be expected to favour a new drug the company wishes to market; cancer research funded by the tobacco industry may be expected to be conducted in such a way as to favour a tobacco-industry perspective, the federal government may put pressure on you to soften or alter your research conclusions/recommendations in contentious areas related to national policy or security (e.g., climate change, trade, terrorism)). • An Indigenous community may expect reciprocity or return of value to the community from your research as a condition of granting you access, often with the additional expectation that their cultural traditions must be respected. • A journal editor may ask you to make specific changes to your research manuscript to meet concerns expressed by reviewers as well as the editor in order to increase the chances that your manuscript will be accepted for publication by that journal. • A university’s human ethics committee may demand that you change your plan for sampling participants so that no participant is asked to provide any names of

10.7

Focusing in on Issues Relevant to Your Specific Research …

343

or contact details for other potential participants (what is called snowball sampling, see Chap. 19). The upshot of this discussion about political influences in research is that you cannot ignore or neglect dealing with them. Instead, you must actively plan for how to deal with those influences that can be anticipated and adapt where their influences emerge downstream. Some demands made by key stakeholders or gatekeepers may place you in an ethical bind that you will need to resolve. It is important to recognise that such dilemmas can create emotional as well as cognitive stress for you as you work through whether you will comply with the demand being made. It is here where ethical as well as personal codes of conduct and beliefs may conflict, and you will be forced to choose a pathway forward. Acceding to an ethically problematic demand may create internal stress and regret for you (as well as public condemnation), even if you achieve proper professional recognition for your research outcomes. Not acceding to an ethically problematic demand may mean that you do not get to do the research you wanted to conduct, and this could have downstream effects on your career. Certainly, not all demands made by stakeholders and gatekeepers create ethical dilemmas or coercive pressures and many requests may actually be made in a collaborative spirit. Many demands can easily be managed through effective negotiation (e.g., considering what you are willing to exchange or promise to gain access; considering how to best express research outcomes to meet specific stakeholder needs), but you need to recognise that this adds to the timeline for your research and it may affect various facets of your research journey, maybe even the nature of your research itself.

10.8

Illustrative Contextualisation and Positioning Arguments

Figure 10.2 displays excerpts from the Abstracts of a recent PhD thesis (Fig. 10.2a) and a recent professional doctorate portfolio (PhD.I) from the University of New England (Fig. 10.2b), both of which were supervised by Ray. Martin Robson (2011) carried out a traditional academic PhD research project, focusing on intuitive decision making by elite Australian leaders. In contrast, Wayne Gregson (2016) carried out a new form of professional doctorate research, called the PhD.I at UNE. For this doctorate, postgraduates must develop and evaluate the potential of an innovation they create in the profession/industry in which they work and embed their research within a portfolio. For Wayne, this meant developing and evaluating his own innovation for the Department of Fire and Emergency Services in Western Australia. At the time of the research, Gregson was the Western Australian Fire and Emergency Services Commissioner, hence his reference to ‘insider’ research in his Abstract. Each Abstract excerpt is annotated to illustrate contextualisation and

344

10 How Should I Contextualise and Position My Study? a) PhD Thesis (Robson, 2011) As workers, managers, leaders, researchers and theoreticians in organisations and in society – indeed, as humans – I argue that we continue to undervalue and underplay the role of the visceral, the tacit, the silent, the shadow, the emotional and the intuitive. Nonrational influences in the public domain, in particular, the organisations that influence our daily lives, have either been ignored or seen as irrational – something to be avoided, negated, managed, corrected, punished, excluded or in the case of intuition, marginalised, hidden and silenced. Educational institutions prepare students for an organisational life in which instrumental rationality is assumed and expected. However, the assumption that leaders in organisations are exclusively rational in their behaviour and decision-making processes is one that has come under increasing scrutiny. Research has shown that leaders use intuition frequently and consider it important to their role and effectiveness. The same research however, has also revealed that intuitions are often masked in analytical terms or suppressed. A contention of this thesis is that the cost of not acknowledging intuition or accounting for and incorporating it in work discourse and practices is high. Intuition disclosure in organisations has never been the focus of empirical research in Australia nor internationally. Studies of intuition to date have been directed at discovering what intuition ‘is’, its powers and pitfalls, and how one can best make use of this subconscious and elusive cognitive capacity. Understanding the nature of intuition and its potential is important, however, I assert that this knowledge is impotent in application unless the social processes surrounding its use and disclosure in the ‘real world’ are also understood. This study employed an approach informed by Grounded Theories to investigate the social processes of intuition use and disclosure at the intrapersonal, interpersonal, organisational and societal levels. Data collected from semi-structured interviews with 27 men and women leaders in significant Australian organisations was analysed using NVivo. Elite leaders were purposively sampled for their influence on, experience and knowledge of, and accountability for, organisational decision-making processes. Their exceptional communication skills provided rich, relevant and revealing data. (Robson, 2011, quoted from Abstract, pp. iii)

Researcher positioning – intrapersonal perspective reflected in 1st person writing

Positioning with other’s research – a critical stance Contextualisation linked to decision makers

Contextualisation – developing a new theory; exploring/filling a perceived gap in knowledge; understanding perspectives in context

Researcher positioning – interpretivist/constructivist pattern of guiding assumptions adopted Positioning with respect to specific research context - research goal Positioning with respect to specific research context -people; key stakeholders Positioning of participants & other data sources – elites corporate leaders with rich contextual knowledge

b) PhD.I Professional Doctorate Portfolio (Gregson, 2016) This Innovation Portfolio Project focuses on the development and implementation of a single workplace innovation, namely the “Portal2Progress” (P2P) to the context of the Western Australian Department of Fire and Emergency Services (DFES). The P2P endeavour sought to harness emergent grassroots innovation ideas within the complexity of the contemporary public sector environment of the DFES, which I lead. The P2P is the Innovation Project that underpins my Professional Doctorate study, which is essentially insider research on the introduction and embedding of P2P as a workplace innovation. Within my role, I was actively involved in the research process and in the innovation project delivery. The organisational goal of this Innovation Portfolio Project was that DFES would benefit practically and culturally from the adoption of the P2P. The P2P mechanism of the cultivation of innovative ideas, percolating within DFES, was intended to make a real difference to the business of the agency; and culturally, by the adoption of those ideas leading to the organisation’s embracing of innovation and learning. The social aim was to add public value to DFES operations through the delivery of improved service to the community and by making a contribution to the field of public sector management. This Innovation Portfolio Project provides a vehicle for the sharing of knowledge, derived from this endeavour. It also provides a reference, available for the benefit of others that might seek to embed an innovation strategy across their organisation. My personal aim from this research was that of self-improvement as a thinker, as a leader and as a scholar. The Innovation Portfolio Project of this workplace research project, articulates the results of my study from a practical, organisational, academic and personal perspective. It also presents my reflections on the contextual conditions I see as more broadly necessary for the successful implementation of change in public service organisations and more specifically, the leadership, organisational structure and power relationships that I believe made change possible in the DFES. Through my reflection on the findings of this study and its significance, I have explored its potential within DFES, the challenges into future and how these might be managed. I also briefly consider the wider impacts for the wider public sector of P2P and what might be achieved by broader adoption into a public sector organisation. (Gregson, 2016, quoted from Abstract, pp. 4-5)

Positioning with respect to specific research context - research goal Positioning with other’s research - innovation stance Contextualisation – improving a process or outcome Positioning of participants & other data sources – organisational members with innovative ideas Researcher positioning – interpersonal and organisation roles & relationships; Contextualisation linked to Wayne’s connection to the research context Positioning with respect to specific research context - people; key stakeholders Contextualisation linked to needs of a professional organisation Positioning with respect to specific research context - people; key stakeholders; larger contextual issues

Positioning with respect to specific research context - larger contextual issues Contextualisation – improving a process or outcome

Contextualisation linked to possible futures

Fig. 10.2 Annotated doctoral thesis and professional doctorate portfolio abstract excerpts showing how each reflected contextualisation and positioning strategies

10.8

Illustrative Contextualisation and Positioning Arguments

345

positioning strategies in action. While contextualisation and positioning discussions received a great deal more development in relevant sections of each postgraduate student’s document, looking at their Abstracts as helicopter summaries of their research should be instructive.

10.9

Key Recommendations

Some important things to remember regarding contextualisation and positioning of your research include: • The four Contextualisation meta-criteria (recall Chap. 9) all deal explicitly with different facets of contextualisation and positioning as these strategies are required in all research, irrespective of pattern of guiding assumptions (in fact, the choice of pattern(s) of guiding assumptions, is itself a researcher positioning activity). • Contextualise the broad focus and intent(s) of your research early on in your research journey. This will not only help you to focus your research but will also help research readers/users understand what you want to do and what your intent (s) might be linked to. You don’t have to name your contextualisation and positioning strategies; rather you reflect their implementation through your writing, as shown in the two examples in Fig. 10.2. You will find that you continually reflect on your contextualisations and positionings as they can assist you in identifying data sources, facilitate data interpretation and learning and, most of all, inform your writing. This is one important function that the contextual sensitivity meta-criterion addresses. Throughout this contextualisation process, don’t lose sight of the relevance of a code of conduct to your obligations as a researcher. • By engaging in critical prospective as well as retrospective researcher positioning (identified with a meta-criterion of the same name), you can set the stage for where you are coming from as a researcher, i.e., ‘what you are bringing to the table’. This is essential as it will help a research reader/user understand why you have made the choices you have during your research journey. Remember that there are several domains of reflection implicated here, the intrapersonal, the interpersonal/social, the professional and the contextual, and all should be addressed. Research readers/users will be on the lookout for inconsistencies between where you say you are coming from and what you actually do. • Positioning with other’s research, identified with a specific meta-criterion, juxtapositioning with other research, is where you locate your research relative to past relevant research. Remember that there are a number of ways to make use of any piece of research you review, each way leading to a specific type of learning (e.g., theoretical, methodological, practical, contextual, gap identification). Whatever use you make of a piece of research, make sure you do so critically and analytically. You want to be able to show exactly how it informs

346

10 How Should I Contextualise and Position My Study?

your research. Be guided by the approximate hierarchy of research outcome quality, but don’t be exclusively driven by it. Be open to the potential value that grey literature and trade/professional/practice-oriented literature can add to your arguments, especially if you are undertaking a professional doctorate. (Refer to Chap. 13 for much more relevant information about reviewing literature.) • Positioning of participants and other data sources, identified with a specific meta-criterion by the same name, is essential to understand as you cannot maintain contextual sensitivity if you don’t understand where participants or other non-human data sources are coming from. Such knowledge helps to place participants in the research context and helps you to accord them an appropriate level of acknowledgement, perhaps even input, into your research processes and data gathering activities. If your participants are Indigenous or the research context is Indigenous, then undertaking this type of positioning, to a deep level, is essential, especially if you are a non-Indigenous person attempt to do research within Indigenous people or communities. Understand that using human participants in your research automatically invokes the need for ethical principles to be applied and a wide range of codes of ethics exist. Realise that, for Indigenous research, there is likely to be a separate code of ethics that is sensitive to the specific Indigenous histories of the country you are working in. • Contextualisation with respect your specific research context(s) represents the localised convergence of your contextualising and positioning activities; where the ‘rubber meets the road’, so to speak. What is important here and why we distinguish this from contextualisation strategies in general is that you focus in on the research context itself to critically understand your role within it as well as the roles that participants, stakeholders, gatekeepers (and your relationships with them) will play. Also critical here is the emergence of your research goals, objectives and questions/hypotheses. Contextualisation with respect your specific research context(s) never stops. You will be continually referring to, reflecting on and perhaps modifying/adapting such contextualisation as your research journey unfolds as well as reflecting on what you learn with respect to your research goals, objectives and questions/hypotheses once your data, interpretations and conclusions are in and you begin to produce a research outcome. This is a second major function that the contextual sensitivity meta-criterion addresses.

References AIATSIS. (2012). Guidelines for ethical research in Australian Indigenous studies. Canberra, Australia: Australian Institute of Aboriginal and Torres Strait Islander Studies. Allen, P., Maguire, S., & McKelvey, B. (Eds.). (2011). The Sage handbook of complexity and management. Los Angeles: Sage Publications. Beach, L. R. (Ed.). (1998). Image theory: Theoretical and empirical foundations. Mahwah, NJ: Lawrence Erlbaum Associates.

References

347

Benzies, K. M., Premji, S., Hayden, K. A., & Serrett, K. (2006). State-of-the-evidence reviews: Advantages and challenges of including grey literature. Worldviews on Evidence-Based Nursing, 2, 55–61. Brunswik, E. (1952). The conceptual framework of psychology. Chicago: University of Chicago Press. Brunswik, E. (1956). Perception and the representative design of psychological experiments (2nd ed.). Berkeley, CA: University of California Press. Bryman, A., & Bell, E. (2015). Business research methods (4th ed.). Oxford, UK: Oxford University Press. Campbell, R., et al. (2012). Evaluating meta ethnography: Systematic analysis and synthesis of qualitative research. Health Technology Assessment, 15(43). Cooksey, R. W. (1996). Judgment analysis: Theory, methods, and applications. San Diego: Academic Press. Chilisa, B. (2012). Indigenous research methodologies. Los Angeles: Sage Publications. Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). New York: Routledge. Creswell, J. W., & Plano Clark, L. (2018). Designing and conducting mixed methods research (3rd ed.). Los Angeles: Sage Publications. Doyle, L. H. (2003). Synthesis through meta-ethnography: Paradoxes, enhancements, and possibilities. Qualitative Research, 3(3), 321–344. Fisher, C. (2010). Researching and writing a dissertation: A guidebook for business students (3rd ed.). Harlow, UK: Prentice Education Ltd. Gigerenzer, G., Todd, P. M., & The ABC Research Group. (1999). Simple heuristics that make us smart. New York: Oxford University Press. Gregson, W. (2016). Harnessing sources of innovation, useful knowledge and leadership within a complex public sector agency network: A reflective practice perspective. Unpublished PhD.I portfolio, UNE Business School, University of New England, Armidale, NSW. Hammond, K. R. (1996). Human judgment and social policy. New York: Oxford University Press. Hammond, K. R., & Stewart, T. R. (Eds.). (2001). The essential Brunswik: Beginnings, explications, applications. New York: Oxford University Press. Hattie, J. A. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London: Routledge. Kaptchuk, T. J. (2001). The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf? Journal of Clinical Epidemiology, 54(6), 541–549. Kaptchuk, T. J. (2003). Effect of interpretive bias on research evidence. BMJ, 326, 1453–1455. Kovach, M. (2009). Indigenous methodologies: Characteristics, conversations, and contexts. Toronto: University of Toronto Press. Lewis, V., & Habeshaw, S. (1997). 53 interesting ways to supervise student projects, dissertations and theses. Bristol, UK: Technical and Educational Services Ltd. Lichtenstein, S. A., & Slovic, P. (Eds.). (2006). The construction of preference. New York: Cambridge University Press. Mauthner, M., Jessop, J., Miller, T., & Birch, M. (2012). Ethics in qualitative research (2nd ed.). London: Sage Publications. NHMRC, ARC, & UA (2018). National statement on ethical conduct in human research 2007 (updated 2018). The National Health and Medical Research Council, the Australian Research Council and Universities Australia. Canberra: Commonwealth of Australia. NHMRC. (2018). Ethical conduct in research with Aboriginal and Torres Strait Islander peoples and communities: Guidelines for researchers and stakeholders. Canberra, Australia: The National Health and Medical Research Council. Robson, M. (2011). The use and disclosure of intuition(s) by leaders in Australian organisations: A grounded theory. Unpublished Ph.D. thesis, School of Economics, Business & Public Policy, University of New England, Armidale, NSW. Schuelka, M., & Maxwell, T. W. (Eds.). (2016). Education in Bhutan: Culture, schooling, and gross national happiness. Singapore: Springer Science.

348

10 How Should I Contextualise and Position My Study?

Senge, P. M. (1994). The fifth discipline fieldbook: Strategies and tools for building a learning organization. New York: Crown Business. Sinclair, M. (Ed.). (2014). Handbook of research methods on intuition. Cheltenham, UK: Edward Elgar Publishers. Tashakkori, A., & Teddlie, C. (Eds.). (2010). Sage handbook of mixed methods in social & behavioral research (2nd ed.). Thousand Oaks, CA: Sage Publications. Thomas, R. M., & Brubaker, D. L. (2008). Theses and dissertations: A guide to planning, research and writing (2nd ed.). Thousand Oaks, CA: Corwin Press. Walsh, D., & Downe, S. (2005). Meta-synthesis method for qualitative research: A literature review. Journal of Advanced Nursing, 50(2), 204–211.

Chapter 11

How Do I Frame and Conceptualise My Research Problem and Questions?

11.1

Choice of Research Frame

Many of my (Ray’s) postgraduate students have told me that they find research methods texts confusing because the term ‘method’ is often used not only to refer to specific ways of gathering data (e.g., interview, questionnaire) but also to more general research approaches like action research, transdisciplinary research, survey research and case study research. The problem is that different levels of conceptual abstraction are at play here, which many authors simply ignore. A data gathering strategy is simply that—a set of actions undertaken in your research to collect a specific kind of data from a specific type of data source under a specific set of guiding assumptions. Action research is not a data gathering strategy, ‘method’ or a ‘methodology’, but a way of holistically conceptualising research. The same is true for ‘methodologies’ like survey research and case study research. What was needed was a higher-order concept for systematically organising research thinking, which has led me to develop the systems-level concept of a research frame. A research frame emerges from the dynamic and synergistic intersection of researcher positioning, research contexts, participants’ contexts and positioning and research sponsor/reader/user contexts, all embedded within the larger social, political and physical worlds. It helps to contextualise your research in a way that provides a more holistic picture of how your research purposes can/will be translated into research strategies and tactics that will generate or apply knowledge and learning that will speak to and influence specific audiences. In this, we see that a research frame embeds ways of creating, applying and disseminating knowledge about the social world that might be relevant to specific stakeholders in that research. Figure 11.1 displays a mindmap of different frames that can be identified for social and behavioural research. Each research frame can stand on its own to conceptualise and contextualise your research. However, it is also important to realise that it is possible to synergistically combine different frames to form hybrids (illustrated by the dashed links connecting frames in the mindmap) in order to achieve specific © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_11

349

11

Fig. 11.1 Expanded mindmap of important frames for social and behavioural science research highlighting some key features of each frame (dashed links show potential synergistic combinations of frames for research)

350 How Do I Frame and Conceptualise My Research …

11.1

Choice of Research Frame

351

research aims. For example, we can envisage a feminist action research frame; a cross-cultural Indigenous research frame; a cross-cultural survey research frame; a transdisciplinary developmental evaluation frame; an evaluative survey frame; an action research developmental evaluation (e.g., participatory action research for social innovation) frame; a descriptive case study frame; an explanatory (theory-testing/experimental) survey frame; or a descriptive/exploratory/explanatory frame (phenomenography, grounded theory or ethnography). While a research frame may or may not implicate specific patterns of guiding assumptions or the use of specific data gathering strategies, they may help you to narrow your choices as well as to identify key areas of literature you should explore. A research frame may also utilise specific concepts and jargon which can facilitate communication with certain types of research sponsors/readers/users (but could also possibly inhibit communication with other types). As well, each research frame comes with its own in-built constraints, expectations and assumptions which you need to be aware of and work with. You might be wondering why quantitative research, qualitative research and mixed methods research are not considered to constitute research frames. The reason is that they are far too general to be helpful in focusing and situating your research. In addition, they only focus on the type of data gathered, which is a very small part of the much bigger set of considerations associated with the choice and implementation of a research frame. Any of the frames shown in Fig. 11.1 may employ strategies for gathering and interpreting quantitative and/or qualitative data. Political dynamics are implicit in certain research frames (e.g., Indigenous research, feminist research, action research, evaluation research) where critical social science or similar non-value neutral patterns of guiding assumptions have been adopted. In these types of frames, you may, from the outset, be oriented toward revealing disadvantage or correcting social perceptions or injustices. Your political agenda here would be one of change and, in some cases, you expect to be an agent of that change, through your expert power as a researcher and, in other cases, those being researched (who may feel politically disadvantaged or silenced, unjustly treated or otherwise held in a position of lesser status in society) may expect your research to aid in carrying their change agenda forward to wider society. Specific research frames may also bring political forces between competing stakeholders to the fore. For example, action research may try to facilitate organisational change in a climate that has been traditionally reluctant to embrace such change. Indigenous or feminist research may try to expose oppressive or colonising practices attributable to what they view as the dominant cultural milieu. Developmental evaluation may try to bring an innovation into a specific context where adoption of innovations has typically been very slow and/or strongly resisted. Whatever the specifics of the political dynamic associated with a research frame, the bottom line is you cannot ignore it. You must engage and manage it in such a way as to facilitate achieving a convincing research outcome while, at the same time, avoiding the adverse consequences of political alienation (e.g., reputational damage, resistance to future funding requests, whether by you or your supervisor, resistance to requests for access made by other researchers).

352

11

How Do I Frame and Conceptualise My Research …

Below, we will discuss the 12 different research frames shown in Fig. 11.1. A brief introduction for each frame will be offered, followed by discussion of relevant considerations associated with researcher positioning(s), research context (s), participant positioning(s), preferred patterns of guiding assumptions and data gathering strategies, and learning focus for research sponsors/readers/users. Following this, some useful references for the research frame will be signalled. Not that for each research frame, we make reference to preferred data gathering strategies; these reference foreshadow the deeper exploration of such strategies in Chap. 14.

11.1.1 Action Research Frame Action research has the explicit goal of pursuing change to improve social/human conditions, often in localised contexts. While there are many ‘flavours’ of action research, they are all hallmarks of the same basic frame: produce localised change for the better. Thus, action research is often associated with the critical social science pattern of guiding assumptions. Within this frame, you can employ a range of data gathering strategies to achieve the goal of targeted improvement and change within a group, institutional or organisational context. Specific data gathering strategies used in action research may reflect a positivist or an interpretivist/ constructivist orientation, but the latter stance is more common as is a preference for qualitative data. One widely-used variant of action research, particularly in the education, management and innovation fields, is participatory action research (McIntyre, 2008), where your research is participant-driven and where one of the participants might be you as the actual researcher, researching within your own or another group, school, community or organisation. In this particular version of the action research frame, guidance would be offered by participatory inquiry assumptions, participants in the research context would generally control the research agenda, and all data gathering would be geared toward obtaining evidence that can be used for learning what things need to be improved or changed, for planning how to achieve improvements or change, and for showing the effectiveness of implemented change and improvement processes or interventions. Action research tends to unfold in a cyclical manner (see Fig. 11.2) moving through a series of stages (diagnose, plan change, implement change/intervene, evaluate change, learn and modify where necessary), the outcome of which may lead to successive research cycles. In this way, the action research frame can be seen to favour Mode 2 knowledge production, knowledge that is co-produced and shared between researchers and participants in localised context(s) where change can be most meaningful and most impactful. • Researcher Positioning(s): You, as the researcher, act a participant (a researcher-as-participant role) therefore becoming an insider in the research context rather than maintaining an objective and more distant hands-off stance.

11.1

Choice of Research Frame

353

Fig. 11.2 Action research cycles

• Research Context(s): Your research focus is on a localised context within which changes and improvements are to occur (e.g., a work group, a classroom, hospital ward, department, community group). Research unfolds in an action research cycle as you and participants work hand-in-hand in the localised context to map, produce, evaluate and further promote change. • Participants’ Positioning(s): Participants (e.g., workers, teachers, nurses, managers, community members) act as quasi-researchers (participants-as-researchers role), actively engaged with you in seeking data to solve problems, plan changes and evaluate changes. They stand to be most immediately affected by successful or failed changes; that is, they are the immediate research users/stakeholders. • Preferred Guiding Assumptions/Data Gathering Strategies: Overarching guiding assumptions tend to be aligned with the critical social science or participatory inquiry paradigms. Qualitative interaction-based (e.g., interviews) and observation-based (e.g., participant observation) strategies also tend to be preferred (interpretivist/constructivist pattern of guiding assumptions) but positivist-oriented strategies (e.g., quasi-experiments, questionnaires, systematic observation) may also be used. For participatory action research, the best fitting pattern of guiding assumptions is participatory inquiry. • Learning Focus for Research Sponsors/Readers/Users: Action research emphasises Mode 2 knowledge production, where you and participants co-produce knowledge in pursuit of change. Similar groups in similar contexts may benefit from what is learned. Knowledge is disseminated primarily through implementation of successful changes, perhaps facilitating the building of communities of practice. Learning primarily resides in patterns in data that signal needs and focus for action/change and/or impacts/success/failure of actions/change.

354

11

How Do I Frame and Conceptualise My Research …

• Useful References: Gray (2014, Chap. 13) provides a basic introduction to action research. Coghlan and Brannick (2014) have written an excellent text that discusses the issues and processes associated with undertaking action research in your own organisation. Greenwood and Levin (2007) provide a well-rounded book on action research from a social change perspective. McIntyre (2008) discusses, in detail, participatory action research.

11.1.2 Evaluation Research Frame Evaluation research focuses on making assessments and judgments about the efficacy, worth, utility and value of a program, project or intervention in specific contexts with the intention of facilitating decision making about that program or intervention. Key research questions in an evaluation may include: Did the program realise its goals/objectives? Did the program have an impact and, if so, for/on whom and in what time frame? Who benefitted from the program and who did not? Were there any unintended effects of the program? Was the program cost-effective? Did users/participants see the program as effective? Should the program be continued/ invested in/modified/extended into other contexts? Evaluation research may have a formative emphasis, which focuses on how a program, project or intervention evolves over time on the basis of feedback from participants and other stakeholders (could be tied to key milestones within the program, project or intervention) and/or it may have a summative focus, which focuses on the outcomes achieved by the end of the program, project or intervention, perhaps in comparison to where participants were prior to its commencement or to participants in other types of programs. Evaluation research can be guided by any pattern of assumptions, but interestingly, different patterns of guiding assumptions are associated with different schools of thought about evaluation. Schools of thought regarding types of evaluation can be broadly differentiated along two dimensions: scientific (looking at/for ‘objective’ outcomes) to constructivist (looking at/for subjective outcomes) and research (learning/discovery-focused) to pragmatic (decision/impact-focused) as shown in Fig. 11.3. Overlaid on the graphic in Fig. 11.3 is the typical pattern of guiding assumptions associated with each quadrant of the 2-dimensional display. The schools of thought in the Scientific-Pragmatic quadrant, for example, tend to focus on what worked and did not work with respect to the program being evaluated—a goal consistent with the critical realist pattern of guiding assumptions. The experimental school in the Scientific-Research quadrant tends to focus on finding causal/theoretical explanations for why things did or did not work—a focus consistent with the positivist pattern of guiding assumptions. The schools in the Constructivist-Research quadrant tend to focus on how participants/users/stakeholders felt about and perceived the program being evaluated—a focus consistent with interpretivist/constructivist patterns of guiding assumptions. Finally, the school in the Constructivist/Pragmatic quadrant tends to take a much more participatory ‘how can things be done better’ action research approach to program evaluation—consistent with the participatory inquiry or critical social science pattern of guiding assumptions. As a more systemic

11.1

Choice of Research Frame

355

Research

Positivist

Experimental

Interpretivist/ constructivist Illuminative Goal-free

Scientific

Constructivist Systems Decision making

Interventionist/ Action research

Goal-based Critical realist

Professional review

Participatory Inquiry or Critical social science

Pragmatic Fig. 11.3 Dimensions for differentiating schools of thought for types of evaluation research (adapted from Gray, 2014, p. 304)

alternative to these other schools of thought, Bamberger, Rugh, and Mabry (2012) developed the concept and process of RealWorld Evaluation (RWE) which was intended to focus on evaluations carried out under constraints. They proposed a 7-step integrated evaluation system to produce more credible outcomes: (1) plan and scope the evaluation; (2) address budget constraints; (3) address time constraints; (4) address data constraints; (5) address political constraints; (6) strengthen the evaluation design and validity of the conclusions; and (7) help clients to use the evaluation. Evaluation research, in the context of each school of thought, will tend to utilise a specific subset of data gathering strategies. For example, the scientific-oriented schools will tend to use manipulative (e.g., experiments or quasi-experiments) and measurement (e.g., questionnaires) data gathering strategies (often with a summative evaluation intent), whereas with the constructivist-oriented schools, interaction-based (e.g., interviews), observation-based (e.g., participant observation) or participantcentred (e.g., diaries or journals) strategies will be preferred (often associated with a formative evaluation intent). Pragmatic schools may use a wider variety of data gathering strategies to assemble a better-rounded picture, often favouring the joint gathering of quantitative and qualitative data. Thus, choice of guiding assumptions and preferred data gathering strategies is wide open through this process, depending upon needs and the kinds of research/evaluation questions to be addressed. The process of RWE is largely consistent with the perspective offered in this book with respect to research configuration and planning—making trade-offs that move the research away from what would be ideal to accomplish to what is realistic and feasible to accomplish. Patton (2012) originated the process of utilisation-focused evaluation, which tends to take a more holistic view of evaluation, encompassing not only participants and stakeholders in the program, project or intervention being evaluated, but also intended users as well as the researchers/evaluators themselves. Utilisation-focused

356

11

How Do I Frame and Conceptualise My Research …

evaluation is thus a more systems-oriented evaluation process; one that can draw upon a wide variety of data gathering strategies that yield both quantitative and qualitative data. • Researcher Positioning(s): Your role is as evaluator, conducting your research to draw conclusions and recommendations for decision makers. Your competencies, positioning and credibility have key impacts on the convincingness of an evaluation and the likelihood that decision makers will heed the recommendations you put forward. • Research Context(s): The research context(s) are defined as any context(s) where and circumstances in which a program, treatment or intervention occurs (e.g., a school, a classroom, a hospital or ward, an organisation or workplace, an executive team). • Participants’ Positioning(s): The participants are those who take part in the program, treatment or intervention and may often be self-selected or required (by their organisation, community or institution) to undergo/experience the program or intervention. They constitute one set of primary stakeholders in the research. Other participants may include staff/consultants who conduct the program, treatment or intervention and clients. • Preferred Guiding Assumptions/Data Gathering Strategies: Evaluation research may be guided by any pattern of assumptions (positivist, interpretivist/ constructivist, critical realist, participatory inquiry, critical social science) depending upon what you wish to learn and how deeply you wish your learning to probe (pursuit of depth will be more strongly aligned with an interpretivist/ constructivist or critical realist pattern of guiding assumptions). A wide variety of data gathering strategies may be used for evaluation research, but experience-focused strategies will likely prove useful since a program, treatment or intervention provides a concrete focal experience for participants. For certain purposes, the participatory inquiry pattern of guiding assumptions may prove very useful, especially where formative evaluation is being carried out. • Learning Focus for Research Sponsors/Readers/Users: Primary stakeholders for learning in this regard are those who will make decisions (e.g., adoption, investment, resource allocation, removal) concerning the program, treatment or intervention. The designers and distributors of a program, treatment or intervention will also want to learn whether and how it works and whether modifications will be needed. Utilisation-focused evaluation explicitly focuses on the longer-term utility/viability of a program, treatment or intervention. Downstream potential participants may also be considered as stakeholders who could be impacted, depending upon what decision(s) are made. Overall, the primary learning outcomes emerge from patterns in data that signal effectiveness and/or efficiency of the program, treatment or intervention being evaluated. • Useful References: Gray (2014, Chap. 12) provides a good overall introduction to evaluation research. Patton’s (2012) seminal book provides a complete discussion of his utilisation-focused evaluation perspective. Donaldson, Christie, and Mark (2009) address the issues of quality and credible evidence in evaluation research, noting that what counts as credible evidence depends upon the

11.1

Choice of Research Frame

357

context and stakeholder expectations of the evaluation. Bamberger, Rugh, and Mabry (2012) fully discuss their realistic, practical and systematic RealWorld Evaluation approach, which presumes that evaluation is a constraints-driven process.

11.1.3 Development Evaluation Frame The Developmental Evaluation (DE) frame is most closely associated with Patton (2011) and focuses on innovation (social, technological, policy) development as an adaptive response to dynamically changing contextual circumstances. DE emerged as a research/innovation process that explicitly acknowledged complexity and unpredictability in social circumstances. Patton positioned DE as a type of utilisation-focused evaluation but one where changing situational contexts and needs work to stimulate innovative responses to meet those needs as well as anticipate productive responses to future emergent needs. This means that DE involves a cyclical approach to research coupled with a deep and reflective learning interconnectivity between the researcher/evaluator/consultant, stakeholders, users and innovators. DE research tends to align with interpretivist/constructivist, participatory inquiry or critical realist pattern of guiding assumptions but need not be tied to those patterns of guiding assumptions. DE is explicitly focused on Mode 2 participatory knowledge co-production. DE may employ a wide range of data gathering strategies in pursuit of its purposes. In fact, Patton (2011) employs the metaphor of bricolage to describe the process of DE. Bricolage describes a process of using whatever tools are necessary, even creating new tools, to achieve the deep learning demanded by DE; thus, DE is inherently multi-method/pluralist in its execution. Patton (2011, pp. 261–262) described at least ten different frameworks of inquiry that can shape, focus and direct a DE effort: 1. Basic descriptive framework—understanding the situation as a prelude to positioning innovation. 2. Fundamental evaluative thinking—linking analysis and meaning to innovation and action. 3. Triangulated learning framework—connecting beliefs and knowledge to action through learning; privileging knowledge over beliefs in stimulating innovation and determining actions; 4. Appreciative inquiry framework—learning focusing on strengths and assets in context and how they might be harnessed for innovation and action. 5. Systems change framework—bringing together perspectives and interrelationships to transcend boundaries in pursuit of social innovation and change at a whole-of-system level. 6. Collaborative framework—harnessing collaboration and other forms of social networking to stimulate innovation and determine actions.

358

11

How Do I Frame and Conceptualise My Research …

7. Values-driven inquiry—focuses on how values intersect the issues of innovation in context, in terms of how innovation is accomplished as well as what is accomplished (interest in the means as well as the ends of innovation). 8. Complexity framework—focusing on achieving an understanding of just how complex the situation is that is demanding innovation, where complexity is influenced by how certain or ambiguous cause and effect relationships are in the context and how much conflict surrounds the existence of those relationships. 9. Wicked questions framework—focuses on working through innovation for handling very tough social issues and problems where there are paradoxes and ambiguities, conflicting values and tensions to manage along the way. 10. Actual-Ideal comparative framework—explicitly looks at monitoring the process of change and adaptation over time and evolving situational dynamics in order to learn how the current situation compares to where things started out, whether desired goals were achieved and identifying where to go from here. 11. Any combinations of above—harnessing synergies between frameworks as well as stimulating the emergence of new frameworks. Note that these inquiry frameworks have an increased focus on more complex issues from more comprehensive viewpoints as you work down the list and this will have implications for choice of the most relevant pattern of guiding assumptions to adopt as well as the data gathering strategies to employ and data sources to gain access to/interact with. This increasing complexity and comprehensiveness will generally move you toward a more pluralist research orientation. • Researcher Positioning(s): You undertake the role of a bricoleur or a jack-of all-trades in your approach to DE, accessing the broadest possible toolkit while being open to creating new tools and approaches where necessary. Reflective learning capability is demanded by all involved in DE, since the focus is on understanding, monitoring and modifying innovative adaptations to dynamically changing/evolving situations and circumstances. • Research Context(s): The dynamically changing situational circumstances for a specific social problem defines the research context for DE. • Participants’ Positioning(s): Participants are stakeholders, users and innovators involved in the research context and stand to benefit from as well as contribute to adaptive innovation to address emergent contextual problems. For DE, a deep level of contextual engagement sought as innovation-focused knowledge is co-produced by all involved. • Preferred Guiding Assumptions/Data Gathering Strategies: Preferred patterns of guiding assumptions and data gathering strategies depend heavily upon which inquiry framework you are implementing and the kind of learning that you are pursuing. There is a strong preference for using pluralist approaches in synergistic ways to achieve the kinds of learning and change being pursued. To meet emerging, perhaps unanticipated needs, methodological innovation is highly likely in DE. Developmental evaluation for innovation tends to be inherently participative in nature, so some hybridisation of other patterns of guiding assumptions with the participatory inquiry pattern may be preferable.

11.1

Choice of Research Frame

359

• Learning Focus for Research Sponsors/Readers/Users: The deep engagement demanded of both you, as researcher, and the participants, stakeholders, users and innovators should result in productive and reflective as well as anticipatory learning. This type of learning balances a focus on the past (‘what we did?’) with a focus on the present (‘what is happening now?”) as well as a focus on the future (‘where do we go from here?”). Contextualised adaptation through innovation is the prime goal for reflective learning in DE. Overall, the primary learning outcomes emerge from patterns in data that provide feedback and input into innovation development, modification and adoption/use. • Useful References: Patton (2011) provides a complete discussion of the principles and processes of developmental evaluation. Patton, McKegg, and Wehipeihana (2015) produced a series of cases studies displaying developmental evaluation in action, through various inquiry frameworks. Cooksey (2011) shows how innovation needs to be contextualised through connectivity between innovators and users which is a key thrust of developmental evaluation.

11.1.4 Case Study Research Frame Case study research is a research frame that employs a range of data gathering strategies to achieve the goal of a deep and/or extensive contextual (potentially including historical) understanding of one or more specific phenomena used to define a ‘case’ (which may be construed as an event, a program, a project, an individual, a relationship, a decision, a group, a school, an organisation, a community). Case study research overall, or specific data gathering strategies used within it, can follow either positivist or interpretivist/constructivist assumptions, and, in some instances, may adopt a critical realist, Indigenous, feminist or participatory inquiry stance. The research goal in a case study typically centres on understanding how and why things happen the way they do. A case study may take an inductive theory development trajectory, a deductive theory testing trajectory or a descriptive trajectory (which means that the case study frame is often combined with the explanatory, exploratory or descriptive research frames; see discussions of these frames below). The focus in a case study may be on a single level of analysis (holistic or whole-of-case) or multiple levels of analysis (embedded) and single or multiple cases may be of interest. Multiple case studies will usually be required if you intend to try to generalise across cases or to make explicit cross-case comparisons (identifying commonalities between cases and uniquenesses within each case). In the latter, you will generally have two levels of analyses to conduct and stories to relate: within-case analyses and stories and between-case analyses and stories. Generally speaking, the greater depth of learning you pursue in a case study, the more resource-intensive (in terms of time, money and data gathering energy) your research will be. Most case study research will be implicitly pluralist and involve the use of several data gathering strategies.

360

11

How Do I Frame and Conceptualise My Research …

A case study guided predominantly by the positivist pattern of assumptions would typically employ quantitative data gathering strategies such as questionnaires or structured interviews in the context of sampling within a single focal organisation or institution (yielding a so-called ‘single case study’). Generalisability in a quantitative case study is necessarily highly restricted to the specific case study organisation, but depending upon your sampling scheme, you may be able to generalise within that organisation (possible if the case study configuration is embedded). To overcome this limitation, you could conduct multiple case studies to enhance contextual cross-comparison and generalisability (a strategy called the ‘comparative case method’). The drawback is that such research is more resource-intensive compared to a single case study approach. This means that a multiple quantitative case study approach may be limited by resource constraints and this might force your sample size (in terms of number of case study organisations included) to be much smaller. Even though the primary data you collect may be quantitative, there would still be an expectation that you would weave a contextual story around the data from each case. Such stories would largely be qualitative in emphasis. Alternatively, you could conduct an interpretivist/constructivist case study, where you might employ a range of qualitative data gathering strategies to flesh out a detailed and rich contextual story of the case group, community or organisation (quantitative data might also be used to highlight within case social and/or behavioural patterns). In single qualitative case studies, the goal is to achieve a deep contextual understanding, usually assuming that what is learned may not be transportable to other cases but may be transportable within the case study group or organisation. You could also conduct comparative case studies within an interpretivist/constructivist perspective. However, the resource-intensiveness of multiple qualitative cases is even greater than for multiple quantitative case studies, implying that your choice of cases needs to be very strategic in order to maximise value for learning relative to resources expended. If you adopt critical, Indigenous, feminist or participatory inquiry, your tendency will likely lean toward emphasising qualitative data collection. • Researcher Positioning(s): Your role is as researcher and typically, but not always, you would be an outsider to the case study context. You explicitly define the case context(s) of interest, be it a person, group, program, organisation or community, and define the boundaries of the case (what, who, when and where). • Research Context(s): The case(s) (and perhaps their sub-groups) provide your research contexts. You go into those contexts to conduct your research. A single case study context may be of interest or multiple case study contexts may be pursued. Your research can focus on the case context as a whole (holistic) or on multiple levels or layers (e.g., clients, administrators, doctors, nurses, wards, departments in a hospital, groups, programs, components of programs) embedded within the case context. It is also possible that a single individual or group can provide the context for a case study (e.g., a clinical case study or so-called n = 1 research or ‘single-case design’).

11.1

Choice of Research Frame

361

• Participants’ Positioning(s): Participants are typically those involved in or with the case context in some way. Some data sources will typically be non-human in nature, comprising case-related documents and other artefacts, which can help to flesh out your contextual knowledge. • Preferred Guiding Assumptions/Data Gathering Strategies: Case study research may be guided by one or several patterns of guiding assumptions, depending upon whether your research is intended to be holistic or embedded. The depth of learning desired and the levels of analysis in which data gathering will focus will also influence your choice of guiding assumptions as well as data gathering strategies. Case study research almost always employs a range of data gathering strategies and the Textual artefact-based strategy (e.g., document analysis) usually forms one important data gathering strategy. • Learning Focus for Research Sponsors/Readers/Users: Research questions typically focus on learning how and why things happened in the case context. Learning from case study research may have several different intentions for research users and stakeholders. You may intend to build theory for other researchers (inductive exploratory case study), test pre-existing theory in a specific case context (deductive explanatory case study), evaluate change or evolution over time (longitudinal case study) or simply wish to provide a descriptive account (descriptive case study). In most cases, your goal will be Mode 1 knowledge production and learning outcomes primarily emerge from patterns in data that help to inform and convey the story or stories about the case study context. However, if you are pursuing Mode 2 knowledge production, you will need to adopt a pattern of guiding assumptions consistent with that intent, such as participatory inquiry assumptions, and your approach will need to be much more inclusive of participant input. • Useful References: Gray (2014, Chap. 11) provides a good overall introduction to case study research. Yin (2014) is perhaps the most well-known proponent of case study research and his book provides a comprehensive coverage of case study research, from inception to completion. Dul and Hak (2008) cover case study research from a business perspective and Kazdin (2011) discusses single-case designs (n = 1) research in detail.

11.1.5 Survey Research Frame Survey research is commonly, simplistically and mistakenly thought of as research conducted using questionnaires (hence, the reason many people inappropriately equate the term ‘survey’ with ‘questionnaire’). However, survey research is more properly and holistically conceived as a research frame which reflects a broad-based research approach, most commonly associated with the positivist pattern of guiding assumptions. This means that you predetermine (i.e., control) exactly what you wish to learn from participants and will most often also delimit exactly how participants will convey information to you. In survey research, participants are often referred to as ‘respondents’ to signal that they are responding to questions put to them by you in a prescribed manner (although this trend is slowly changing in

362

11

How Do I Frame and Conceptualise My Research …

favour of ‘participant’ as ‘respondent’ implicitly devalues/dehumanises both the participant him- or herself as well as the information they are providing to you). Survey research seeks a broad-base of learning and knowledge at a more surface level (often through the use of many specific questions about different constructs). Thus, the trade-off decision between breadth and depth of focus favours breadth in survey research. Some depth can be achieved through asking many questions about a specific topic area, but there is often a practical limit to what participants are willing or able to handle. Breadth is generally achieved through access to large representative samples of participants; the larger and more representative the sample the better. The earliest versions of survey research were population censuses conducted by states or countries (these would be characterised as demographic survey research). Such censuses still occur today, but survey research now has a more general remit: to facilitate learning about some population of interest using a large sample of members from that population (where, for a census, the sample equals the population). The kinds of research questions that can be addressed in survey research may focus on what participants do, what they think, feel, expect and/or prefer, what their essential characteristics are and what they know about. Generally, research questions may posit certain patterns of relationships between various constructs (e.g., gender will relate to income level; job satisfaction will relate to salary level or commitment to the organisation) but stop short of pushing for strong causal inferences. This is because, unless you are very careful and have sufficient control over context or your measurement processes, survey research will not be able to convincingly demonstrate cause-effect relationships. Survey research is oriented toward producing Mode 1 knowledge where you, from within your disciplinary background, are the driving the research agenda, often with the goal of dissemination to other researchers. Some survey research may have a more applied focus, where the results are intended to provide information for decision making. However, this would not constitute genuine Mode 2 knowledge production because the role of participants is passive (which is the sense conveyed by the term ‘respondents’) rather than active in the knowledge production process. Survey research generally favours Measurement-data shaping strategies (e.g., self-report questionnaires, objective tests and assessments) and/or structured interviews, that yield quantitative or quantifiable data. For survey research conducted under an interpretivist/constructivist pattern of guiding assumptions (not common, but certainly possible), the Transformative data-shaping strategy and the Textual artefact-based strategy are implemented using open-ended (i.e., free-response) questions. Note that it is possible to include both highly structured questions and open-ended questions—a hybrid strategy often employed in ‘mixed methods’ research. Self-report questionnaires can be administered face-to-face (individually or in a group) or online in a web-based format. These types of questionnaires require a great deal of front-end design work and pilot testing to ensure they function as intended. In these cases, you are completely ‘hands-off’ when it comes time to gather the data, which means the questionnaire must function independently. Survey research that employs structured interviews basically uses

11.1

Choice of Research Frame

363

the interview as a vehicle for the oral delivery of a questionnaire. This requires interviewers to be trained to a high standard of consistency to ensure that errors are not introduced into the data by virtue of the interview situation itself. For specific purposes (e.g., in certain types of marketing research on consumer decision making), it may be possible to embed an experimental manipulation within the questionnaire itself, which effectively pairs the Manipulation experience-focused strategy with the Measurement data-shaping strategy (see Chap. 14). Survey research is typically intended to target large samples and employs statistical procedures designed to facilitate generalisation from sample to population (accompanied by estimated error limits, which get smaller with larger and larger sample sizes; larger samples yield greater precision). In order for statistical generalisation to work effectively, formal probability sampling procedures need to be used, such as random or stratified random sampling (see Chap. 19). The goal is to obtain a sample that is representative of the population in all essential characteristics. In many cases, it may not be feasible or even possible to implement a formal probability sampling scheme, in which case, a non-probability sampling scheme must be used. The price paid for this trade-off is less convincing capacity to generalise. • Researcher Positioning(s): You generally adopt a more hands-off ‘objective’ approach to data gathering, consistent with the positivist pattern of guiding assumptions, even where a structured interactive mode of data gathering is employed. For some modes of questionnaire administration (e.g., mail, online), the measurement instrument or questionnaire takes on the de facto role of ‘researcher’ (albeit an inanimate one). This means that all aspects of the design of the instrument become absolutely critical for you to get right and this will generally require a great deal of your effort (meaning you need to plan for an extensive instrument development period in your project timeline). • Research Context(s): Generally, survey research is done in the field, away from the context where you work. However, what constitutes the field is open and may range from simply the general public (relatively context-free; may even be at home with an online marketing or local government questionnaire) to shoppers intercepted in a shopping centre to employees in an organisation. • Participants’ Positioning(s): In most cases, the participant’s role is as a respondent to predesigned questions (consistent with the positivist pattern of guiding assumptions). However, including some open-ended questions invites the participant to express his or her own views on an issue and this can be a useful strategy for maintaining their interest in completing the questionnaire. The survey population is often identified according to expectations that members have knowledge, preferences or perceptions about some issue, behaviours, context or process that you wish to gain access to through targeting a sample from that population. In other cases, participants may be targeted simply because of who they are and what demographic characteristics they possess (as with a population census for a country). Irrespective of the reasons for inclusion in a sample, participants generally play a more passive role in your research and you will generally be uninterested in what or how they think outside the bounds

364

11

How Do I Frame and Conceptualise My Research …

of the data gathering strategy being employed to conduct the research. Since you determine the nature and content of any questions asked, the only real power a participant has is over whether they will choose to participate and how they will respond. Modern ethical principles dictate that, aside from survey research conducted for mandated organisational, institutional or governmental purposes, participants have the right to make all decisions regarding their participation and their responses. This effectively gives participants the power to influence the nature and composition of your sample, which can then affect your capacity to generalise from sample to population. Furthermore, participants’ will determine for themselves whether to answer specific questions and the more sensitive the information you request, the more likely participants will decline to respond, which has the effect of potentially creating missing data problems for you when it comes time for analysis. • Preferred Guiding Assumptions/Data Gathering Strategies: Survey research generally aligns with the positivist pattern of guiding assumptions, involving the creation and validation of a measurement instrument, be it a questionnaire or structured interview. This means that, in general, Measurement data-shaping strategies are preferred (sometimes coupled with an experience-focused strategy). However, for specific research purposes, you may adopt an interpretivist/ constructivist pattern of guiding assumptions, but this necessitates the use of a less formal instrument, using open-ended questions, instead of rating scales. • Learning Focus for Research Sponsors/Readers/Users: The Survey research frame emphasises breadth of learning (and often generalisability of what is learned) over depth of learning—an orientation generally consistent with positivist guiding assumptions. Such research gains its power for learning from its focus on large representative samples. Executives, managers, educators, government officials and other kinds of decision makers may use the results from survey research to inform their decisions about a specific issue (e.g., public policy, community relationships with local government, abortion, climate change), event (e.g., planned infrastructure development, impending election, recreational activity), process (e.g., financial, hospitality, sales or medical services) or object (e.g., a product, yourself, or another person/people, such as one’s supervisor or work/social network). The most productive learning from survey research concerns distributions of characteristics and patterns of relationships. Survey research, unless very carefully conceived and highly controlled, is a poor research frame for testing causal hypotheses (the explanatory frame, discussed below, is a much stronger frame to work within if strong cause-effect inferences are desired). Population-based survey research such as a census may yield learning that can influence public policy at a national, or possibly international, level. Primary learning outcomes from survey research emerge from patterns in data that are inferred to be present in some population of interest (however widely or narrowly defined). • Useful References: Punch (2003) provides a sound introduction to survey research. Fowler (2014) provides more in-depth coverage of survey research and associated data gathering strategies. Ruel, Wagner, and Gillespie (2015)

11.1

Choice of Research Frame

365

provides a complete review of survey research, its design principles and associated data gathering strategies, including coverage of various modes of survey instrument administration. Krosnick (1999) explores some of the contentious issues associated with survey research and Sue and Ritter (2007) focus their attention on online survey research.

11.1.6 Descriptive Research Frame The Descriptive research frame encompasses any research that purports to describe, summarise, count or otherwise identify, record, analyse and reflect on important characteristics of and relationships between relevant data sources. Most descriptive research is oriented toward Mode 1 knowledge production, producing descriptive stories and/or databases for other researchers to utilise and build upon. Descriptive research may be population-based (i.e., providing census data) or may focus on characterising specific nations, groups, organisations/institutions, events or samples. Typical research questions tend to be of the ‘what’, ‘when’, ‘where’, ‘who’ and/or ‘how’ types (e.g., What is observed? What is happening and where? What is the situation? Who lives/works/studies here? How is this outcome being achieved? When does/did event X occur?) rather than seeking deeper meaning and/or patterns of causal relationships. Some types of descriptive research may provide data for secondary databases (for example, a financial database (e.g., https://www.globalfinancialdata. com/); an employment database (e.g., http://www.oecd.org/employment/onlineoecd employmentdatabase.htm); census database (e.g., http://www.abs.gov.au/); or an institutional database (e.g., https://www.universitiesaustralia.edu.au/australiasuniversities/key-facts-and-data#.Vi2WXH4rLVY)) with the data gathering exercise often being mandated by government or industry initiative. Such descriptive databases then can be accessed by researchers for other purposes and types of research. Descriptive secondary databases may also be used to inform the creation/modification of social, institutional and legislative policies. Thus, in many cases, the production of a descriptive secondary database is done to meet the needs of stakeholders other than the researchers who design, gather and record the data in the database. Descriptive research generally pursues breadth rather than depth of learning and may conducted under a wide range of patterns of guiding assumptions, yielding quantitative and/or qualitative data. The general intention is to create and display a ‘picture’ or story about some context (and participants within that context). Descriptive research, on its own, cannot explain what is observed and/or recorded, it only notes what things are observed and/or recorded. In some cases, descriptive research may only be a part of your goal, especially if your research is being guided by an interpretivist/constructivist pattern of assumptions. Here, description may be one goal of your research, namely to establish observational and interpretational context, before proceeding to or as a part of exploratory or explanatory research (another signal that research frames can be blended or hybridised). For example, a descriptive ethnography has a goal to produce what is called ‘thick description’,

366

11

How Do I Frame and Conceptualise My Research …

which is description coupled with emerging interpretive work being associated with what is being observed. • Researcher Positioning(s): Your role often tends to be more observational than interactional in nature, especially if your research is guided by the positivist pattern of assumptions. You are there to record what is observed rather than to interpret what is observed. In some cases, you may be contextually embedded with the goal of providing a descriptive story of some research context (as with a descriptive ethnography or case study guided by an interpretivist/constructivist pattern of assumptions). • Research Context(s): The research context can be widely varying for this research frame, but generally tends toward naturalistic contexts, away from your context. For descriptive research feeding into a secondary database, your research context tends to be more broadly defined, say at an industry or institutional level of analysis and the data gathered tend to be aggregated to that level. • Participants’ Positioning(s): For research guided by the positivist pattern of guiding assumptions, participants tend to be respondents to pre-designed descriptive questions. Under other patterns of guiding assumptions, participants may be more centrally involved with you, where your research seeks to produce a descriptive and contextualised story. For descriptive research feeding into a secondary database, data sources will tend not to be individual participants, but a particular individual who produces and/or reports data aggregated across many individuals and/or groups within a context of interest. • Preferred Guiding Assumptions/Data Gathering Strategies: The majority of purely descriptive research tends to be guided by positivist assumptions, gathering quantitative data for reporting descriptive patterns of characteristics. Data gathering strategies in such research will tend to be measurement-oriented, systematic observation and/or structured interviews yielding quantitativelycoded data. If your research is guided by an interpretivist/constructivist pattern of assumptions, qualitative data will tend to be gathered via interaction-based and/or observation-based data gathering strategies. Here is where ethnographic research is typically situated, for example. • Learning Focus for Research Sponsors/Readers/Users: Descriptive research is often conducted to provide information for decision makers and policy makers. Learning for such stakeholders will hopefully translate into meaningful decisions and policies. As well, descriptive research, and especially research that produces secondary databases, will provide data for other researchers to work with. Learning here will emerge from the research conducted using the secondary database (e.g., a financial or economic database) and will generally concern research questions that were not the focus of the originator(s) of the database. Primary learning outcomes emerge from patterns in data that help build up a coherent picture or narrative about some context or population of interest. • Useful References: Babbie (2011, pp. 95–96) gives a brief but very clear discussion of the nature and intentions of descriptive research. Gray (2014, pp. 36– 37 and 56–57) provides further details on descriptive research, distinguishing it

11.1

Choice of Research Frame

367

from exploratory and explanatory research. Tripodi and Bender (2010) discuss, in detail, descriptive research in the context of social work. Mittal (2011) reviews descriptive research conducted in marketing contexts. Lambert and Lambert (2012) discuss qualitative descriptive research in nursing contexts.

11.1.7 Exploratory Research Frame The Exploratory Research frame encompasses preliminary research into new fields of investigation, situations and/or contexts (so-called ‘greenfields’ research). Of the three more general research frames of descriptive, exploratory and explanatory research, exploratory research can be considered to be the frame most open to building knowledge in new areas and new ways. Depending upon your research goals, you might pursue either Mode 1 or Mode 2 knowledge production; the latter, of course, engaging much more participatory input as well as demanding much more of your engagement with the context in order to build up perspectives on the issues being explored and to determine the needs to be met by further research. Exploratory research addresses many of the same types of research questions as descriptive research, but with a more explicit goal of building knowledge to stimulate, guide and provide input to further research. Theories and hypotheses are typically not tested in exploratory research. However, some ways of doing exploratory research may lead to preliminary theoretical ideas and hypotheses, reflecting a theory-building emphasis (e.g., providing initial inputs into grounded theory development) consistent with Mode 1 knowledge production. In such cases, research questions that ask ‘why’ things are happening become important to address. Exploratory research may be conducted under a wide range of patterns of guiding assumptions, but in some disciplines, such as marketing research, may be guided by more interpretivist/constructivist patterns of assumptions in early phases of the research. This signals that many researchers use exploratory research early in their research investigations to gain a deeper understanding of contexts and issues prior to embarking on their main research phase. Exploratory research tends to reflect more depth of focus than breadth of focus, since the learning achieved tends to open up more questions than it resolves and generalisable or transportable findings are not being pursued. In fact, one of the distinguishing features of the exploratory research frame is that it is oriented toward opening up lines of investigation, rather than trying to come to a definitive answer about any context or issue. • Researcher Positioning(s): Your role is generally to be a learner, to engage at some level with a new research context of interest with a view to learning about that context and what might be useful to take forward as input into further research. You generally have a higher level of uncertainty with respect to what might be learned within this research frame—surprises may lie around every corner.

368

11

How Do I Frame and Conceptualise My Research …

• Research Context(s): Any context may be useful in exploratory research, but often those contexts closer to where potential participants live, study and/or and work will be more productive. In many cases, your research context will not have been the previous target of research, which creates the need for exploration. • Participants’ Positioning(s): Participants often take on the more active role of helping you to learn about the research context, what the issues are, what is important and so on. Here, interpretivist/constructivist guiding assumptions may be the most useful to adopt. In exploratory research, the researcher-participant relationship is somewhat closer than for descriptive research and, in some cases, even explanatory research. Under the positivist pattern of guiding assumptions, the participant takes a more passive role, but one where you cannot anticipate all that might be learned. • Preferred Guiding Assumptions/Data Gathering Strategies: A wide range of patterns of guiding assumptions are possible to adopt, but often interpretivist/ constructivist or participatory inquiry assumptions tend to be preferred. Exploratory research tends to work best with interaction-based, observationbased or artefact-based data gathering strategies. In sequential research configurations (to be discussed in Chap. 12), exploratory research often occupies the first phase of the research, providing input into subsequent phases. • Learning Focus for Research Sponsors/Readers/Users: Exploratory research can be an end in itself by providing other researchers with signals for new and further directions for research, preliminary ideas for theorising as well as input into how to more effectively configure further research. You may be learning in order to satisfy your curiosity or to determine what might be feasible to do in further research within a specific context including learning how best to shape the next phase of your research. Primary learning outcomes for exploratory research emerge from patterns in data that provide signals and directions for future research endeavours as well as help to build up a coherent picture or narrative about some context of interest. • Useful References: Babbie (2011 pp. 95–96) provides a brief but very clear discussion of the nature and intentions of exploratory research. Davies (2006) gives a useful overview of exploratory research and Stebbins (2001) provides a self-contained discussion of exploratory research in the social sciences.

11.1.8 Explanatory Research Frame The Explanatory Research frame encompasses any research that has the goal of explaining and/or theorising why things happen. Explanatory research chiefly pursues Mode 1 knowledge production, seeking and testing theoretical accounts for events. It has a strong association with the positivist pattern of guiding assumptions where your goal is to test deductive causal and theoretical propositions, hypotheses or predictive relationships then make inductive inferences back to the originating theory. Explanatory research also has strong ties to the Measurement data shaping strategy and experience-focused data gathering strategies, because quantitative data

11.1

Choice of Research Frame

369

are often pursued within this frame. Under the positivist pattern of guiding assumptions, explanation equates to understanding the theorised causal linkages between constructs and perhaps making predictions of future behaviours based on those theorised linkages (making predictions is one manifestation of generalisation). To do this convincingly, (1) causal factors must be observed before effects are observed, (2) effects must change if their putative cause(s) change and, importantly, (3) alternative plausible causes for observed effects must be ruled out (via control procedures). The stronger the causal inferences intended, the more tightly controlled your research context(s) must be and the more your research will be focused on breadth of learning (i.e., the pursuit of generalisation) rather than depth of learning. There are, however, other perspectives on what counts as explanatory research. For example, explanatory research also encompasses inductive theory-building approaches such as grounded theory, which tend to be guided by interpretivist/ constructivist patterns of assumptions. In this case, theoretical explanations and accounts emerge from the data instead of being proposed by you prior to your research being conducted. Furthermore, emerging theoretical propositions can be tested by gathering more contextualised data. Here, your focus is more on depth of learning (i.e., immersion in context) rather than breadth of learning. Explanatory research may also be conducted under critical realist assumptions, where your goal may be to achieve an understanding not only of external causal patterns but also of people’s perceptions and understandings of those patterns. In this case, depth and breadth foci tend to be more balanced. It is fair to say that, of all the possible ways of framing social and behavioural research, the explanatory research frame is the most sensitive to and most strongly shaped by the pattern of guiding assumptions you adopt. This is largely attributable to the differing roles of and perspectives on causality and theory within the different patterns. • Researcher Positioning(s): Your role in the explanatory research frame will be heavily dependent upon the pattern of guiding assumptions you adopt. Under the positivist pattern of guiding assumptions, your role will involve prior theorising about what you hope to find coupled with maintaining a more ‘objective’ distance from what you are observing so you do not contaminate the data being gathered. Under an interpretivist/constructivist pattern of guiding assumptions, your role will become embedded in whist maintaining a ‘researcher attitude’ in the context of research, in as far as it is possible and feasible to do so. Here, you pursue subjective perspectives that are not your own in order to build up a theoretical account for those perspectives. Under critical realist assumptions, you may adopt a more balanced stance, to critically examine both internal (within participants) and external (in the world inhabited by the participants but distinct from any particular participant’s viewpoint) perspectives. • Research Context(s): The research context for the explanatory research frame may range from a highly controllable experimental laboratory to a naturalistic field setting. Contextual choice is pivotally connected to the pattern of guiding assumptions adopted. Under positivist guiding assumptions, your capacity to influence or manipulate what happens in the context will determine choices.

370

•

•

•

•

11

How Do I Frame and Conceptualise My Research …

Under a non-positivist pattern of guiding assumptions, you will tend to prefer naturalistic settings, where you have minimal capacity to influence what happens. Participants’ Positioning(s): The positioning of participants, in terms of their role from your perspective, is also critically dependent upon the pattern of guiding assumptions you adopt. Under the positivist pattern of guiding assumptions, the larger context(s) and life histories of participants are largely irrelevant; participants are simply inter-substitutable data sources whose idiosyncratic differences can hopefully be assumed to contribute nothing more to your research than random errors. Under non-positivist patterns of guiding assumptions, who participants are, where they have come from and the role(s) they may play in the context(s) of your research all become relevant and important for you to learn about. Participants thus become a primary source of information for your emergent theorising. Preferred Guiding Assumptions/Data Gathering Strategies: As we have seen above, a wide range of patterns of guiding assumptions can be accommodated in the explanatory research frame, but each has a critical influence on the choice of data gathering and sampling strategies employed. How open you will be to employing a diverse range of data gathering strategies will also depend upon the pattern of guiding assumptions adopted. In particular, the more strongly you intend to make external causal and generalisable inferences, under the positivist pattern of guiding assumptions, the less open you will tend to be about choice of data gathering strategies, favouring, in particular, experience-focused and data-shaping strategies. Learning Focus for Research Sponsors/Readers/Users: Most explanatory research tends to engage in Mode 1 knowledge production, trying to add to the disciplinary core of knowledge about social and behavioural phenomena, often through publication in peer-reviewed academic journals. Thus, other researchers are the primary learners from such research and that learning may influence what they investigate and how in the future. Explanatory research tends not to ascribe high value to application of what is learned to concrete problems and practices, except in so far as such applications may be envisaged as emerging downstream or where the explicit intent is to construct and test predictive forecasting or decision-making models (in the economic or financial disciplines, for example). Primary learning outcomes emerge from patterns in data that signal defensible inferences about causes and effects or that provide coherent and consistent accounts for what has been observed. Useful References: Babbie (2011 pp. 97–101) provides a very concise introduction to the nature and intentions of explanatory research. Cook, Campbell and Peracchio (1990) provide an overview to the logic of conducting explanatory research in the form of experiments and, especially, quasi-experiments. The emphasis here is on explanatory research from a positivist perspective, where you try to maximise the control you have over context, whether in the laboratory or in the field. Bryant and Charmaz (2007) compile an excellent of collection of contributions that explore issues and considerations associated with grounded theory and grounded theorising as an interpretivist/constructivist approach to explanatory research.

11.1

Choice of Research Frame

371

11.1.9 Cross-Cultural (Cross-National or Comparative) Research Frame The cross-cultural research frame encompasses any research that crosses cultural boundaries, either to make direct comparisons between two or more cultural contexts (comparative orientation) or where a member of one cultural context is doing research in another cultural context (cross-cultural learning orientation). The cross-cultural research frame is also associated with the term cross-national research and, when different cultures are being explicitly compared, comparative research. In the comparative approach to cross-cultural research, you are looking for patterns, similarities, differences and uniqueness across the cultural groups you are interested in (e.g., comparing Southeast Asian and Middle Eastern cultures with respect to their attitudes toward risk taking in business). If the comparative inferences are intended to be relevant to or generalised to the national level (e.g., to Vietnam and Saudi Arabia), this gives rise to cross-national research. A fair proportion of comparative cross-cultural research is conducted under the positivist pattern of guiding assumptions, which requires you to be very careful about defining and measuring constructs so as to ensure that they have the same meaning (connotations and denotations) across all the cultures you are comparing. Data gathering strategies, in particular those that pursue connecting with people and those that provide structured experiences for people, have to be tailored to meet the needs, expectations and conventions of all cultural groups to be compared. For example, cross-gender interviews are inappropriate in many Middle Eastern cultures. Eye contact or shaking hands may be appropriate in one cultural group, but not another. Direct questioning may work in some cultural groups, whereas indirect questioning may be more appropriate in others. Cross-cultural questionnaires are a very common data gathering strategy employed in comparative research. Here the cross-cultural research frame intersects the survey research frame to gain the efficiency and breadth of coverage in data gathering required. However, the drawback is that the preparation time for such research is usually massive and very time/ labour-intensive, often requiring several preliminary data gathering and pilot testing stages. Constructing and demonstrating the equivalence between the forms of a questionnaire for different cultural groups is a difficult task to accomplish in a convincing manner. In some cases, the use of specific language concepts to write questions can be vexing if the concepts have different meanings or cultural sensitivities (e.g., questions about personal and societal values, power relationships in social and/or work groups, family relationships, sexuality, religion, ethnic relationships and so on are implicated here). In a cross-cultural learning approach, you are looking to understand behavioural and social patterns within another cultural group, of which you are not a member (e.g., an Australian researcher studying participation in higher education in the Middle East). A good deal of anthropological research has this intention, for example. This is outsider research requiring you to remain value-neutral, non-judgmental, culturally sensitive and open-minded about what you observe

372

11

How Do I Frame and Conceptualise My Research …

within that cultural group. Here, interpretivist/constructivist guiding assumptions may better suit your research purposes and judicious maintenance of a research journal must be one of the strategies you employ. Cross-cultural learning research will likely require much longer timelines than research conducted within other frames. Such learning takes time and some degree of immersion in the cultural context being studied, in order to maximise the depth and value of the learning achieved. Your outsider status will necessitate constant attention to potential power and knowledge sharing issues, especially if you are gaining access to more privileged types of knowledge within the cultural group being you are studying. Additionally, you must be extra cautious not to fall into the trap of ‘going native’, which means gradually losing your perspective as a researcher and moving more closely toward adopting and reflecting the perspective(s) of those in the cultural group you are researching. If ‘going native’ occurs, your research may suffer irreparable damage to convincingness. Thus, you must take steps to minimise the risk of this happening, which adds to the overall complexity of your research project. It is possible to combine the cross-cultural learning approach with the comparative approach in instances where you want to compare cultural groups and you are a member of one of the cultural groups involved in the comparisons. In this case, the complexity of the research takes a quantum leap because you must manage to offset the potential problems associated with both cross-cultural comparison and cross-cultural learning. • Researcher Positioning(s): You take on an outsider role if you are gathering data from participants from a sample whose ethnic/national/cultural background differs from yours. Here you must take steps to ensure you have sufficient understanding of the cultural group and how to behave within it to facilitate effective and meaningful data gathering and this may require one or more preliminary stages in the research. You take on an insider role if you are gathering data from members of your own cultural group. Much more cultural and local knowledge can be taken for granted in this situation, but care must be taken when you then make comparisons with other cultural groups. The issue here is ensuring, as far as possible, conceptual and contextual equivalence where comparisons are being made. • Research Context(s): Any research context may potentially be involved in cross-cultural research depending upon the nature and sampling of the participants. For some types of cross-cultural research, you may be a visitor in the home nation of participants (as, for example, when an Australian researcher does research in China with Chinese students as participants). For other types (especially studies with comparative intentions), a cross-cultural research team may be involved, where each team member gathers data from a sample of their own national group, either in a laboratory or a field setting. • Participants’ Positioning(s): Understanding the positioning of any participants in cross-cultural research demands that you acquire and enact a deeper understanding of the cultural context and backgrounds of those participants. An insider researcher might have an easier time here. In cross-cultural research,

11.1

Choice of Research Frame

373

there is always the risk that unintentional, possibly negative, power dynamics may influence the conduct of the research and the quality of the data obtained, especially where there has been a past history of such dynamics between the culture you come from (e.g., a researcher from the US) and the cultures that participants come from (e.g., participants from Iraq or Cuba). This risk is at its most potent in Indigenous research, which is one reason why Indigenous research is considered to be a distinct research frame (see discussion below). The risk is also potent if the cross-cultural research also crosses religious boundaries (as, for example, when a Christian researcher attempts to do research in a predominantly Islamic nation). It is important to note that cross-cultural research may be essentially undertaken within the boundaries of a single nation, if that nation is itself multicultural in composition and that composition is reflected in the sample or if the nation has or has had strong historical and/or political divides between geographic areas (such as between northern and southern states in the US) or demographic groups (e.g., a dominant middle- or upper-class group and poor minority groups). • Preferred Guiding Assumptions/Data Gathering Strategies: Cross-cultural research may be undertaken under the guidance of a wide variety of patterns of assumptions. Where the positivist pattern of assumptions is adopted, your research will require close attention to the design and pretesting of any measurement instruments/questionnaires. This will normally require additional preliminary stages in your research. Where an interpretivist/constructivist pattern of guiding assumptions is adopted, you will often need to spend more time in context in order to achieve meaningful interpretations of what you hear and see (participant observation is a common data gathering strategy here). If a critical social science or Indigenous pattern of guiding assumptions is adopted, you will need to spend time trying to understand the political, perhaps oppressive contexts, the participants have or may be experiencing before attempting any critique. The most dangerous thing you can do, as a researcher in this situation, is to unilaterally impose you own views, values, ideology on what you are looking for, seeing and hearing. A more interpretivist approach to gathering qualitative data will almost always be demanded. • Learning Focus for Research Sponsors/Readers/Users: Primary learning outcomes from cross-cultural learning research where researcher and researched come from different cultural backgrounds emerge from patterns in data that help you gain/achieve insights into/understanding of life/work/study/relationships and so on in the cultural group you are researching. Primary learning outcomes from cross-cultural research that has an explicit comparative intent (e.g., cross-national research) emerge from patterns in data that effectively and convincingly reveal commonalities between and uniquenesses within different cultural groups. In particular, highlighting commonalities in cross-cultural patterns feeds learning that reinforces what may be generalisable across cultural boundaries (i.e., reinforcing and valuing connectedness). Highlighting contrasting cross-cultural patterns feeds learning that signals diversity and perhaps the perils of assuming commonality (i.e., reinforcing and valuing

374

11

How Do I Frame and Conceptualise My Research …

distinctiveness). Both directions for learning may provide concrete guidance for future actions and applications of knowledge. • Useful References: Ember and Ember (2009) provide a general introduction to cross-cultural research methods. Liamputtong (2010) discusses qualitative cross-cultural approaches in some detail. De Vaus (2008) reviews research designs for comparative and cross-national research. Harkness et al. (2010) discuss the ins and outs of implementing the survey research frame in multicultural and multinational contexts (a hybrid combination of research frames).

11.1.10

Indigenous (Indigenist) Research Frame

Indigenous research is a more recent evolution of cross-cultural research. Indigenous cultures encompass a myriad of Australian Aboriginal and Torres Strait Islander cultures, New Zealand Māori cultures, Inuit cultures in Canada, Native American cultures in the North, Central and South Americas, African cultures, Pacific Islander cultures, Southeast Asian cultures and so on. Indigenous research refers to the development of knowledge of and about a specific Indigenous culture in its native context(s). Such research tends to be more strongly aligned with interpretivist/constructivist patterns of guiding assumptions, with a strong critical/ emancipatory/empowerment orientation. Because of the unique nature of traditions associated with how Indigenous knowledge is acquired, accumulated, shared and ‘owned’ by the Indigenous communities, Indigenous knowledge production is not consistent with either of the western concepts of Mode 1 or Mode 2 knowledge production. Instead, Indigenous research tends to be undertaken from a critical stance that is anchored in the recognition that the colonisation of Indigenous nations and cultures (including the non-Indigenous research practices associated with the colonising culture) has led to Indigenous knowledge being stolen, co-opted, usurped, suppressed or hidden, distorted or completely subsumed or destroyed by the colonising culture. Part of the role of Indigenous research is to recover, privilege and preserve the content, power, strength and ownership of Indigenous knowledge where it resides. Another important purpose for Indigenous research is the co-creation and sharing of knowledge about Indigenous lifespaces, social mores, cultures, experiences and worldviews with wider and likely non-Indigenous audiences to foster learning about, respect for and understanding of Indigenous ways of knowing, ways of being and ways of doing (Martin & Mirraboopa, 2003). Given the uniqueness of the human systems and spaces within which Indigenous research operates, distinct sets of ethical principles for guiding such research have been devised in several countries. Two examples: in Australia, there are the Guidelines for Ethical Research in Australian Indigenous Studies established by the Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS; see http://aiatsis.gov.au/research/ethical-research/guidelinesethical-research-australian-indigenous-studies); in Canada, there is Chapter 9 of the Tri-Council Policy Statement (as interpreted by the Panel on Research Ethics of the

11.1

Choice of Research Frame

375

Canadian Government), entitled Research Involving the First Nations, Inuit and Métis Peoples of Canada (http://www.pre.ethics.gc.ca/eng/policy-politique/ initiatives/tcps2-eptc2/chapter9-chapitre9/). We can identify two different approaches for conducting Indigenous research. The first approach involves an Indigenous person studying his or her own culture or another Indigenous culture (this appears to be more consistent with Martin and Mirraboopa’s (2003) Indigenist research concept). If this describes your situation, this approach would be characterised as ‘insider’ research in that you share at least some commonalities with the Indigenous cultures you are doing research with. The second approach involves a non-Indigenous person studying an Indigenous culture, thereby constituting ‘outsider’ research. If this describes your situation, this approach is much riskier and likely to be much less revealing and trusting if you are working on your own, because of its negative association with historically oppressive colonisation effects, especially if you are a member of the oppressing culture (e.g., a white Australian university researcher studying aspects of the Anaiwan Indigenous cultural context in the New England region of New South Wales). The ‘outsider’ approach has a better chance of success if you work collaboratively with a team that involves Indigenous people from the cultural group that is being studied, but it may mean that you will need to set your own predilections and preferences aside in favour of Indigenous traditions. Nicholls (2009) argued that this is possible through a multi-layered process of self, inter-personal and collective reflexivity embodied in a participatory methodology that works to build and maintain relationships and attendant social obligations. Furthermore, such ‘outsider’ research will have a much longer timeline, simply because it will take that much longer to build up trusting relationships within the cultural group you want to do research with. What is learned in the Indigenous research frame needs to be carefully negotiated and knowledge sharing and dissemination carefully and sensitively managed, with continual attention to what that learning and sharing brings back to the Indigenous community (i.e., following the principle of reciprocity). • Researcher Positioning(s): Researcher positioning may emerge from one of three specific places: (1) insider researcher whose background and experiences are congruent with/same as the Indigenous background of participants (e.g., research in or from the same regional group or language background); (2) insider researcher whose background is Indigenous but whose experiences are incongruent/not same as the Indigenous background of participants (e.g., from a different regional group or language background); or (3) outsider researcher who comes from a non-Indigenous background. • Research Context(s): Potential contexts for Indigenous research include (a) natural Indigenous contexts inhabited by Indigenous participants or (b) non-Indigenous natural contexts with Indigenous inhabitants or where Indigenous and non-Indigenous people are co-mingled. The (a) contexts may provide insights into the ‘within-culture’ aspects of Indigenous life as well as into tensions and adaptations that emerge as those contexts interact with/respond to outside pressures whereas the (b) contexts may provide insights into

376

•

•

•

•

11

How Do I Frame and Conceptualise My Research …

‘between-culture’ as well as ‘within-culture’ tensions and adaptations that occur when Indigenous and non-Indigenous people live together. Participants’ Positioning(s): Participants may come from shared or different Indigenous backgrounds. They may occupy traditional knowledge keeping/ spiritual/elder/gender/social or other roles within their Indigenous group or society. They may also occupy roles within non-Indigenous society and this may, in fact, be a point of focus for the research (e.g., Indigenous employees of a university; Indigenous members of parliament or local government). Preferred Guiding Assumptions/Data Gathering Strategies: Qualitative data gathering strategies with a strongly interpretivist/constructivist bent (often with a critical orientation) are typically preferred, although there have been some recent developments with respect to Indigenous quantitative data gathering strategies (see Walter & Anderson, 2013). Thus, there is a strong preference for data gathering strategies that connect with people (e.g., semi-structured or unstructured interviews; participant observation) or explore the handiworks produced by people (e.g., stories; oral histories; rituals and symbols). The strong interpretivist/constructivist leaning usually provides a better fit to the research as subjective world views and the roles of language, symbols, rituals and relationships become more prominent foci for research. In some cases, this interpretivist/constructivist leaning will have overtones of participatory inquiry as well (where researcher and participants jointly engage in knowledge sharing). Indigenous research may adopt a critical orientation when social/political/ spiritual emancipation from colonisation effects, pressures and power relationships (including how knowledge itself is acquired and shared) become an important focus. Here, the Indigenous research frame shares some common agenda orientations with the Feminist research frame (to be discussed below). Learning Focus for Research Sponsors/Readers/Users: Primary learning outcomes for non-Indigenous sponsors/readers/users emerge from patterns in data that provide a strong cultural understanding or highlight social justice/equity/ identity/social policy issues. Primary learning outcomes for Indigenous sponsors/readers/users emerge from patterns in data oriented toward cultural development/preservation, conflict management, political action or consensusbuilding. Primary learning outcomes for a mixed audience of Indigenous/ non-Indigenous sponsors/readers/users emerge from patterns in data oriented toward displaying a strong cultural understanding (especially with respect to Indigenous-non-Indigenous relations), policy development, reconciliation, conflict management or social equity. Useful References: Smith (2012) is an important text that sets forth an agenda for Indigenous research methodology which ‘de-colonises’ the typical non-Indigenous research approaches. Chilisa (2012) is a general text that provides an excellent introduction to a diverse range of Indigenous research methods. Kovach’s (2009) text addresses issues associated with Indigenous research methodologies, including the issues of ethics and reciprocity. Walter and Andersen (2013) develop a clear and highly relevant discussion of an Indigenous quantitative methodology.

11.1

Choice of Research Frame

11.1.11

377

Feminist Research Frame

The Feminist research frame focuses on the gendering of experiences for men and women in society, communities, organisations, schools and families and the power, political, social, educational and behavioural implications of those experiences. Feminist research is typically aligned with critical social science guiding assumptions where there is a presumption of unequal power relationships between men and women (as well as other marginalised groups, where relevant), which needs to be understood, exposed and undone/rectified/transformed. This is an explicitly emancipatory goal, aimed at countering the ‘masculinised’ or ‘androcentric’ knowledge produced by typical social science research approaches. Many feminist researchers have rejected objectivity, the predominantly masculine-orientation and implied power relationships embodied in the positivist pattern of guiding assumptions and with this has generally come a rejection of quantitative measurement and associated research methods. More recently, however, this rejection has been seen as imposing too many constraints on the possibilities for feminist research and this has produced a movement back to a position where fitness to research questions has become a more important criterion for methodological choices. In that light, transparency in methodological choices becomes very important for feminist researchers to demonstrate. Inherent subjectivity and biases in knowledge production must be acknowledged and understood which means that feminist research must therefore be considered as value-laden rather than value-free (Letherby, 2011). Thus, interpretivist/constructivist guiding assumptions coupled with a critical attitude and qualitative data gathering strategies are considered to be more appropriate for accessing the authentic subjective worldviews, experiences and stories of women (and, where relevant, of men). In some ways, the term ‘feminist research’ can be overly constraining and even misleading. The Feminist research frame need not be concerned solely with gendered knowledge creation and transformative change. It may also be concerned with creating knowledge and fostering change from the perspectives of other marginalised groups. For example, queer research (e.g., Browne & Nash, 2010) can be seen as a variant of the Feminist research frame where the focus shifts to the sexual (as distinct from gendered) lives and experiences of lesbian, gay, bisexual, transgender and queer people (LGBTQ+ , where the + can be read to include types of identities such as intersex, asexual, pansexual, polyamorous). The goals of such research have the same critical emancipatory emphasis as feminist research but focusing instead the differing authentic worldviews and experiences, oppression and injustices associated with claiming to be of one or more LGBTQ+ orientations in a non-LGBTQ+ world. Note the deliberate use of the research quality criterion of authenticity in this discussion. Authenticity in the voices being sought and heard is extremely important in all variants of the Feminist research frame. This places a great burden of responsibility upon you, as the researcher, to ensure that any data gathered are authentic with respect to the voices of those who provided the data. You are not trying to tell your own story but those of the participants in your

378

11

How Do I Frame and Conceptualise My Research …

research (acknowledging, of course, that your own story may interact or resonate with participants’ stories). • Researcher Positioning(s): Feminist researchers frequently adopt a critical stance from the outset. The goal of many feminist researchers is to conduct research and produce knowledge for women as pathways for transformation and change rather than just about women (an emancipatory goal, see the discussion in Letherby, 2011, pp. 64–65). Feminist researchers will typically, but not always, be female, thus sharing a common cultural and perhaps experiential basis with their participants or with females that are the focus of the behaviours of participants within their larger societal context. For some feminist researchers, other characteristics besides gender (such as sexual orientation/identity, disability or ethnicity), which may be associated with marginalised groups, are also of interest. It is here where the label ‘feminist’ research may be seen as artificially limiting in focus. • Research Context(s): Feminist research tends to unfold within the natural experiential contexts of the research participants, who often tend to be women (but may be men as well). Since some feminist researchers are open to other characteristics, perceptions and sources of identity that are associated with marginalised groups, this may mean gaining access to the experiential contexts for those marginalised groups (e.g., lesbian, gay, queer, bisexual, intersex, transgender, disabled, poverty-stricken, homeless). • Participants’ Positioning(s): The positioning of women participating in feminist research may actually be multidimensional in terms of roles: the role(s) they occupy as currently defined and shaped by men and the roles they define and shape for themselves. Thus, a female on a Board of Directors may be positioned as a Board member, defined predominantly by males; a Board member by virtue of the enactment of an affirmative action policy; or as a woman who has successfully navigated the white waters of the male-dominated business culture and risen to the top. In fact, the intention of feminist research might be to explore and understand the tensions between these various roles with a view toward correcting imbalances in power and perception. Men may also participate in feminist research, but typically have done so where comparing and contrasting perceptions and experiences is an important focus for the research. Letherby (2011), however, argues that extending feminist research to be inclusive of men’s experiences must become an important goal if a more balanced view of the gendering of social and life experiences is to be achieved. In many cases, research that includes perspectives from both marginalised and non-marginalised groups of participants (however defined) can add value to knowledge and impetus for social action. • Preferred Guiding Assumptions/Data Gathering Strategies: In feminist research, qualitative data gathering strategies have been strongly preferred associated with strong interpretivist/constructivist leanings. Thus, there is a strong preference for data gathering strategies that connect with people (e.g., semi-structured or unstructured interviews; participant observation) or explore

11.1

Choice of Research Frame

379

the handiworks produced by people (e.g., diaries/journals; texts and multimedia artefacts). Positivist guiding assumptions and quantitative measurement are generally held to be incongruent with a feminist perspective, unless the perspective is radically modified toward inclusiveness of feminine experiences and constructs. Some feminist researchers do value the ‘objectivity’, external reality focus and value-neutrality of the positivist pattern of guiding assumptions and its association with quantitative measurement data, but only where the female experience is taken into account on an equal basis with male experience. More recently, the character of ‘objectivity’ associated with quantitative research has given ground to the possibility that quantitative approaches may be revealing in their own ways—the choice of data type being made on the basis of fitness to the research question at hand (Letherby, 2011). • Learning Focus for Research Sponsors/Readers/Users: Primary learning outcomes for male sponsors/readers/users emerge from patterns in data that open up new perspectives and interpretations of experiences, perhaps leading to transformative change through unfreezing entrenched attitudes and patterns of experience. This may require retelling of stories in ways that will enhance the chances of convincing those who may have previously not shared such awareness by creating/stimulating awareness for why action and change might be needed. Primary learning outcomes for female sponsors/readers/users emerge from patterns in data that generate new pathways and leverage for transformational and/or political change, build leverage for unfreezing entrenched attitudes and patterns of experience and relationships, create political leverage for rebalancing/rectifying inter-gender relationships (e.g., enriching the reasoning and logic of the convinced or moving those prepared to be convinced toward being convinced, perhaps sufficiently enough to stimulate action and change). Primary learning outcomes for mixed gender audiences emerge from patterns in data that help to share, compare and contrast perspectives and interpretations of experiences, create leverage to help the ‘converted’ and ‘unconverted’ move closer together in perspectives to achieve a common vision for action and change. This is the most difficult audience to reach because of the great care needed to craft a story that speaks to both sides of any issue. • Useful References: Hesse-Biber (2014) provides a comprehensive primer on feminist research practices, including coverage of feminist perspectives on a range of data gathering strategies, such as in-depth interviews, focus groups, media research and survey research. Hesse-Biber (2012) provides a higher-level, more probing and more action/change oriented set of contributions that highlight the implications that a feminist perspective might have for research ethics, knowledge production, theorising and social action. She also offers additional insights for different data gathering strategies as well as for other research frames (such as action research, evaluation research, survey research and experimental explanatory research). Letherby (2011) focuses on rounding out and enriching our understanding of feminist research. Importantly, Letherby explodes some persistent myths, such as (1) feminist research is inherently qualitative (some argue that it must be so), whereas Letherby argues that

380

11

How Do I Frame and Conceptualise My Research …

methodological choices should fit the research problem at hand and that any method might be viable; and (2) feminist researchers have no interest in men’s experiences and perspective, whereas Letherby (p. 67) argues that “we need to consider the social construction of both femininity and masculinity and focus our research on women and men’s experience”. Browne and Nash (2010) present a range of contributions in the emerging area of queer methods and methodology, highlighting the implications of queer theorising for social science research and methodology.

11.1.12

Transdisciplinary Research Frame

The Transdisciplinary research frame has an explicit Mode 2 knowledge focus. It attempts to bridge the divide between theory and practice and between academic and practitioner/professional. Transdisciplinary research is typically highly participatory and collaborative in emphasis, drawing upon different academic disciplines and non-academic practitioner and community inputs into the research. Transdisciplinary research typically reflects a critical realist pattern of guiding assumptions, focusing on problem-solving. However, it may also be useful to adopt participatory inquiry guiding assumptions in transdisciplinary research. Such research is ultimately intended to foster innovation and influence decision making in specific contexts. Leavy (2011, p. 30, adapted from Table 1.1) describes six essential principles of transdisciplinary research: • Issue- or problem-centered—problem at the center of research determines use of disciplinary resources and guides methodology; • Holistic or synergistic research approach—problem considered holistically through an iterative research process which produces integrated knowledge; • Transcendence—researchers build conceptual frameworks that transcend disciplinary perspectives in order to effectively address the research problem; • Emergence—placing the problem at the center of research (transcending disciplinarity) cultivates the emergence of new conceptual and methodological frameworks; • Innovation—researchers build new conceptual, methodological and theoretical frameworks as needed; and • Flexibility—iterative research process requires openness to new ideas and willingness to adapt to new insights. Transdisciplinary research employs traditional data gathering strategies but deployed in ways to maintain openness to, and harness synergies across, discipline boundaries instead of being constrained by those boundaries. Equally, transdisciplinary research seeks to break down the boundaries between theory and practice by diversifying the stakeholder influences and inputs to and ownership of the research. Stakeholders thus become active collaborators in the research. In this way, research moves beyond academic constraints toward stimulating change and innovation

11.1

Choice of Research Frame

381

where it matters most, be it in a community, a business, the environment, government, non-government organisation or another focal context. One goal, therefore, is to produce research that is useful to the public. The practice of critical engagement (see, e.g., Fear, Rosaen, Bawden, & Foster-Fuishman, 2006) is one way of approaching the transdisciplinary research frame. It demands that you, as academic researcher/scholar, engage directly with the public as you undertake your research so that the knowledge produced is meaningful/useful to that public. Note that critical engagement implies a somewhat asymmetric power relationship between academic/scholar and public in favour of the academic/scholar. One extension of the critical engagement approach is community-based transdisciplinary research which attempts to enact a more balanced power relationship with the public (Leavy, 2011). Community-based transdisciplinary research is explicitly collaborative and team-based in conception and execution, something entirely consistent with the participatory inquiry pattern of guiding assumptions. To accomplish this requires much attention to the building of trust and rapport amongst all involved in the research. It must be culturally sensitive to the publics involved in the collaboration and generally oriented toward producing meaningful change (similar in thrust to the action research frame) and avoiding/removing injustice. Community-based transdisciplinary research focuses on harnessing different ways of knowing, which is why collaborative participation is so important. Innovation and flexibility are essential to the undertaking of community-based transdisciplinary research, which tends to align more closely with data gathering strategies that emphasise connecting with people. In order to maximise the dissemination of what has been learned from community-based transdisciplinary research, creative methods, suited for targeting specific audiences, can be employed for depicting and sharing research meanings and implications. This means that the value for learning and presentational character meta-criteria are especially important to attend to in such research. Leavy (2011) describes another approach to transdisciplinary research which is arts-based. Here, perhaps, is the epitome of the creative potential of transdisciplinary research. In pursuit of research that is most useful to the public, non-traditional ways of building, displaying and applying knowledge are employed. The goal is to engage thinking and perception (across all sensory channels) through the use of various types of performance (theatre, dance, music, drama), imagery (photography, painting and sculpture), multiple medias (blogs, film and online) and language genres (narratives/ stories, literature, poetry, scripts and screenplays). Meaning is made by connecting with both the head and the heart of the public, which makes pathways for change and innovation more accessible and open (here again, the value for learning and presentational character meta-criteria emerge as critical to attend to). The arts-based approach to transdisciplinary research may also be useful if the research is intended to elicit meaning from, have meaning for and/or be useful to cross-cultural and/or Indigenous audiences. • Researcher Positioning(s): As transdisciplinary research has a multidisciplinary focus, it is typically more effective with a collaborative research team that is

382

•

•

•

•

11

How Do I Frame and Conceptualise My Research …

willing and able to seek out, learn from and build upon feedback from various sources. Appropriate attention must be given to the leadership of this team. If the approach is community-based, multiple stakeholders need to be involved in the research team. The goal is to look for synergies between disciplines and be willing to look beyond the boundaries of one’s own discipline (including one’s own preferred patterns of guiding assumptions) as well as being willing to be adaptive and flexible in approach. There is no room in transdisciplinary research for academic ‘snobbery’ and detachment. A Mode 1 knowledge production mindset will actively limit/constrain the potential and effectiveness of Mode 2 knowledge production as well as undermine the building of trust. Research Context(s): Transdisciplinary research is necessarily research done in the field, in professional or community contexts where practices can be influenced and/or innovation can occur. The research context usually involves the cultivation of a knowledge co-development relationship between researchers and participants. Context is where the research problem is centred. Participants’ Positioning(s): Research participants are those whose are involved with, impacted by or are stakeholders in the research to address the problem. They are co-creators of knowledge relevant to the problem(s) at hand. In community-based transdisciplinary research, stakeholders may become members of the research team which may mean they take on dual roles: researcher and participant. In a very real way, effective transdisciplinary research reaches out to the public in general, even if every member of the public cannot be explicitly included in the research itself (they could be considered ‘downstream’ stakeholders, for example). The research therefore, should have meaning for/be useful to all relevant audiences in the research context. Preferred Guiding Assumptions/Data Gathering Strategies: Working across disciplines generally means adopting a diverse range of guiding assumptions and employing multiple data gathering strategies. As indicated above, the critical realist and participatory inquiry patterns of guiding assumptions are typical choices. Transdisciplinary research is inherently pluralist in its orientation, often harnessing data gathering strategies as well as learning dissemination strategies in creative ways. Whatever data gathering strategies are adopted, they must be executed with a view to building and protecting trust. Contributions must be valued as well as protected from misuse or abuse. Learning Focus for Research Sponsors/Readers/Users: Primary learning outcomes from transdisciplinary research for public audiences emerge from patterns in data that meaningfully connect what has been learned to what must be done. They should provide impetus and invitation for social action, involvement, advocacy and innovation. Primary learning outcomes from transdisciplinary research for scientific audiences emerge from patterns in data that signal as well as evaluate useful directions, shapes and forms for social action and for innovations and their effective adoption (Lang et al., 2012). The scientific community would also be looking for potential directions for new research and innovation development. Primary learning outcomes from transdisciplinary research for public policy audiences emerge from patterns in data that signal

11.1

Choice of Research Frame

383

needed changes in societal practices/policies relevant to addressing specific problems that are sensitive to the diversity within different public groups/spaces. This then may influence decision making regarding how resources are deployed within communities. • Useful Resources: Leavy (2011) provides a succinct and very accessible discussion of the essentials of transdisciplinary research, including communitybased and arts-based research practices. Bergmann et al. (2012) provides a primer for transdisciplinary research practice, translated from the original German. Lang et al. (2012) discuss the possibilities for knowledge production and decision making via transdisciplinary research in sustainability science. Fear et al. (2006) reflects on the authors’ lived experiences in pursuit of critical engagement; a story that also illustrates the interpretivist technique of autoethnography as a way of interpreting and displaying the journey a person has taken (for example, through their life; in the context of a specific role they have undertaken; with respect to a specific project).

11.2

What Constitutes a Researchable Problem?

One of the most daunting aspects of planning your own research project is deciding what your research problem is, what it implies in terms of paradigm assumptions, research frames, choices of data sources and data gathering strategies choices and how that research problem should be translated into research questions. Ideas for a research problem can come from any source; the trick is to find a problem that you can make your own, that is acceptable to your supervisor(s) and that will lead to a feasible project to finish under the constraints that you face. You should realise that an important part of what makes a problem ‘researchable’ is the degree of your interest in the topic: a higher degree of interest will generally translate into enhanced motivation, commitment and focus down the track. (Note that White, 2009 is a very good general reference you could tap into for a more in-depth exploration of how to develop good research questions than we have room to explore here.) What we will do in this and subsequent sections of the chapter is to provide you with some prompts and tools to help you clarify your research problem and come to a clear statement of your research questions and/or hypotheses. Research problems and their associated questions/hypotheses emerge from the synergistic consideration of various contextualisations, positionings and research framing. This is an important point to grasp. Your research journey seldom commences with a clear idea of the research problem and attendant questions/ hypotheses in mind. Your research problem is generally identified, pruned, scoped, narrowed, winnowed and otherwise shaped as you contextualise and think through your own positioning, the research frame to be adopted, and positioning with other research as you read the literature. Moving from research problem to research questions/hypotheses brings in considerations of research context(s) and positioning of participants and other data sources. Some researchers, as part of their

384

11

How Do I Frame and Conceptualise My Research …

thinking to focus their research, will set out goals and objectives for the research which stand intermediate between the research problem and its translation into more specific research questions/hypotheses. In fact, whether you will end up setting out research questions or hypotheses will depend upon the pattern of guiding assumptions you have adopted as it intersects your research frame and other positioning considerations. For most researchers, getting from research problem to research questions/hypotheses is an iterative nonlinear activity and there is no single step-by-step logic process that can be set out to guide this part of the journey—it must be lived. What we can say, though, is that an important aspect of the process is to get your research problem focused to the point where it becomes feasible and realistic for you to tackle. This usually means that, at the start, your research problem is generally amorphous and often far too large to be manageable. Your goal is to whittle down the problem to a manageable form through making conscious decisions about what is and is not important for you to consider in your research; e.g., What are my boundaries? What/who do I include and what/who do I exclude and why? Where do I wish to be by the end of the project, with respect to the research problem? What do I want to be able to say and what will I not be able to say? If your problem needs to be pruned in order to become feasible to research, you should know what you are giving away and why. If there are new angles on the problem that emerge as being of interest, you need to be clear about the implications of following one or more of these new angles. Adopting a research frame can help in this process as can strategically exploiting the range and diversity of literature that you read. Your own interests and capacities play a role here as well. All of this thinking is important to record in your research journal. Potential sources of ideas for your research topic are very diverse (Allard-Poesi & Marechal, 2001; Cavana, Delahaye, & Sekaran, 2001; White, 2009): • Conversations with peers, professional colleagues, community members or with your supervisor(s) can suggest a research problem that you might find interesting to pursue. Such conversations may be formal or informal; what is critical is the active exchanging of ideas. Such conversations with your supervisor(s), for example, can provide you with insights into their interests and areas of expertise. Here you need to be careful that it is your genuine interest that is engaged, not the projection or grafting of the interests of others onto your interests. These conversations may have a useful side benefit which is that you may gain insights into preferred or more suitable patterns of guiding assumptions to adopt and you may learn about, perhaps even meet, key stakeholders. • Pursuing hunches and following up informal observations, perhaps at your place of work, is a common source of research topics. While this might seem a very unscientific and anecdotal means of getting to a research topic, it is nonetheless a very potent pathway, especially for professional doctorate research. Your research project can then become the rigorous and critically defensible pathway for checking these anecdotal impressions. In that pursuit, you may find the Exploratory research frame to be the best fit to your needs. Many an important scientific discovery was achieved through exploration that

11.2

What Constitutes a Researchable Problem?

385

followed a hunch or a little snippet of an observation by bringing it under more rigorous scrutiny. It may also be useful to consider a Transdisciplinary research frame here, especially if you want to produce new knowledge in concert with others outside the boundaries of specific disciplinary constraints. • Building on/extending the literature is a commonly recommended pathway for identifying a research problem or topic. By reading some convincing literature, even previous PhD theses, in a general area of interest, a specific focal problem may capture your fancy, perhaps leading you to ask, ‘what if I did this, as suggested by article X?’ If you recall the meta-criteria for gauging convincingness discussed earlier in Chap. 9, one specific meta-criterion is fertilisation of ideas. A research article worth its salt will address this meta-criterion by suggesting future directions for research following on from their own investigation. It is entirely acceptable for you to pick up one of these suggested directions and run with it as your own research topic. Furthermore, as another meta-criterion is handling of unexpected outcomes, it may be that you find a convincing study or a previous postgraduate thesis that found some surprising relationships and those findings could provide you with a kick-start stimulus for your own research topic. Following this pathway may lead you to consider adoption of an Explanatory research frame, a Case Study research frame, Cross-Cultural research frame or a Survey research frame. • Plugging gaps in the literature/pursuing new understanding is another commonly recommended pathway for identifying a research problem or topic. By reading the literature in a general area of interest, a specific gap or unanswered question may emerge, perhaps leading you to ask, ‘no one has addressed this issue before, what if I do?’ Recall that another meta-criterion for gauging convincingness is acknowledgement of limitations. A convincing study will be transparent about its limitations and you can build on this because the limitations, by definition, identify a need or gap to be filled. Alternatively, you may find weaknesses in the author’s contextualisation or conceptualisation of his/her study or with their research approach itself—these may point to gaps, such as important omitted variables, sampling constraints, measurement problems, untapped data sources or interpretation difficulties. Following this pathway may lead you to consider adoption of an Explanatory research frame, a Case Study research frame or a Survey research frame. If achieving new cultural understandings are of interest, the Cross-Cultural research frame or Indigenous research frame may be worth considering. • Following a new angle, approach or method is a slightly different take on ‘plugging the gap’, where, in the course of your reading the literature, you discover that no one has considered or implemented a particular approach to a research problem, leading you to think, ‘why don’t I try a new approach?’ It is not uncommon, for example, for a research issue in the domain of industrial/ organisational psychology to have been investigated solely under the positivist pattern of guiding assumptions. Your innovation may be to take an alternative paradigm approach by adopting, say, a constructivist perspective on the problem. The same can occur with the use of specific research frames or

386

11

How Do I Frame and Conceptualise My Research …

methodologies, where you see the possibility of using or even devising a new frame or methodological approach to the problem. • Solving a concrete institutional, organisational or community problem can sometimes be the stimulus for a research topic. This may come about through your own employment, where, for example, your employer becomes aware of your postgraduate research status and points to an important organisational problem for which they would like an evaluation of potential solutions. Alternatively, you may have identified a specific organisational issue you want to explore or develop an innovation to deal with as a result of your own experiences or informal observations. These applied pathways for identifying a research problem are more likely to occur in professional doctorates than for PhDs but should not be discounted for a PhD. This is especially the case nowadays as the boundaries between PhDs and professional doctorates are becoming more blurred (see, e.g., Kehm, 2006; Neumann, 2005). Here, you may find an Action research frame, Evaluation research frame, Developmental Evaluation research frame or Transdisciplinary research frame to be useful. Also possible here is a more direct application of the critical social science pattern of guiding assumptions where you adopt a critical stance on some issue. Here, the Action research frame, Feminist research frame, Indigenous research frame or Transdisciplinary research frame could be of value. • Informing decision-making with respect to some issue of interest is related to the previous applied pathway but is more focused on research being done to provide information for decision-makers to use. Here, your research topic emerges from someone’s need to make an important decision in a defensible and evidence-based manner. An Evaluation research or Developmental Evaluation research frame would be useful here as would the Action research frame or, in cases where broader reach is required, the Survey research frame. • Developing an innovation whose potential you want to explore/evaluate involves an applied focus for your research for which the Developmental Evaluation research frame is ideally suited. Here your research problem may focus not only on the development processes surrounding your innovation but also on research processes surrounding its potentials for adoption and use. A broad definition of innovation can be accommodated here, encompassing concrete innovations, such as an instrument, device, computer software or app, traffic, crop or pest management processes, as well as social innovations, such as new management processes, policy instruments, marketing strategies, community engagement processes, training programs, change management processes and the like. As Quinton and Smallbone (2006), Fisher (2007) and White (2009) have noted, the question of a ‘researchable’ problem raises several practical issues that you will have to address. The points below can be used as prompts as you begin to think about your research topic.

11.2

What Constitutes a Researchable Problem?

387

• Can you successfully investigate the problem within the constraints (time, resources, work, family or community commitments, etc.) that you must operate under? This is a critically important consideration that you must confront and, however undesirable it might be, it will work to shape, most likely ‘prune’, your research problem into a practically manageable form. Here, you may need to touch base with a number of different people to identify these constraints (e.g., your supervisor(s), colleagues, your spouse, your children, your boss). Another aspect to this consideration may be the question of whether your investigation of a specific research problem will be consistent with your career aspirations or ambitions. For example, if you are employed full-time, the last thing you would want to do is conduct research within your own organisation that could potentially threaten your job or career progression. Equally, you may have an aspiration to move from employment in the private sector into an academic position. This, too, would have an impact on the type of research problem you might pursue (e.g., for such a move, it may be desirable to choose a research problem that will give you ample room to deal with theoretical issues in a more mainstream discipline area rather than in a ‘fringe’ area). • Is your problem or topic one that you are interested in? Unless you can answer yes to this question, you will end up conducting your research without the important motivational drive you will need to bring the project to completion. The risk of this happening is higher if you are working with a supervisor who perhaps has a research grant and needs a postgraduate student (i.e., you) to conduct a study within the parameters of the grant topic. Of course, if the grant will provide you with necessary resources and financial support during the course of your study, that will add incentives for you. Just be aware that having resources and support available for a project that is basically handed to you, but for which you lack intrinsic interest, may adversely affect your motivation to ‘stay the course’ and give the project the detailed attention it will need. You really do have to own the topic to maximise your motivational potential. • Do you have the methodological and analytical skills you need to successfully complete the project? Fortunately, this question points you more toward identifying the skills deficits you may need to rectify before you embark on investigating the research problem you want to pursue than toward delimiting and circumscribing your choices. In other words, when you are working to identify and shape your research problem and strategies, be aware of the skills that may be demanded and plan to acquire those skills. Don’t let your lack of skills in a specific area determine your research problem choices if you can avoid it—you can always learn new things. What absolutely won’t work is to persist in pursuing a particular research problem without addressing the skills you will need to successfully complete the project. For example, we have seen students persist in pursuing a positivist questionnaire investigation within a Survey research frame, which required some fairly sophisticated skills in statistical analysis, actively avoid taking steps to acquire the statistical skills they needed (because they hated and feared mathematics). Instead, they operated in the hope that they could get someone else to do the analyses for them—a very poor strategy for success and it

388

11

How Do I Frame and Conceptualise My Research …

can create some thorny ethical issues regarding how much of the end product would actually be the student’s. A further drawback is that every skill deficit you allow to persist, in the face of the research problem you really want to explore, creates weaknesses in your capacity to tell a convincing story about the research in the thesis, dissertation or portfolio itself because you don’t have the knowledge and experiences you need to tell the story properly. • Is your research problem one that your supervisor(s) can provide sound advice on and that would meet the requirements and expectations of your department or school? This question goes to the central issue of the suitability of your research topic within the postgraduate educational context you must work in. You should always double-check that the research problem you want to explore will meet institutional expectations and requirements. For example, if you are in a management school, it may not be suitable for you to investigate a marketing research problem. This consideration may also influence the availability of supervisory competence in your chosen area. If you do choose an acceptable topic, from an institutional perspective, be sure you can get access to the supervisory expertise you will need. For some research topics, particularly those that have a multi- or interdisciplinary focus, you may have to travel further afield than your specific department or school to access all the supervisory expertise you need. Don’t be afraid to consider adding a co-supervisor from another department or school if that person can give you access to valuable and complementary advice and expertise. • Is your problem amenable to empirical enquiry? That is, can empirical data (either primary data or secondary data) be obtained that will adequately address the problem? Your answer to this question may depend partly on the discipline in which you are doing your research and partly on institutional expectations and requirements. In some areas, a conceptual or theoretical thesis may be acceptable (say, in an area such as ethics or law), but in most areas, the expectations would be for your thesis, dissertation or portfolio to have an empirical research component. Furthermore, the expectation may be for the thesis, dissertation or portfolio to involve primary data collection (data you collect yourself) rather than relying on secondary data (data collected by others, perhaps for other purposes), although this will certainly vary by discipline (for example, in finance, accounting and economics research, reliance on secondary data sources, such as financial or economic databases, may be the norm rather than the exception). It is important that you check out these expectations before you get too far down the track in shaping your research topic. • Can you get ethically-appropriate access to the sources of data that you will need to successfully complete the investigation of the problem? The problem of access to the sources of data you require to address your research problem is not trivial. If your problem is to understand how boards of directors in organisations or executive teachers in schools make their decisions, but you cannot access any organisations or schools that would allow you to talk to, survey or observe board members or executive teachers, you won’t be able to do your research. Equally, in accounting, finance and economics research, if your institution cannot provide you

11.2

What Constitutes a Researchable Problem?

389

with the access you need to requisite financial or economic databases, you will not be able to complete your study. Again, these are things to check out early on. In many cases, it will be your supervisor(s) who can provide advice on access. Perhaps they have a network of connections into organisations that you can link into or they may be able to negotiate access to requisite databases. Finally, note that this question includes the qualifier “ethically-appropriate” with respect to access. It is important to check, early on, whether your research problem may present you with ethical issues that might be difficult or impossible to surmount. For example, your research problem may need to rely on a snowball sampling strategy (see Chap. 19) to gain access to the right people for you to collect data from. However, the ethical approval guidelines in some universities may prohibit the use of snowball sampling because, in a very strict sense, it requires people to give you names of other people to contact which may violate the right to privacy for those other people. If this is the case and your research problem is dependent upon this approach for success, you may need to reshape your research problem.

11.3

What Tools Can I Use to Help Me Identify and Clarify My Research Problem?

Identifying your research problem is largely a process of moving your thinking from an amorphous mish-mash of ideas to a more coherent understanding of what it is you really want to investigate. You could begin by reading textbooks or journal articles in a general area of interest to you (such as teaching effectiveness in disadvantaged geographical areas, leadership in local government, farmer adaptations during droughts, power dynamics between nurses and doctors or strategic decision making by CEOs). Your initial spark of an idea might be triggered by something specific you observed (e.g., you saw something a leader or person in power did or heard something they said; you saw a shopper take a long time to decide which tinned tomatoes to buy; you have a child who experienced cyberbullying by schoolmates). You could be having an informal chat about a news story you just heard on the radio with one of your friends where the conversation revolves around trying to figure out why the Reserve Bank of Australia decided to raise interest rates by a half a percent and you suddenly realise that this is a problem you could actually try to get an answer to in a research project. The point is that ideas for research topics and problems can come from anywhere; what you need is to develop ways to capture and organise those ideas, so you have a basis from which your specific research topic/problem can emerge. There are several tools you can use to help you in this regard; you might use just one tool, or you might use some of them sequentially as you begin to home in on your research problem. Several of these tools are designed to help stimulate and harness your own creativity as you work your way toward a concrete research problem.

390

11

How Do I Frame and Conceptualise My Research …

• Recording notes and observations in your journal: Here you can begin to leverage the value of keeping your research journal from the very beginning. You could record any ideas, snippets of conversation, observations, hunches and/or concerns that occur to you as you think about identifying your research problem. Write down the things that interest you and any questions that come to you. You could then use the contents of your notations as grist for consideration in any of the processes described below. Sometimes, though, the mere act of writing down all these things can help your mind crystallise the problem. • Making linked lists: Here you generate a primary list of possible research ideas. Your list could be ordered according to your degree of interest or according to some other criterion. You could then ‘explode’ each primary idea in terms of its implications (thoughts, pros, cons, opportunities, constraints, etc.) for your research problem by linking a secondary list to each item on the primary list; items on your secondary list could be further ‘exploded’ by linking to a tertiary list and so on. How deeply you ‘explode’ your lists and what issues they surface is entirely up to you and depends on how complex your emerging research problem is (which, in itself, could be a clue as to how feasible the research problem might be for you to address during your postgraduate candidature). Table 11.1 shows a couple of examples. Notice that the first primary idea is ‘Dishonest behaviour in organisations’ suggesting this might be the most interesting research problem to explore. The secondary list explodes into four potential directions the problem could move in. Each of these is further exploded in the tertiary list. Notice that the contents of the tertiary list seem to be flagging some critical points of difficulty (cons) that could emerge if this particular take on the secondary list item was followed through. Compare this to the second primary item related to “Cyberbullying by schoolmates”. It would seem that this second research problem area might be more feasible to attack, but only from a certain angle (i.e., the more general issue of ‘What constitutes cyberbullying?’). The items on the secondary list could be used to qualify your research problem more tightly or could describe aspects of a single more general research problem. • Concept mapping or mind mapping (Quinton & Smallbone, 2006, pp 40–41; Buzan, 2018): Mind mapping is a technique invented by Buzan (2003) that provides a nonlinear full-colour method for organising concepts, ideas and issues associated with a central concept or idea. A concept map is a related technique that can be used to display different types of relationships between ideas and concepts. Mind maps will always have a central or focal idea, whereas concept maps are not quite so constrained. This means that a concept map can be used to capture patterns and flows (e.g., passage of time, cause and effect) of linkages between concepts. However, in terms of helping you to identify and clarify your research problem, a concept map, because of its flow orientation, may be a bit less useful than a mindmap. Figure 11.4 illustrates the use of a mind map to help characterise a research problem in sustainable business strategy. You can see that the mindmap not only permits issues to surface but also anticipates associations and potential connections that you could explore in your research; these could add dimensions to help shape your research problem.

• Cyberbullying by schoolmates

• What would make a person want to steal organisational property or a colleague’s ideas?

• Dishonest behaviour in organisations

• What kinds of kids are cyberbullied and why?

• What kinds of kids engage in cyberbullying and why?

• What would organisational stealing behaviour be related to? • What constitutes cyberbullying?

• How prevalent is this sort of thing?

• How could you stop this sort of behaviour?

Secondary List

Primary List

Table 11.1 Illustration of linked lists Hard to ask people about this What if I found out something really bad? Could I even get ethics approval for such a project? I’d have to know what is going on first Where would I get data for this? I can see privacy issues looming here Hard to ask people about this Would companies really want to know? I’d have to be able to observe people stealing then get their details—no way! • I could probably ask parents and teachers about this • How would identify parents and teachers to talk to and would they be willing to talk to me? • I know people who have been cyberbullied, so they might be worth talking to • Do I restrict my focus just to schoolmates, or should I go broader (e.g., workmates, community members, members of minorities)??? • I can see many problems with addressing the issue from this angle—not feasible??? • How would I identify cyberbullies? • What would my ethical responsibilities be if I talk to kids who admit to cyberbullying? • I could probably ask parents and teachers about this too, if they felt safe to talk to me • Could I talk to kids who have been cyberbullied and how would I make it safe for them to do so? • I could find out some potentially dangerous things about possible cyberbullies and their families here—then what would I do? Risky here? • How could I verify cyberbullying has occurred?

• • • • • • • • •

Tertiary List

11.3 What Tools Can I Use to Help Me Identify and Clarify My Research Problem? 391

392

11

How Do I Frame and Conceptualise My Research …

Fig. 11.4 A mind map for identifying and clarifying a research problem focusing on sustainable business strategy (drawn using Inspiration software)

• Force-field diagram (Quinton & Smallbone, 2006, pp. 39–40): Quinton and Smallbone recommended a rather ingenious use for an analytical technique, more often used in the management or organisational change, called a force-field diagram. It is a diagram that displays the ‘forces’ that you can envisage to be working ‘for’ (i.e., pros) and ‘against’ (i.e., cons) a particular research problem conceptualisation you are considering. You can even separately rank, rate or weight the strength of the various ‘for’ and ‘against’ forces, and add up the ‘for’ weightings and the ‘against’ weightings separately to form an overall impression of the feasibility of your research topic or problem—if the ‘againsts’ outweigh the ‘fors’, this would suggest an infeasible research topic. Figure 11.5 illustrates a force-field diagram for a research problem in the accounting discipline, drawn using the Inspiration software package. • The Research Matrix (Smyth & Maxwell, 2008): Smyth and Maxwell conceived the Research Matrix as a tool for helping you as well as your supervisors to conceptualise, plan and manage your research project. Your research journal would be a perfect document to keep track of versions of your Research Matrix. The Research Matrix is simply a two-way table that helps you to organise your thinking about your research problem. It is organic in the sense that it grows and takes shape as your project evolves; it is dynamic in that its structure can be flexibly altered to meet your own needs. The basic skeleton of the Research Matrix, adapted from Smyth and Maxwell (2008, p. 9), is shown in Table 11.2, with some illustrative content incorporated. Note that as the matrix evolves, you can include more and more downstream choices and implications as your thinking progresses.

11.3

What Tools Can I Use to Help Me Identify and Clarify My Research Problem?

393

Fig. 11.5 A force-field diagram for identifying the pros and cons of a research problem and establishing feasibility (in this example, the rating numbers in the boxes range from 1 = relatively unimportant and weak force, to 5 = extremely important and potent force)

The example matrix in Table 11.2 includes two columns additional to those shown by Smyth and Maxwell, namely, the column labelled “Potential research frame” and the column labelled “Guiding assumptions underpinning the problem/ questions”. These columns are important additions which will help you to link a research frame and guiding assumptions to your research problem; especially useful in a pluralistic research approach. The columns have also been slightly modified to allow for the use of the matrix in helping to shape your research problem. • BEAR strategy (Bring Every Available Resource, King 2002a): King (2002a, p. 121) suggests that the BEAR strategy is especially useful for “situations, tasks and problems that are perceived as impossible, intractable, unsolvable, extremely difficult, complex and non-routine”. The last three situational categories of “extremely difficult, complex and non-routine” are especially relevant to the identification and clarification of a research problem (we would certainly hope that the research problem you settle on would not be “impossible, intractable, unsolvable”!). What the BEAR strategy refers to is using all of your thinking resources, logical and analytical, intuitive and sensory and everything in between to conceptualise and analyse a problem. We could characterise the problem of homing-in on your research topic as:

Evaluation research frame

Interpretivist/ constructivist pattern, but could be addressed under positivist guiding assumptions as well

Interpretivist/ constructivist pattern or positivist pattern

Explanatory research frame or survey research frame

Documents on developers in actual universities; key policies; organisational structure; reflections on past development events

Documents on developers in actual universities; key policies; organisational structure

Information on the university and higher education context in Australia

Information needed to address research problem

Higher education; human resource management and organisational development and change literature Theories on organisational development and adult learning; organisational development; training; universities; mentoring Organisational development methods; training methods; trainees; universities; stakeholders; clients; evaluation

Key literature/ keywords relevant to the research problem

?

?

?

Type of data needed to address research problem

?

?

?

Potential data gathering strategies and analytical approaches

?

?

?

Time-frame

11

Q2. Who do organisational developers ‘develop’ within universities, how and how well?

Interpretivist/ constructivist pattern

Exploratory research frame

Understanding the roles of organisational developers in Australian universities Q1. What do organisational developers do in universities and why?

Pattern of guiding assumptions under-pinning the research approach to the problem

Potential research frame

Potential research problem

Table 11.2 Research matrix illustration

394 How Do I Frame and Conceptualise My Research …

11.3

What Tools Can I Use to Help Me Identify and Clarify My Research Problem?

395

– resident in the BEAR problem-definition space, where problems, tasks, anomalies, issues, hypotheses and suggestions are built up, explored and sorted through; – a general open-ended problem, which is ill-defined (unclear boundaries and limits, at least initially), broad (covering perhaps several related disciplines or areas), possibly reliant on ‘soft’ data/information (such as your informal perceptions and hunches) and may encompass only weak linkages between components or aspects (especially where the literature is equivocal about what is known or about the nature of relationships); – responsive to conceptual or lateral thinking and exploration, using creativity enhancing tools such as mind maps, brainstorming, concept maps and metaphors; – producing outcomes such as conceptual maps or pathways to potential representations of your research problem and possibly signalling the need for multiple approaches; and – providing broad-level directional guidance for further and more detailed exploration, focus and planning. The BEAR strategy is not a sequence of steps per se but a description of an overall process. The ultimate point of the BEAR strategy is to continually remind you to be open-minded and flexible as you confront the problem of getting to your research topic. Bring all your resources and talents to bear on the problem, realising that logic and analysis is only part of the process. If you are interested, King extends and expands his BEAR strategy to encompass new tools for conceptualisation and problem-solving such as the versatile matrix and the versatile map (2002b). A versatile matrix (King 2002b, p. 127) lists all the different types of thinking strategies that can be brought to bear at different points in the problem identification ! problem-solving process. A versatile map (King 2002b, p. 129) is a variant of a mind map designed to help you unpack your entire research problem. It does this by focusing your attention on three distinct, yet interconnected, thinking contexts called ‘spaces’: the problem-definition space, the methods space and the solutions space. Thinking in the problem definition space is most closely tied to identification of your research problem. Thinking in the methods and solutions spaces is most closely tied to unfolding your planned approach to addressing the research problem. In this way, you can begin to see the implications of your research problem for the methodological and analytical choices you may have to make downstream. King (2002b) includes a practical example of a versatile map in the appendix to his paper (pp. 132–136). Figure 11.6 shows a simplified template for a versatile map. • Trigger questions: This is a simple self-questioning technique that you might find useful in helping to focus on a preliminary statement of your research problem. The questioning pattern has a kind of U-shaped logic pattern to it: it starts out general, gets more specific, then ends up with a more general statement. Start with the question ‘What?’ and write down the general research area of interest. Then cycle through a series of questions to qualify and focus your response to the ‘What?’ question in terms of aspects of that response that may be of interest to

396

11

How Do I Frame and Conceptualise My Research …

Fig. 11.6 Template for King’s versatile map for creative thinking during the research process (loosely adapted from King 2002b, p. 129)

you: Who? Where? How? Why? When? Answers to some of these questions are likely to be fairly specific, but that is OK. What you want to do is to try to bring all your responses together to make a preliminary but coherent statement of a research problem. This statement should have a more generalised form than your specific responses in that it should sit at a somewhat higher level of abstraction. Think of this statement as what you might use as the lead-in sentence in your elevator pitch. Let’s try an example in the research area of ‘leadership’ for a PhD student employed in the Organisational Development Unit in a university: – – – –

What?—I want to study something in the leadership area. Who?—I want to study Heads of Departments in universities. Where?—I want to study people at my university. How?—I want to learn how Heads of Departments can get their staff to become involved in applying for research grants, by talking to Heads and their staff? – Why?—Staff keep getting told by senior management to get engaged in research grant activity but there are still a lot of staff not engaged and I’d like to know why. – When?—I’d like to get some answers before the next round of Australian Research Council Discovery and Linkage applications are due in. – Preliminary Statement of the Research Problem: Encouraging involvement in research grant-getting activities in universities.

11.3

What Tools Can I Use to Help Me Identify and Clarify My Research Problem?

397

Executive decision making under stress

Managerial decision making when companies are under survival threat

How do managers make decisions when their companies are undergoing a merger?

How do managers make decisions when their company is in legal trouble?

Do managers in banks undergoing a merger make faster and less consultative decisions?

Do managers in energy companies become hyper-vigilant and indecisive when their company is in legal strife over pollution problems?

How do managers make decisions when their companies are downsizing?

Do managers in local city councils make their downsizing decisions using performance criteria or personal preferences for employees?

Do managers in local city councils emphasise financial or human concerns when their council is downsizing?

Fig. 11.7 A relevance tree for a research problem in the area of decision-making

• Relevance Tree (Fisher, 2007): Fisher (2007) discusses another diagrammatic approach to help you conceptualise and fine-tune your research problem. The relevance tree is a technique for organising the issues, questions and considerations that emerge from your thinking about the research problem. The tree is organised so that the most general issue or consideration is at the top and more specific issues/questions emerge as you work your way down. Different branches of the tree signal different pathways that your research problem clarification could follow. Followed to its logical end, the relevance tree could end up with specific potential research questions at the bottom. Choices along the branches help to shape your final research problem; in the diagram, you could encircle your preferred choices or pathways. Figure 11.7 illustrates a relevance tree for research in the area of decision-making. Note how the considerations become increasingly focused from top to bottom; the aspects encircled by the dashed line suggest the most feasible take on the research problem, with potential research questions emerging at the bottom. King (2002c) presents a nice compendium of websites and software resources for enhancing and harnessing creative thinking. Some other potential software resources for organising and displaying your ideas and facilitating the creative connections between them include (all the listed websites offer downloadable trial versions of the package):

398

11

How Do I Frame and Conceptualise My Research …

• Inspiration (http://www.inspiration.com/): a software package that can help you organise mindmaps, concept maps, diagrams and/or outlines. • AXON Idea Processor (http://axon-research.com/): a software package that offers a wide range of creativity stimulating and organising frameworks, including diagrammatic and mapping methods and metaphorical and word game techniques. • SmartDraw (https://www.smartdraw.com/): another mind mapping package, but also can facilitate concept mapping as well. • Mindjet Mind Manager (http://www.mindjet.com/): a combined mind mapping/ concept mapping package offering a useful range of visualisation methods. • ConceptDraw (http://www.conceptdraw.com/en/products/mindmap/main.php): a concept/mind mapping software package.

11.4

The Emergence of Your Research Questions/ Hypotheses

The form of and language used in stating research questions or hypotheses are sensitive to underlying pattern of guiding assumptions. Figure 11.8 takes your emergent research contextualisations and positionings pathway from Fig. 10.1 as the starting point and diagrammatically expands the considerations associated with research questions and research hypotheses as they evolve from your research problem. The end goal of a critical literature review (about which more will be said in Chap. 13) is to set out, in a well-argued form, your intended research questions or hypotheses as they have emerged from your conceptualisation of the general research problem. There is both a science and an art to how you frame and state your research questions and/or hypotheses (Creswell, 2003). They must be consistent with your chosen pattern of paradigm assumptions and expectations and they must be consistent with the intended positioning and framing of your study. They may be stated assertively, yielding hypotheses (e.g., ‘Managers trained in decision-making will make faster and more accurate decisions than managers who are not so trained’), or phrased as a more open and general question (e.g., ‘How do managers account for how they make important and complex decisions?’). They are more specific than your general research problem statement and your choice of language for wording research questions and/or hypotheses will be critical decisions you will have to make. Depending upon your research problem, you may have just a single emergent research question/hypothesis, or you may have a series of connected research questions/hypotheses. In general, it is better practice to have a few well-stated research questions/hypotheses than to have many exceedingly precise questions or hypotheses. We have seen PhD students attempt to investigate up to 40 very precise research hypotheses, which is far too many to do justice to in the context of a thesis, dissertation or portfolio project. Such a strategy is a signal that the research problem has not been sufficiently conceptualised, narrowed down and pruned for feasibility.

Open Research Question

Effect Constructs of interest [Dependent variables/ Criteria]

Strong causes determines results in changes affects increases/decreases

Weak influences predicts impacts explains

Associative associated with correlated with related to

Used in the Explanatory & Evaluation research fames; perhaps in the Action research frame, the Case Study research frame or some forms of research in the Survey research frame (especially marketing & decision making research) Hypotheses propose patterns of relationships between constructs to be looked for and evaluated – testing the how & why of observations State in affirmative form (e.g., X causes Y), not null form (e.g., X does not cause Y) Constructs need to be operationally defined and quantitatively measured via some procedure Possible to state hypotheses in comparative form: e.g., X will be more effective than Z in changing Y Examples of relational connectors:

[strong, weak, associative]

Relational Connector

Hypothesis

The Emergence of Your Research Questions/Hypotheses

Fig. 11.8 Depiction of the distinctions between open research questions and research hypotheses

Used in Exploratory & Descriptive research frames and some approaches in the Survey research frame Signals constructs of interest, but often not anticipated patterns of relationship Focus on identifying/counting who, what, when, where categories, often in the context of demographic research

Causal Constructs of interest [Independent variables/ Predictors]

The research problem/research questions/research hypotheses emerge from the synergistic unfolding of contextualising, framing and positioning strategies

Could be useful in virtually any research frame Signals context(s)/situation(s)/event(s) on which learning is focused Signals whose perspectives are being sought (e.g., participants) Signal an interest in achieving depth of understanding May signal categories or types of behaviour of interest Avoids anticipating any patterns or relationships Uses open-ended wording that leaves room for many possible outcomes to be encompassed

Following from Fig. 10.1

11.4 399

400

11

How Do I Frame and Conceptualise My Research …

11.4.1 Hypotheses Hypotheses are associated with the positivist pattern of guiding assumptions and assert theorised connections between specific constructs that are to be tested. Your literature review should clearly lead the reader to expect the constructs and association patterns that are implicated in your research hypotheses. The last thing you want to do is surprise the reader by incorporating constructs in your research questions that you have not previously discussed and reviewed in the literature. Your literature review should act like a funnel, leading the reader to logically expect the hypotheses you set forth. As shown in right-hand branch of Fig. 11.8, hypotheses have two key components: (1) identification of key constructs on either side of a (2) relational connector. Constructs, as we will see later in Chap. 18, require operational definition and measurement. In hypotheses, constructs play one of five different roles. Constructs, when translated into measured variables, can be: (1) independent (i.e., putative causes), (2) dependent (i.e., putative effects), mediating (i.e., putative intermediate causes), (4) moderators (i.e., putative conditional causes) or (5) extraneous (i.e., anticipated alternative plausible causes to be explicitly controlled for). You must keep these roles straight in all of your research hypotheses. We have seen a number of theses where, for example, the student has confused the role of independent and dependent variables in the research question, leading to incorrect analyses being used and meaningless conclusions being drawn, to say nothing of confusing the reader. Once all your relevant constructs have been identified and measured/quantified for a hypothesis, statistical analysis is then used to test for the existence of the proposed relational pattern. Hypotheses tend not to signal context or participants. Good practice is reflected if hypotheses are stated using affirmative language, stating outright what relational pattern you hope to substantiate, rather than using statistical null hypothesis language. Nothing is more boring to a reader than to read a list of null hypotheses (e.g., ‘age will not influence job satisfaction’) when it is obvious that you are expecting a relationship. State what you do expect. Hypotheses may vary in terms of the strength of causality they theorise to exist: strongly causal, weakly causal, associative. Strongly causal hypotheses state the precise causal patterns that you are looking for, clearly pointing to the constructs thought to be causes and those thought to be effects. Such hypotheses are most more likely to be used in the Explanatory and Evaluation research frames. The language used to express strong causal hypotheses signals that strength: e.g., “X causes Y”, “X determines Y”, “X results in Y”). Such hypotheses require strong theoretical arguments to justify them, with the logic well-anchored in previous literature. Choosing wording like “X influences Y” or “X predicts Y” provides weaker causal statements than “X causes Y” and allows you to soften your causal claim while still signalling a directional intent for the theoretical inference. Weak causal hypotheses may be more appropriate if the theoretical justification for strong causal hypotheses is ambivalent or contradictory. Associative hypotheses back away from making causal assertions because the research area may be new (as with the Exploratory research frame), the research focus is case-based (as with the Case Study

11.4

The Emergence of Your Research Questions/Hypotheses

401

research frame), the research intent may be descriptive (as with the Descriptive research frame) and/or where theoretical justification is lacking (as with certain ways of implementing the Survey research frame). Such hypotheses instead make claims of association, using language such as “X is associated with Y”, “X is correlated with Y”, “X is related to Y”. Technically, once you set out a hypothesis to be tested, it should not be changed downstream within the same research project. To do so might expose you to claims of running a fishing expedition. In cases where you are testing precise theoretical predictions, you would be expected to state hypotheses to reflect the anticipated relationships between two or more constructs or anticipated differences between different groups (Tharenou, Donohue, & Cooper, 2007; White, 2009). However, you must be careful in stating anticipated directions of difference. If you have no theoretical basis on which to say that ‘X will increase Y’, then back away from predicting the direction by simply stating that ‘X will produce significant changes in Y’. Similarly, if you have no compelling reason to expect that “Group A will significantly improve in performance relative to group B”, then soften your expectation by stating “Group A performance will significantly differ from Group B”. This is important because stating a direction will often require you to use a slightly modified form of hypothesis testing called a ‘one-tailed test’ and, if you hypothesise the wrong direction, then you will be in a real mess with respect to the convincingness of your conclusions. That is why you should have a strong theoretical basis to make such precise predictions.

11.4.2 Research Questions Open research questions (follow the left branch in Fig. 11.8 to the left forked pathway) tend to be associated with interpretivist/constructivist patterns of guiding assumptions where they do not anticipate what might be found. Instead an open research question points more toward general focal interests and perhaps to types of behaviour/phenomena of interest and often, you may have just a single general research question. Open research questions use looser language more focused on whose perspectives are to be understood in what circumstances (e.g., “what do A, B and C think about event or situation P?”, “How do A, B and C accomplish tasks 1, 2 and 3?”, “What can we learn from A, B and C?”, “Why and how has situation P come about in context G?”, and tend to shy away from naming specific constructs or relationships of interest. Open research questions set out the contexts(s) and phenomena that are your focus for learning, without second-guessing what patterns might emerge. Research questions may also focus on specific participants or types of participants, where relevant. Furthermore, open research questions may evolve or be further refined or shaped during the course of your research journey as a consequence of what you have learned earlier on (something that should not occur under the positivist pattern of guiding assumptions, unless you use a sequential research configuration—see Chap. 12). Thus, there would be an expectation of flexibility and evolution attached to your research questions in that, as your analyses progress, further questions may evolve to pursue greater depth and focus. As Creswell (2003, p. 107) so clearly put it with respect to qualitative research questions, you should “[e]

402

11

How Do I Frame and Conceptualise My Research …

xpect your research questions to evolve and to change during the study in a manner consistent with the assumptions of an emerging design”. It is possible to employ open research questions in research guided by the positivist pattern of guiding assumptions (follow the left branch in Fig. 11.8 to the right forked pathway), especially where an Exploratory or Descriptive research frame has been adopted or where census-type questionnaires are being used in the Survey research frame. Constructs are still of interest in such questions, but relational patterns between them are typically not proposed. Positivist open research questions tend to focus on identifying/counting categories of who, what, when and where with respect to some population of interest. They are often employed in demographically-oriented research.

11.4.3 A Generic Illustration Figure 11.9 illustrates the possible evolutionary pathways from a general research problem statement to research questions/hypotheses. The uppermost pathway shows the evolution of possible positivist research hypotheses. You can see the progression from a positivist general statement of the research problem to a general research question to more specific hypothesis statements. The statements all use affirmative language signalling what you hope to be able to show and vary in strength of hypothesised causal connectivity as you scan downward from the weakest statement at the top (a relational hypothesis in the lightest box) to the strongest hypothesis (darkest box). Note that the strong hypothesis is not only affirmatively-phrased (i.e., ‘will be more effective’) but also makes a directional prediction. This prediction this would only be justified if there was prior research evidence or theory suggesting that face-to-face communication would be more effective than email communication in other research contexts. The lower pathway shows the evolution of possible interpretivist/constructivist research questions. Here again, you can see the progression from a general statement of the research problem to an open general research question to more specific research questions. The specific research questions propose different types of participants and specific contexts as points of focus and employ more open language that does not anticipate what might be learned but does signal either a descriptive (‘what’, ‘how’, or ‘when’) or explanatory (‘why’) intent. It is important to realise that the shape and specificity of your research problem as well as your questions/hypotheses may be influenced by a number of contextual factors (depicted by the 4-way arrow box). These certainly include your choice of pattern of guiding assumptions, but may also include what goals you are pursuing overall, what research frame you have adopted, what the literature or other secondary data sources will or won’t permit you to say, what extant theory may have to say, what stakeholders in the research may be expecting you to say and even the constraints you are operating under. The influence of constraints is especially

11.4

The Emergence of Your Research Questions/Hypotheses

403

Are attitudes toward safety procedures & practices related to accident rates? What determines whether or not employees follow safety procedures & practices?

The culture of an organisation will influence attitudes toward safety procedures & practices. Management-employee communication practices predict level of compliance with safety procedures & practices.

Interested in studying employee attitudes toward & compliance with safety measures & practices in organisations

Influences on problem/ question/hypothesis form and specificity: • Goal(s) to be achieved • Choice of research frames • Relevant positionings • Guiding paradigm assumptions • Previous literature/theory • Secondary data sources • Relevant Stakeholders • Constraints

Strong Hypothesis: Face-to-face communication of a new safety practice or procedure will be more effective in enhancing compliance than email communication.

What can we learn from managers and employees about safety procedures & practices in a manufacturing workplace?

What do safety procedures & practices mean to employees and how are they enacted?

What can workers tell us about why they do or don’t comply with safety procedures & practices in coal mines? How & when do employees enact or avoid enacting safety procedures & practices in a manufacturing company?

Fig. 11.9 Moving from a general research problem (on the left) to more specific research questions or hypotheses, influenced by various contextual and positioning considerations (the 4-way arrow)

important to recognise, because they can influence how you configure and execute your study. For example, you may wish to test fairly precise causal hypotheses about particular construct relationships, but because you cannot obtain permission to conduct a controlled experiment in an organisation, you may have to settle for conducting survey research which is generally a weaker methodology for inferring causality. This constraint thus forces you to rethink your study and will move your hypotheses away from stating strong causal expectations toward more associative or descriptive expectations.

11.5

Some Concrete Examples from Recent PhDs and Professional Doctorate Portfolios

It will be useful at this point to provide some concrete illustrations of research problems, questions and hypotheses from actual PhD theses and professional doctorate portfolios, supervised by Ray. Also shown is evidence for choice of research frames and patterns of guiding assumptions for each project. Table 11.3 provides six illustrations: three conducted under predominantly positivist

404

11

How Do I Frame and Conceptualise My Research …

assumptions (Peter Fieger, Sujana Adapa and Michael Muchiri) and three conducted under predominantly interpretivist/constructivist assumptions (Ziyad Alghannam, Martin Robson and Wayne Gregson). We say predominantly because several of these postgraduate research projects were pluralist in configuration, however, the dominant paradigm guided the evolution of the research questions. The examples show that there is a range of different styles for presenting research problems, questions and hypotheses that will remain consistent with adopted guiding assumptions. Also note that different postgraduate students embedded different levels of contextualisation in their presentations. Note that hypotheses were only relevant under positivist guiding assumptions, but not always specified. You can see that as one moves from research problem, to research questions, to hypotheses there is an increase in specificity and focus. Under positivist guiding assumptions, the move also reflects an increased focus on anticipating or predicting outcomes. In contrast, under interpretivist/constructivist assumptions, anticipation or prediction of outcomes is actively avoided to prevent a student’s preconceptions from driving what they are looking for. This will necessarily lead to research questions of less specificity and precision and more open-endedness. In the examples associated with interpretivist/constructivist assumptions, notice that the research questions often use the words “what are”, “how and why”, “explore if” or “how do”, indicating a desire to find out what is occurring rather than anticipating what might occur. Note also how Martin, Wayne and Ziyad all added contextual detail to their research problem statements, to varying degrees. This is also a common practice for interpretivist/constructivist research. In Wayne’s case, he offered contextual detail that clarified the applied innovation focus of his research; he was both CEO of the organisation in which he conducted his research and he was the creator of the innovation that he was developmentally evaluating. Note how Martin’s arguments for guiding assumptions moved toward a new set of guiding assumptions that blended some of the features of positivism with some of the features of interpretivism. Note that in the positivist examples with specific hypotheses, the way the hypotheses were stated varied according to the strength and direction of prediction the student was willing to make. Michael had a strong theoretical basis for his study and, therefore, felt justified in setting out hypotheses that predicted specific directions for outcomes. His hypotheses were set out in a more narrative form and signalled the need to control for certain extraneous constructs (e.g., demographic characteristics). In contrast, Sujana did not have quite as strong or unequivocal a theoretical base for her study and, therefore, set out hypotheses that simply predicted relationships to exist (reflected in the words “will significantly predict”), without predicting a direction. Peter’s EdD portfolio did not propose hypotheses because he was breaking new ground in constructing institutional performance measures).

Table 11.3 Concrete examples, using explicitly quoted material, from recent PhD theses and professional doctorate portfolios illustrating research problems, questions and hypotheses; quotes that implicate research frame and pattern of guiding assumption choices are also shown Research problem statement “The aim of this research portfolio is to develop a framework that enables comprehensive performance measurement in the Australian TAFE sector” (p. 3) “develop new quantitative methodologies for a suite of performance measures, as well as to make some concrete actionable policy recommendations.” (p. 5) “This research develops methodologies that facilitate the measurement of the performance of educational institutions and determine how different performance measures relate to each other. The study relies mostly on secondary data that were aggregated to the institutional level.” (p. 22)

Research frame and pattern(s) of guiding assumptions adopted

Evaluation research frame: “the proposed study will aim to investigate the value of completing a VET (Vocational Education and Training) qualification”. (p. 21) Paper 3 Descriptive research frame: “primarily concerned with the development of performance indicators that deal with outputs.” (p. 32) Paper 1 Explanatory research frame: “study will determine how these methods (parametric and nonparametric efficiency scores) compare in the context of Australian vocational education.” (p. 20) Paper 2 Pattern of guiding assumptions: “Specifically, this is a quantitative data-driven study, with the aim of generalisable and replicable results. These considerations about the intended research illustrate that a positivist set of guiding assumptions is employed to conduct the study with a hypothetico-deductive model that underpins it.” (p. 22)

Thesis author and title

Fieger (2015) Efficiency and Effectiveness in the Australian Technical and Further Education System EdD professional doctorate portfolio, comprising three linked papers Example research questions: “What is the impact of TAFE institutes on labour market outcomes such as employment after training and post training salaries after demographic and institutional characteristics are taken into account? (p. 34) Paper 1 “How is institutional efficiency distributed within the ATS and which environmental variables help to predict patterns of efficiencies and why?” (p. 116) Paper 2 “What predictors are there for actual completion (of VET awards) and how useful are they?” (p. 178) Paper 3

(continued)

The literature was not specified enough to permit precise hypotheses to be put forward, plus, since part of the purpose was creating and evaluating new performance measures, the research questions had to be more general

Hypotheses

Some Concrete Examples from Recent PhDs and Professional…

Research questions

11.5 405

Research questions “The principal objective of this thesis is to assess the predictive power of antecedents to consumers’ continued and frequent use of internet banking in an Australian context. Specifically, this thesis examines technology factors such as relative advantage, compatibility, complexity, trialability and result demonstrability; channel factors such as perceived self-efficacy, perceived risk, perceived trust and perceived personalisation; social factors such as subjective norm and normative beliefs; and value factors such as perceived benefits and perceived costs and their predictive relationships with consumers’ continued and frequent use of internet banking.” (p. 12)

Research problem statement “The goal of this study was to investigate the factors that influence how consumers continue to use, and how frequently they use, internet banking in Australia. …The research in this thesis is a response to a gap in existing literature which requires the application of more integrated theory testing and the identification of factors that influence the continued and frequent use of internet banking in order of importance to consumers. (p. iii)

Research frame and pattern(s) of guiding assumptions adopted

Survey research frame combined with the Explanatory research frame: “The present study attempted to test theoretical propositions using operationally defined constructs measured via survey methodology (Arnold et al. 2005).” (p. 92) Pattern of guiding assumptions: “This aligned the research with the general tradition of objectivism and positivism (Church & Waclawski 1998). The intention was to evaluate the explanatory contributions of specific sets of antecedent constructs in a hierarchical fashion so that each construct could be seen in light of its ability to predict the usage patterns of consumers using internet banking.” (pp. 92–93)

Thesis author and title

Adapa (2010) An Investigation of Factors Influencing the Continued and Frequent Use of Internet Banking by Australian Consumers PhD thesis

Table 11.3 (continued)

11 (continued)

Selected hypotheses that were tested: “H1A: Identified technology factors will significantly predict continued usage of internet banking over and above the influence of the demographic control variables. H1B: Identified technology factors will significantly predict frequency of internet banking usage over and above the influence of the demographic control variables.” (p. 90) “H2A: Identified channel factors will significantly predict continued usage of internet banking over and above the influence of the demographic control variables and technology factors. H2B: Identified channel factors will significantly predict frequency of internet banking usage over and above the influence of the demographic control variables and technology factors.” (p. 90)

Hypotheses

406 How Do I Frame and Conceptualise My Research …

Research questions “RQ1: How and why did the key decision-makers in a Middle Eastern company make and enact a strategic decision to implement and adopt a Western-developed HRS, and its associated management practices, into their workplace? RQ2: How did the strategic decision to apply and adopt a Western HRS and its associated management practices influence the company and its individuals?” (p. 4)

Research problem statement “What are the implications of adopting a Western HRS (Human Resources System) into a Saudi company?” (p. 3)

Research frame and pattern(s) of guiding assumptions adopted

Cross-Cultural research frame: “This research investigated the importation of Western-originated human resource systems (HRSs) and associated management practices into a Middle Eastern (ME) Saudi company.” (p. iii) Pattern of guiding assumptions: “The epistemological stance chosen for this study is constructivist because it is suitable for understanding what the STC’s employees thought and how they felt and acted in response to the importation of a Western HRIS system. Thus, the interpretive approach is not a frame of mind, but a set of assumptions about how the researcher will approach the research and its methodology. These assumptions focus on the social construction of meaning (i.e., perspectives of people within the STC and Oracle, which reflects a relativist ontological orientation; this is where and why subjectivity also becomes relevant).” (pp. 4–5)

Thesis author and title

Alghannam (2016) Importing Western Human Resource Systems and Practices into a Saudi Company: A Case Study Analysis PhD thesis

Table 11.3 (continued)

(continued)

Not appropriate given the adopted pattern of guiding assumptions

Hypotheses

11.5 Some Concrete Examples from Recent PhDs and Professional… 407

Research problem statement “… my primary task was to develop, implement and evaluate a mechanism to capture and utilise ideas within DFES. By combining my personal experience through the lens of my academic learning, I formulated the principal research focus to be on how I, as CEO of the Western Australian Fire and Emergency Services Authority, could improve the capture of ideas from members of staff and volunteers through a knowledge portal known as the P2P. (p. 86)

Research frame and pattern(s) of guiding assumptions adopted

Action research frame: “The study of human activity does not always readily fit within an understood scientific, conventional research approach. For this reason, the overarching research configuration for this Innovation Portfolio Project is guided by an Action Research/Developmental Evaluation Framework. … This perspective supports the suitability of Action Research as it allows a qualitative social research approach with the dual objectives of action and research—action to stimulate change in a community or organisation, and research to increase understanding of the system under investigation (Dick, 1993).” (p. 85) Developmental Evaluation frame: “It can be argued that developmental evaluation is the fusion of these different types of thinking; the bringing together of rigorous inquiry and change-oriented leadership. There is the ongoing consideration of what is happening and the preparedness and adaptability to make changes during implementation. Conceptualising what is occurring and adjusting key reform elements in real time are the basis of such an approach.” (p. 90)

Thesis author and title

Gregson (2016) Harnessing Sources of Innovation, Useful Knowledge and Leadership within a Complex Public Sector Agency Network: A Reflective Practice Perspective PhD.I professional doctorate portfolio

Table 11.3 (continued)

Formulated as a series of steps for addressing the research problem: “The overall objective of this project is to explore if: • there are a number of elements that can be identified that may impact project implementation, in a positive and/or negative manner; • a key factor in harnessing innovation is to adopt an approach whereby the source of the innovation comes from engaging the workforce; and • facilitating factors are optimised and restraining factors minimized through ongoing workforce engagement during the implementation process.” (p. 86)

Research questions

11 (continued)

Not appropriate given the adopted pattern of guiding assumptions

Hypotheses

408 How Do I Frame and Conceptualise My Research …

Robson (2011) The Use and Disclosure of Intuition(s) by Leaders in Australian Organisations: A Grounded Theory PhD thesis

Thesis author and title

Table 11.3 (continued)

Explanatory research frame/ Descriptive research frame: “GT (Grounded Theory] is therefore compatible with the stated aims of this research, which is to describe and explain the basic social processes in relation to the disclosure of intuition in Australia organisational contexts”. (p. 97)

Pattern of Guiding Assumptions: “I selected interpretivist/constructivist paradigm assumptions to guide the research configuration in this study. This perspective is appropriate when the research purpose is to understand and describe meaningful social action in specific contexts (Neuman, 1997). This perspective is also consistent with the purpose of this study and the proposition that participants in P2P (the ‘Portal2Progress’ innovation) would be a rich source of knowledge about its implementation and operation. An interpretive methodological perspective suggests that the researcher is a subjective participant in the research process rather than an objective observer. (p. 94)

Research frame and pattern(s) of guiding assumptions adopted

“What are the social processes of intuition use and disclosure by Australian leaders in organisations?” (p. 15)

Research problem statement

“Main question 1: How do the participants (organisational leaders) interpret, use and value intuition in their decision-making and leadership? Main question 2: What are the social processes of intuition disclosure by Australian leaders in organisations?” (p. 15)

Research questions

(continued)

Not appropriate given the adopted pattern of guiding assumptions

Hypotheses

11.5 Some Concrete Examples from Recent PhDs and Professional… 409

Muchiri (2006) Transformational Leader Behaviours, Social Processes of Leadership and Substitutes for Leadership as Predictors of Employee Commitment, Efficacy, Citizenship Behaviours and Performance Outcomes PhD thesis

Thesis author and title

Table 11.3 (continued)

Survey research frame combined with the Explanatory research frame: “The goal of this study was to examine the separate and combined effects of transformational leadership behaviour and social processes of leadership within the context of substitutes for leadership as predictors of key individual and organisational outcomes in Australian local councils. The research answered the recent call by Podsakoff et al. (1996) for more integrative leadership theory testing. A survey research methodology was used to gather quantitative and qualitative data from nine local councils in New South Wales comprising 177 employees sampled across a wide range of council divisions and job levels.” (p. iii)

Pattern of guiding assumptions: “Thus, Dey’s position is in concert with Layder’s view that GT is neither strictly ‘interpretivist nor positivist’ (Layder 1998, p. 133). I support this dual and seemingly paradoxical epistemological stance in relation to GT, based on the notion of a stratified ontology (outlined in Chap. 2 and 3), which assumes that the researcher is both separate and not separate from the data, concurrently”. (p. 104)

Research frame and pattern(s) of guiding assumptions adopted

“This study is intended to add to our knowledge of how leadership manifests itself in local government councils and how it impacts on organisational commitment, citizenship behaviours, collective efficacy beliefs and outcomes expectancy, organisational efficacy and performance outcomes of councils’ employees.” (p. 6)

Research problem statement

“The current study aims to examine whether leadership influences employees’ performance outcomes. It also examines the moderating and mediating effects of substitutes for leadership on performance outcomes.” (p. 45)

Research questions

11 (continued)

Selected hypotheses that were tested: “1. Demographic characteristics would be associated with employees’ organisational commitment, collective efficacy beliefs and outcomes expectancy, organisational efficacy, organisational citizenship behaviours and performance outcomes. As demographic characteristics were not of major research interest, their influence would be statistically controlled for in a bid to provide clearer tests of the remaining propositions. 2. After controlling for relevant demographic influences, transformational leadership, transactional leadership and social processes of leadership will be predictive of employees’ organisational commitment, collective efficacy beliefs and

Hypotheses

410 How Do I Frame and Conceptualise My Research …

Thesis author and title

Table 11.3 (continued)

Pattern of guiding assumptions: “The present study assumed the existence of an objective physical and social world, whose nature could be apprehended, characterised and measured with relative certainty, and which would yield data that approximated to an objectively verifiable reality (Arnold et al. 2005). The research was conducted on the premise of the existence of prior fixed relationships within phenomena, which could typically be investigated with structured instrumentation. The empirical study involved testing a theory, measuring quantifiable variables, testing hypotheses, and drawing inferences about the chosen phenomena from the sample to the stated population. … the proposed study could be classified as a positivist study” (p. 51)

Research frame and pattern(s) of guiding assumptions adopted

Research problem statement

Research questions

outcomes expectancy, organisational efficacy, organisational citizenship behaviours and performance outcomes. The leadership influence will be both direct and indirect. Furthermore, the impact of transformational leadership and social processes of leadership will be positive in these relationships whereas the impact of transactional leadership will be negative.” (p. 49)

Hypotheses

11.5 Some Concrete Examples from Recent PhDs and Professional… 411

412

11.6

11

How Do I Frame and Conceptualise My Research …

Key Recommendations

Some important things to remember to implement regarding research frames and identifying your research problem/questions/hypotheses are: • It is important to devote some serious time to choosing the research frame(s) that will best fit your research purposes. This choice will typically emerge from thinking about your positioning as researcher, research contexts, participants’ contexts and positioning and research sponsor/user contexts. Don’t neglect to consider the wider contexts surrounding your research as well, e.g., social, political and physical worlds, and you should realise that it is from these larger contextual domains that many stakeholder issues can emerge. Remember that research frames can work on their own or may be synergistically combined with other frames, depending upon your needs. Committing to one or more research frames will help you to further contextualise your research in a way that will provide a more holistic picture of how your research purposes can/will be translated into research strategies. This then will focus your efforts to generate and/or apply and/or share knowledge and learning that will speak to and influence and perhaps even involve specific audiences and relevant stakeholders. • Let your interests guide you in the selection of a research problem area but realise that research problems can emerge from many potential sources, even hunches or informal observations. Research problems may also present themselves to you via a community member, organisation or employer. For postgraduate research topics, a good strategy for identifying your research problem would be to combine the above considerations with familiarity with the literature of the area in order to identify a gap in knowledge or a new development or angle to follow. This will help you to build a coherent argument about why your chosen research problem is important to pursue—something you will need to do in your write-up stage. • Feasibility is a key factor that will likely lead to pruning and constraining your research problem. This means that you cannot/should not identify your research problem and research frame(s) without factoring into your thinking issues such as time, funding, work and family commitments and constraints. Part of feasibility is linked to considerations of your own skill sets and potential skill deficits as well as where your supervisor(s) are coming from and what they as well as your department or school expects. Your goal is to identify a researchable problem consistent with those expectations and in consideration of any ethical implications. • Use whatever organising and analytical support tools you feel comfortable with to help you to organise your thinking about your research problem and its subsequent evolution into research questions and, where appropriate, research hypotheses. Whatever you do, use your research journal to its fullest extent to record this part of your journey—you will find this record invaluable at write-up time when you will have to reconstruct your thinking in specific thesis or portfolio chapters.

11.6

Key Recommendations

413

• Identify and word your research problem and research questions in a manner consistent with your research frame, pattern of guiding assumptions and intended contextualisations and positionings of your study. Not even the best researchers get this right every time, but, as a postgraduate researcher, getting this right goes a long way toward convincing an examiner that you know what you are doing and you know how your assumptions have influenced what you are doing (i.e., it will give you a big plus on the convincingness meta-criterion!). Remember that interpretivist/constructivist patterns of guiding assumptions, in contrast to positivist guiding assumptions, will require a looser, far less prescriptive and far more open expression of the research problem/questions in order to show that your preconceptions are not prematurely influencing what you are looking for. Conversely, under the positivist pattern of guiding assumptions, your expression of the research problem/questions is expected to be much tighter and focused toward anticipating outcomes and predictions of relationships. Furthermore, unless you are doing a purely exploratory or descriptive study, you would generally be expected to construct hypotheses which serve as the concrete expressions of your anticipated outcomes and predicted relationships between constructs. • Remember that specificity generally increases from stating your research problem to statement of your research questions to statements of your research hypotheses. You can provide some contextualisation in the statement of the research problem, but, by the time you get to hypotheses, you will need to be very specific and focused with very little room left for contextualisation. • If your research frame, guiding assumptions and research questions demand hypotheses, remember the following hints: – State your hypotheses affirmatively—be clear and transparent about what you are expecting to find. – Be very careful in your wording choices and ordering. Hypotheses don’t need to have a causal intent, they may simply have a relational intent. However, you want to ensure that you don’t inadvertently imply causal intent through an inappropriate word choice. For example, simply saying that ‘X will influence Y’ will be enough to suggest causal intent to a reader. If your methodology will not support such intent, then you will have a problem convincing the reader that you can properly assess the hypothesis as stated. Instead, soften your language away from this risk by stating something like ‘X will be associated with Y’. – Ordering of constructs in a hypothesis statement is also important. Remember that independent variables are the putative causes in any hypothesis and dependent variables are the putative effects. Thus, the roles of both the independent and dependent variables must be semantically clear. In a generic ‘if X changes, then Y will change’ phrasing, independent variables will define the X construct(s) (sets the condition for the ‘if’) and dependent variables will define the Y construct(s) (sets the conditional outcome for ‘then’). A variant of this phrasing, ‘X will predict Y’ (frequently used in

414

11

How Do I Frame and Conceptualise My Research …

survey research, for example), uses the same positional identities for independent and dependent variables, but use of the verb ‘predicts’ softens the causal intention because predicting something does not necessarily imply that you understand the causal mechanisms. Another variant of the phrasing, ‘Y will change when X changes’, simply swaps the positions, but not the roles, of the independent and dependent variables—giving essentially the same meaning as ‘if X changes, then Y will change’. – Hypotheses may incorporate references to other types of constructs such as mediators, moderators and/or extraneous variables. Reference to extraneous variables is perhaps more common and would usually be signalled through a reference to a need to control for their influence when testing for the hypothesised relationship between independent and dependant variables (as Michael Muchiri and Sujana Adapa did in their hypotheses shown in Table 11.3).

References Adapa, S. (2010). An investigation of the factors influencing the continued and frequent use of internet banking by Australian consumers. Unpublished PhD thesis, School of Economics, Business & Public Policy, University of New England, Armidale, NSW. Alghannam, Z. (2016). Importing western human resource systems and practices into a Saudi company: A case study analysis. Unpublished PhD thesis, UNE Business School, University of New England, Armidale, NSW. Allard-Poesi, F., & Marechal, C. (2001). Constructing the research problem. In R.-A. Thietart (Ed.), Doing management research: A comprehensive guide (pp. 31–50). London: Sage Publications. Babbie, E. (2011). The basics of social research (5th ed.). Belmont: Wadsworth Cengage Learning. Bamberger, M., Rugh, J., & Mabry, L. (2012). RealWorld evaluation: Working under budget, time, data, and political constraints (2nd ed.). Los Angeles: Sage Publications. Bergmann, M., Jahn, T., Knobloch, T., Krohn, W., Pohl, C., & Schramm, E. (2012). Methods for transdisciplinary research: A primer for practice. Frankfurt: Campus Verlag. (translated from the original German edition by RC Faust). Browne, K., & Nash, C. J. (Eds.). (2010). Queer methods and methodologies: Intersecting queer theories and social science research. London: Ashgate. Bryant, A., & Charmaz, K. (Eds.). (2007). The Sage handbook of grounded theory. Los Angeles: Sage Publications. Buzan, T. (2003). The mind map book (Rev ed.). London: BBC. Buzan, T. (2018). Mind map mastery. London: Watkins. Cavana, R. Y., Delahaye, B. L., & Sekaran, U. (2001). Applied business research: Qualitative and quantitative approaches. Milton, QLD, Australia: John Wiley & Sons. Chilisa, B. (2012). Indigenous research methodologies. Los Angeles: Sage Publications. Coghlan, D., & Brannick, T. (2014). Doing action research in your own organization (4th ed.). London: Sage Publications. Cook, T. D., Campbell, D. T., & Peracchio, L. (1990). Quasi-experimentation. In M. D. Dunnette & Hough, L. M. (Eds.), Handbook of industrial and organizational psychology (Vol. 4, pp. 491–576). Palo Alto: Consulting Psychologists Press, Inc.

References

415

Cooksey, R. W. (2011). Yours, mine or ours: What counts as innovation? Journal of Agricultural Education and Extension, 17(3), 283–295. Creswell, J. W. (2003). Research design: Qualitative, quantitative and mixed methods approaches (2nd ed.). Thousand Oaks: Sage Publications. Davies, P. (2006). Exploratory research. In V. Jupp (Ed.), The Sage dictionary of social research methods (pp. 110–111). London: Sage Publications. de Vaus, D. (2008). Comparative and cross-national designs. In P. Alasuutari, L. Bickman, & J. Brannen (Eds.), The Sage handbook of social research methods (pp. 249–264). London: Sage Publications. Donaldson, S. I., Christie, C. A., & Mark, M. M. (2009). What counts as credible evidence in applied research and evaluation practice?. Thousand Oaks, CA: Sage Publications. Dul, J., & Hak, T. (2008). Case study methodology in business research. Oxford: Butterworth-Heineman. Ember, C. R., & Ember, M. (2009). Cross-cultural research methods. Lanham: AltaMira Press. Fear, F. A., Rosaen, C. L., Bawden, R. J., & Foster-Fuishman, P. G. (2006). Coming to critical engagement: An autoethnographic exploration. Lanham: University Press of America. Fieger, P. (2015). Efficiency and effectiveness in the Australian Technical and Further Education system. unpublished EdD portfolio, UNE Business School, University of New England, NSW. Fisher, C. (2007). Researching and writing a dissertation: A guidebook for business students (2nd ed.). Essex: Pearson Education. Fowler, F. J. (2014). Survey research methods (5th ed.). Los Angeles: Sage Publications. Gray, D. E. (2014). Doing research in the real world (3rd ed.). Los Angeles: Sage Publications. Greenwood, D. J., & Levin, M. (2007). Introduction to action research: Social research for social change (2nd ed.). Thousand Oaks, CA: Sage Publications. Gregson, W. (2016). Harnessing sources of innovation, useful knowledge and leadership within a complex public sector agency network: A reflective practice perspective. Unpublished PhD.I portfolio, UNE Business School, University of New England, Armidale, NSW. Harkness, J. A., Braun, M., Edwards, B., Johnson, T. P., Lyberg, L., Mohler, P. P., et al. (2010). Survey methods in multinational, multiregional, and multicultural contexts. Hoboken, NJ: John Wiley & Sons. Hesse-Biber, S. N. (Ed.). (2012). Handbook of feminist research: Theory and praxis. Los Angeles, CA: Sage Publications. Hesse-Biber, S. N. (2014). Feminist research practice: A primer (2nd ed.). Los Angeles, CA: Sage Publications. Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). New York: Oxford University Press. Kehm, B. M. (2006). Doctoral education in Europe and North America: A comparative analysis. Wenner Gren International Series, 83, 67–77. King, R. (2002a). When to use creativity. In T. Greenfield (Ed.), Research methods for postgraduates (2nd ed., pp. 121–125). London: Arnold. King, R. (2002b). Synthesis and evaluation of creativity. In T. Greenfield (ed.), Research methods for postgraduates (2nd ed., pp. 126–136). London: Arnold. King, R. (2002c). Software and websites for creativity. In T. Greenfield (ed.), Research methods for postgraduates (2nd ed., pp. 137–140). London: Arnold. Kovach, M. (2009). Indigenous methodologies: Characteristics, conversations, and contexts. Toronto: University of Toronto Press. Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537–567. Lambert, V. A., & Lambert, C. E. (2012). Qualitative descriptive research: An acceptable design. Pacific Rim International Journal of Nursing Research, 16(4), 255–256. Lang, D. J., Wiek, A., Bergmann, M., Stauffacher, M., Martens, P., Moll, P., et al. (2012). Transdisciplinary research in sustainability science: Practice, principles, and challenges. Sustainability Science, 7(1), 25–43. Leavy, P. (2011). Essentials of transdisciplinary research: Using problem-centred methodologies. Walnut Creek, CA: Left Coast Press.

416

11

How Do I Frame and Conceptualise My Research …

Letherby, G. (2011). Feminist methodology. In M. Williams & W. P. Vogt (Eds.), The Sage handbook of innovation in social research methods (pp. 62–79). Los Angeles, CA: Sage Publications. Liamputtong, P. (2010). Performing qualitative cross-cultural research. New York: Cambridge University Press. Martin, K., & Mirraboopa, M. (2003). Ways of knowing, being and doing: A theoretical framework and methods for indigenous and indigenist re-search. Journal of Australian Studies, 27(76), 203–214. McIntyre, A. (2008). Participatory action research. Thousand Oaks, CA: Sage Publications. Mittal, V. (2011). Descriptive research. In J. Sheth & N. Malhotra (Eds.), Wiley international encyclopedia of marketing (Vol. 2). New York: Wiley. http://onlinelibrary.wiley.com/doi/10. 1002/9781444316568.wiem02002/full. Muchiri, M. (2006). Transformational leader behaviours, social processes of leadership and substitutes for leadership as predictors of employee commitment, efficacy, citizenship behaviours and performance outcomes. Unpublished PhD thesis, New England Business School, University of New England. Neumann, R. (2005). Doctoral differences: Professional doctorates and PhDs compared. Journal of Higher Education Policy and Management, 27(2), 173–188. Nicholls, R. (2009). Research and indigenous participation: Critical reflexive methods. International Journal of Social Research Methodology, 12(2), 117–126. Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: The Guilford Press. Patton, M. Q. (2012). Essentials of utilization-focused evaluation. Los Angeles, CA: Sage Publications. Patton, M. Q., McKegg, K., & Wehipeihana, N. (Eds.). (2015). Developmental evaluation exemplars: Principles in practice. New York: The Guilford Press. Punch, K. (2003). Survey research: The basics. London: Sage Publications. Quinton, S., & Smallbone, T. (2006). Postgraduate research in business: A critical guide. London: Sage Publications. Robson, M. (2011). The use and disclosure of intuition(s) by leaders in Australian organisations: A grounded theory. Unpublished PhD thesis, School of Economics, Business & Public Policy, University of New England, Armidale, NSW. Ruel, E. E., Wagner, W. E., & Gillespie, B. J. (2015). The practice of survey research: Theory and applications. Los Angeles, CA: Sage Publications. Smith, L. T. (2012). Decolonizing methods: Research and indigenous peoples (2nd ed.). London: Zed Books. Smyth, R., & Maxwell, T. W. (2008). The research matrix: An approach to supervision of higher degree research. Milperra, NSW: Higher Education Research and Development Society of Australia. Stebbins, R. A. (2001). Exploratory research in the social sciences. Thousand Oaks, CA: Sage Publications. Sue, V. M., & Ritter, L. A. (2007). Conducting online surveys. Los Angeles, CA: Sage Publications. Tharenou, P., Donohue, R., & Cooper, B. (2007). Management research methods. New York: Cambridge University Press. Tripodi, S., & Bender, K. (2010). Descriptive studies. In B. Thyer (Ed.), The handbook of social work research methods (2nd ed., pp. 120–130). Los Angeles: Sage Publications. Walter, M., & Anderson, C. (2013). Indigenous statistics: A quantitative research methodology. Walnut Creek, CA: Left Coast Press. White, P. (2009). Developing research questions: A guide for social scientists. London: Palgrave Macmillan. Yin, R. K. (2014). Case study research: Design and methods (5th ed.). Los Angeles, CA: Sage Publications.

Chapter 12

How Do I Scope, Shape and Configure My Research Project?

In order to make your research feasible and realistically achievable, you will need to make scoping and shaping choices, pertaining to the nature of the research activities that you will use to gather the evidence you need and configuring choices, pertaining to the patterns and connections between those research activities. In short, you are focusing on how you intend to navigate the ‘Data Triangle’ (recall Fig. P.1). Such choices move you toward the ‘pointy end’ of research where you assemble the evidence you need to address your research questions/hypotheses. Appropriately scoping and configuring research is often a very iterative thinking forward-working backward type of process where your ‘final’ (in quotes to signal that nothing is ever really final in research—adaptations and trade-offs are almost always necessary) research configuration does not emerge until you have explored the implications of earlier choices for later choices and have revisited some of those early choices in light of impediments or constraints you experience or can foresee down the track. The iterative scoping and configuring process can often be aided by conversations with other people, including peers, colleagues and relevant stakeholders and gatekeepers, as they may bring to light issues you have not yet considered or have insufficiently considered. If you are a postgraduate research student, important voices in these conversations will be your supervisor(s), postgraduate peers, key stakeholders and gatekeepers and, where appropriate, a research funding or commissioning organisation (perhaps associated with your supervisor(s)). If you are a professional doctorate student, other important voices will reside in your workplace, your community or communities of practice, and other stakeholders.

© Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_12

417

418

12.1

12 How Do I Scope, Shape and Configure My Research Project?

Scoping and Shaping Your Research: Working Within Constraints

In social and behavioural research, constraints are everywhere exerting their influence on the focus, scope and shape of the research that you would really like to do. On the one hand, there is the research project that you would actually like to carry out in an ideal world (a world without constraints)—call this your ‘initial ideal’ study. In many ways, this is the research that many research methods texts typically try to get you to plan—the ideal study. On the other hand, there are the myriad of constraints that you must deal with and respond to in some way and each of those constraints can force you to give away or trade-off some aspect of your initial ideal study in order to make your research more realistic and feasible to achieve. The direction of such trade-offs is almost always unidirectional—away from the initial ideal toward what is realistic and feasible to accomplish; thus: Initial Ideal ! Realistic and Feasible. This is why you will never find a perfect or flawless social or behavioural research study produced or published anywhere by anyone. All social or behavioural research has flaws, some more serious or damaging than others, created by certain trade-offs the researcher had to make in order to get the job done. The goal for you in your research is to manage these trade-offs in such a way that they can still lead to a convincing piece of work, a cognitive process we call ‘scoping and shaping’. Trade-offs away from your initial ideal should always be made with an eye toward their implications for the ultimateconvincingness of your research. If you make a trade-off (which may not be under your control but may instead be the result of outside pressures on you), then you need to think about what you could do to help offset or ameliorate the loss, at least to some extent. Sometimes, new opportunities may present themselves to you during a research project and one or more of these opportunities may help to compensate for the detrimental effects of certain trade-offs. Because of this, you need to be alive not only to the constraints you face but also to potential opportunities that may help to defuse, circumvent or recover from the impact of some of those constraints. This means that, in many cases, there may be trade-off decisions that actually improve on the initial ideal, by reshaping what you end up doing and what you will learn. In short, you need to be highly adaptable. You must set boundaries to limit the focus of your project and to constrain your ‘attack’ on your research questions. You must be clear as to what is ruled in for your project to consider and what is ruled out. Such decisions need to be deliberate and well-reasoned and constitute the major part of the process of ‘scoping and shaping’ your research. Table 12.1 presents some concrete illustrations (in the form of mini-scenarios) of the central messages we are trying to convey here. All the mini-scenarios in Table 12.1 reflect the influence of constraints on scoping and shaping a research project, moving research away from an ideal form to a more feasible form. Proper scoping and shaping of your research project is absolutely essential for postgraduate research projects and it is part of the role of supervisor(s) to help you in this regard.

What constraint does the researcher face?

A large representative sample is unrealistic for the researcher to obtain

Four of the six targetted schools do not give permission to approach their teachers for research purposes

What is the researcher’s ideal goal?

A researcher wishes to obtain a large random and representative sample of data sources for a cross-sectional questionnaire, guided by the positivist pattern of assumptions

A researcher wants to access teachers in the six largest metropolitan schools in Melbourne in order to conduct his research and be able to compare what is learned across the six schools

An unfortunate confluence of the planned research process and the internal/external event or process in each school, leading the Principal to decline

The declining schools may be undergoing some internal or external event or process that the research could potentially interfere with

The researcher is an academic doing unfunded research and her department is unable to pay for access to the necessary database for sampling

Lack of access to a relevant population listing for implementing a random and representative sampling scheme

The Principals of each declining school would be the key gatekeepers controlling access to their institution

The researcher is a postgraduate student with very limited access to financial support

Lack of financial or support resources available to the researcher for gathering data from a large number of data sources

The topic of the research might have inadvertently encouraged each declining school to adopt a negative position with respect to access

Who or what might impose the constraint?

How might the constraint come about?

The researcher may need to alter his research questions and perhaps his overall research goals and intentions, depending upon which schools from which metropolitan areas decline and which schools are chosen to replace them

Have a plan B in place to approach other necessarily smaller schools to fill up the sampling spaces created by the declining organisations

(continued)

Capacity to generalise may be weakened, but if useful comparative data from the additional survey questions emerges, some cautious statements about the population might be defensible

Capacity to generalise to a population is sacrificed; however, the shift in guiding assumptions and use of semi-structured interviews may permit a richer and deeper story to be told

What does the researcher sacrifice or gain if trade-off is made or the opportunity is followed up?

Sample from data sources that can be easily, and perhaps locally, accessed and ask additional relevant questions that will allow for some comparisons between sample composition and any available information on the target population in order to make judgments about representativeness

Change guiding assumptions to an interpretivist pattern and the data gathering strategy to semi-structured interviews so it does not require a large sample

What research tactic (i.e., trade-off) or emergent opportunity might be useful to ameliorate/circumvent it?

Table 12.1 Some illustrative scenarios showing how trade-offs from ideal to realistic and feasible might occur because of a specific constraint and the potential implications of this (some illustrations of the benefits of following up emerging opportunities also appear)

12.1 Scoping and Shaping Your Research: Working Within Constraints 419

What constraint does the researcher face?

The university’s Ethics Committee declines to approve the conduct of the research, demanding that the researcher alter her methods to avoid deception

After three weeks, the response rate to the questionnaire was only 12%; far too low for the researcher to achieve statistical conclusion validity with his analyses

During the pilot test, a large number of Chinese participants refused to finish the questionnaire, once they had gotten about three-quarters of the way through it

What is the researcher’s ideal goal?

A university researcher needs to disguise the true purpose of her research by deceiving participants into thinking the study is about workplace stress in order to get accurate and valid data in her study of dishonesty in the workplace

A researcher plans to achieve a 75% response rate for his 100-item questionnaire administered to consumers in a shopping centre

A researcher employs a pilot test in order to fine-tune her multi-cultural questionnaire before proceeding to her main data collection phase. Her goal was for the pilot test to show that the instrument would work for the intended sample

Table 12.1 (continued)

When interviewed about their experience of the questionnaire, a number of Chinese participants indicated that they found some questions to be culturally offensive and many quit after encountering the third such question

A 100-item questionnaire is very long and that may put potential participants off as well, leading them to end their participation before they complete it

Shoppers may be too busy or may be put off by being asked to complete the questionnaire

University policy requiring all research involving human participation conducted by staff members to be vetted and approved by a central committee; a policy that directly implements the Australian national guidelines for the ethical conduct on human research

How might the constraint come about?

The participants themselves are imposing this constraint as they are the ones who have been offended by the questions

Shoppers themselves impose these constraints, likely conditional on what they are doing/dealing with at the time of being asked to participate or during the process of participation

The Federal Government as well as many of the professional associations for research academics (e.g., the Australian Psychological Association), mandate that all universities must adhere to the national guidelines for the ethical conduct on human research

Who or what might impose the constraint?

(continued)

The researcher could improve the potential response rate from the Chinese cultural group but could also inadvertently create problems for members of other cultural groups of participants

The incentive may alter motivations to participate, which could influence questionnaire responses (paid participants may try to give the researcher the answers/ responses they think he wants) Offer participants a desirable incentive (e.g., a payment or chance at a winning a paid holiday at a desirable resort) for their participation in a longer questionnaire Rewrite the offending questions in a way that will not offend members of that cultural group, building upon feedback from the Chinese pilot test participants

The researcher may have to sacrifice some of the constructs he wanted to measure, which may force changes to his research questions/hypotheses

Outcome will depend upon Ethics Committee acceptance of the deception amelioration measures; this would require very strong arguments and perhaps allocation of additional resources

Provide extensive support mechanisms and debriefing protocols to employ post-deception with participants, to ameliorate or undo any deleterious effects of being deceived Reduce and streamline the length of the questionnaire, so it will impose less of a perceived time commitment on the part of participants

The researcher may sacrifice capacity to convincingly evaluate her original research questions/hypotheses

What does the researcher sacrifice or gain if trade-off is made or the opportunity is followed up?

Reconfigure the research so that deception is not required

What research tactic (i.e., trade-off) or emergent opportunity might be useful to ameliorate/circumvent it?

420 12 How Do I Scope, Shape and Configure My Research Project?

The researcher could explore alternative analytical techniques (such as partial least squares, data transformations or nonparametric procedures) in order to analyse the data

Scoping and Shaping Your Research: Working Within Constraints (continued)

The researcher would not be able to test his hypotheses in quite the same way, or perhaps with the same fidelity, as he could have had he configured a proper study with SEM firmly in mind as the analytical strategy; also might mean he would have to seek an alternative publication outlet

The data gathered are ill-suited for SEM, because they do not satisfy appropriate statistical assumptions

An academic marketing researcher receives feedback from marketing colleagues he asked to review an early draft of his manuscript and each colleague has indicated that to get published, he should use structural equation modeling (SEM) as his data analysis strategy

The researcher has little knowledge of and no experience in using SEM

The researcher could suffer an internal conflict by knowingly moving to adopt less suitable guiding assumptions in order to realise an instrumental gain for her career (she might also conduct less convincing research if she is less experienced in conducting research under a positivist pattern of guiding assumptions)

The researcher could reconfigure her research to be guided by positivist assumptions in order to enhance her chances of getting published in the school’s preferred journal

The researcher did not configure his investigation so that the data yielded would be suitable for SEM

Researcher would enhance chances of publication without sacrificing her preferred guiding assumptions and possibly her research goals, but in the process may not meet the expectations of her school

The researcher could pursue another journal outlet of equal or near equal international status, but one that would be demonstrably receptive (as revealed by the journal’s publication policy or the nature of previous articles published) to research guided by an interpretivist pattern of assumptions

The journal editor has a positivist orientation and selects editorial board members and peer reviewers who share a similar background/worldview. The head of school also imposes this constraint believing that publication in this prestigious journal will bring additional kudos to the school as well as a boost for her staff member’s career

The majority of the peers that submitted manuscripts get sent to for review have research experiences accrued from many years of conducting and publishing their own research guided by positivist assumptions

The researcher reviews a number of issues of the journal for the past five years and discovers that with just four exceptions, only quantitative research articles guided by positivist assumptions have been published

On the advice of her head of school, a researcher aspires to publish her work in the well-respected and highly ranked International Journal of X; her research is guided by an interpretivist pattern of assumptions

What does the researcher sacrifice or gain if trade-off is made or the opportunity is followed up?

What research tactic (i.e., trade-off) or emergent opportunity might be useful to ameliorate/circumvent it?

Who or what might impose the constraint?

How might the constraint come about?

What constraint does the researcher face?

What is the researcher’s ideal goal?

Table 12.1 (continued)

12.1 421

The student could experience internal conflict if the recommended project does not match his interests or capabilities and could experience lack of ownership of the research which could hamper his motivation to complete the project Student could experience enhanced motivation as a consequence of increased ownership due to being able to follow his own interests and ideas, but may not be able to access the level of resources (or even the professional network) that working with supervisor Y might have provided This strategy would certainly remove the power dynamic issue and make it easier to recruit participants, but it would alter the researcher’s role from being an insider to being an outsider (meaning that the researcher would lack detailed knowledge of the company she was studying (since it was not one she worked for) and would have to implement additional data gathering strategies to develop such knowledge)

The student could capitulate and agree to undertake a PhD project as suggested by supervisor Y, or

The student could opt to seek out another supervisor, perhaps one without as strong an international reputation but willing to be more open about the type of project they are willing to supervise

The company manager may need to switch her focus to a different company where her power relationships with potential participants will not be an issue

The background, training and prior research experiences of the supervisor creates a preference for certain projects and it is these sorts of projects she pursues funding for

Employees themselves impose this constraint with their decision not to participate, for whatever reason (and the researcher may not be able to learn these reasons)

The supervisor has funding for certain projects and is seeking postgraduate students to carry out some of the research work involved

Employees may decline to participate because they perceive a potential threat if their supervisor gets wind of what they think and say and, perhaps, they may be suspicious about the researcher/manager’s agenda in conducting the study

Y, as supervisor, will only supervise certain types of projects and this student’s proposed project does not fit that agenda

Many of the employees she approaches to participate in her study decline

A postgraduate student wants to do his PhD under the supervison of internationally-known academic Y

A company manager wishes to study morale and related issues within her own company for her management PhD. She has been employed by the company for 10 years

(continued)

What does the researcher sacrifice or gain if trade-off is made or the opportunity is followed up?

What research tactic (i.e., trade-off) or emergent opportunity might be useful to ameliorate/circumvent it?

Who or what might impose the constraint?

How might the constraint come about?

What constraint does the researcher face?

What is the researcher’s ideal goal?

Table 12.1 (continued)

422 12 How Do I Scope, Shape and Configure My Research Project?

The participants themselves are imposing this constraint, within the opportunity provided by the researcher’s interviewing approach

The Principal, acting on parents’ concerns, makes access to the school conditional on the researcher agreeing to not split up classroom cohorts

It may be that the people she is interviewing have had some tough work experiences or hold some unpopular or marginalised points of view and, when invited to express their views during the interview, grab the opportunity to talk in desperation

Parents may have expressed views that splitting up classroom cohorts for research purposes could have longer-term deleterious effects on classroom experiences and relationships, especially if one program worked and the other didn’t

Early on, she realises that the interviews are averaging 90 min or longer in length resulting in extraordinarily long transcripts, making completion and analysis of 60 such interviews much less feasible within her timeframe

The researcher learns that classroom cohorts may not be split up for purposes of the research which means that random assignment of students to programs is not permitted

As part of a program evaluation project, a researcher wishes to evaluate the efficacy of two different programs for teaching primary school mathematics and to do this convincingly, he wants to run a true experiment where students in each classroom are randomly allocated (by a coin toss) to one or the other program

Who or what might impose the constraint?

A postgraduate student plans to conduct 60 in-depth semi-structured interviews with employees in a troubled organisation for her PhD to ensure a sufficient and well-rounded story can be told

How might the constraint come about?

What constraint does the researcher face?

What is the researcher’s ideal goal?

Table 12.1 (continued)

The student would have a more feasible project within the time constraint but would likely sacrifice some degree of sufficiency because she cannot interview all of the people she wanted to talk to (some of whom might have had very interesting or surprising points of view) The researcher would sacrifice some degree of internal validity (power to infer cause and effect, i.e., different programs caused different outcomes) because what was unique about an entire classroom cohort (and its teacher interactions and classroom climate) would coincide with a specific program, thus obscuring what actually caused observed changes in math performance —the program or the specific classroom context

The student could reduce the number of planned interviews to a more manageable number

The researcher could configure a quasi-experiment using intact classrooms where different classrooms are randomly assigned to different programs, meaning that all students in a classroom experience the same program

Scoping and Shaping Your Research: Working Within Constraints (continued)

This strategy would create the need for more training and for procedures to ensure they conduct interviews in the desired manner (adds to research timeline), likely requiring extra resources to get additional interviewers ready (tough for postgraduates)

What does the researcher sacrifice or gain if trade-off is made or the opportunity is followed up?

The student could use more interviewers to help in collecting her data, or

What research tactic (i.e., trade-off) or emergent opportunity might be useful to ameliorate/circumvent it?

12.1 423

What constraint does the researcher face?

The researcher encounters gaps in years for some sampled hospitals in that data are missing for certain indicators for certain years

What is the researcher’s ideal goal?

A researcher uses a publicly available secondary health services database to study yearly trends, across a 10 year time period, in various performance indices for a sample of public hospitals

Table 12.1 (continued)

Failure of hospitals to make data publicly available on certain indicators and/or problems with updating and maintaining the database itself

How might the constraint come about?

The database itself imposes the constraint (or perhaps the people responsible for maintaining it), but ultimately, the hospitals whose data comprise the database determine exactly what data they are willing to make public and how accurate such data are

Who or what might impose the constraint?

This would alter the generalisability of the research findings and would leave the researcher having to explain what differences there might be between hospitals that reported complete data and hospitals that didn’t—this could be a tough explanation to come up with If an alternative data source could be found, this could improve the researcher’s situation a bit, but would create either the need to demonstrate that the data from the alternative source can be traced to the same hospitals as those from the original database or the need to argue that complementary indicators from the alternative source can be defensibly used as proxies for the data from which values were missing in the original database

The researcher could decide to sample only hospitals that have complete data over the time period required, or

The researcher could see if another database might provide more complete or complementary data

(continued)

Missing data estimation is a contentious area and requires some assumptions to be made; the researcher will have to argue harder to make the study convincing

What does the researcher sacrifice or gain if trade-off is made or the opportunity is followed up?

The researcher could consider using specialised statistical techniques to estimate the missing data values, or

What research tactic (i.e., trade-off) or emergent opportunity might be useful to ameliorate/circumvent it?

424 12 How Do I Scope, Shape and Configure My Research Project?

What constraint does the researcher face?

The researcher is denied access to certain meetings because discussions include highly strategic commercial-in-confidence matters

What is the researcher’s ideal goal?

A researcher undertakes a participant observation investigation, guiding by an interpretivist pattern of assumptions, of the CEO and senior executives of a large bank. She is interested in understanding how the CEO builds and maintains relationships with the senior executives

Table 12.1 (continued)

This could work quite well for the researcher, but would make data analysis and reporting much harder as great care would need to be taken to avoid referring to or building interpretations based on commercial-in-confidence discussions. Accepting this limitation would mean the researcher would have to be transparent about the gaps in interpretations created by the inability to observe these meetings. This strategy could also work well for the researcher, but would rely on meeting attendees’ openness and accuracy of recall about relationships and behaviours within the meeting during the interviews and the researcher’s own ability to avoid treading into commercial-in-confidence territory

The researcher could agree to sign a non-disclosure agreement covering any strategically sensitive information in return for being able to observe during these meetings, or

The researcher could accept this constraint and delimit findings accordingly, or

The researcher could see if there are other ways to gather at least some information about these meetings by interviewing individual attendees after each meeting about CEO-senior executive relationships and behaviours during the meeting (steering clear of commercial-in-confidence topics altogether)

The CEO may impose this constraint, perhaps as a condition imposed on the researcher in exchange for continuing to have access to the company for her research

A highly competitive banking industry may make the company very cautious in what is revealed to an outsider

(continued)

What does the researcher sacrifice or gain if trade-off is made or the opportunity is followed up?

What research tactic (i.e., trade-off) or emergent opportunity might be useful to ameliorate/circumvent it?

Who or what might impose the constraint?

How might the constraint come about?

12.1 Scoping and Shaping Your Research: Working Within Constraints 425

The consultant does not use an appropriate strategy for engaging the teachers while acknowledging their fears

In the end, this could be advantageous for the consultant if the teachers develop greater ownership of the process and its outcomes which could also enhance the chances of successfully changing school practices; on the downside, the consultant would sacrifice some capacity to use his expertise to facilitate change and to influence the direction the process takes

Teachers resist working with the consultant because they fear he will not understand or appreciate their particular situation and judge their competence in the classroom harshly as a consequence

The consultant finds it very difficult to engage the teachers in working together to improve their approaches to dealing with ‘tough’ students

A teaching development consultant with action research expertise is brought into a school to work with teachers to develop more effective and safer ways of dealing with challenging student behaviours

For either or both constraints, the consultant would need to use a more participatory research approach, where teachers play a stronger role in examining and changing their own practices, with the consultant sitting more in the background (i.e., the consultant does not take advantage of his ‘consultant’ status and becomes more of a process observer/facilitator)

This opportunity would provide a partial amelioration of the problem, by providing access to at least some perspectives of former managers; the researcher would need to be careful about clarifying the situation of the three interviewees compared to other former managers that would have been missed (e.g., addressing why the interviewees were at the workshop and not other former employees?)

An oppportunity emerges for the researcher when he encounters three former middle managers of the company at a weekend leadership development workshop program and all three agree to be interviewed

The Director of the HR Dept. makes the decision not to give out the names in order to comply with the relevant privacy legislation

Privacy legislation may forbid the company giving out any details on former employees, under penalty of a hefty fine if a violation or complaint is reported

The Human Resources Department of the company will not give out names of formerly employed middle managers, so that the researcher cannot approach such people for interviews

A researcher wants to interview current middle managers of a company as well as former middle managers, who no longer work for the company

Teachers create this constraint as a consequence of their fears and/or

What does the researcher sacrifice or gain if trade-off is made or the opportunity is followed up?

What research tactic (i.e., trade-off) or emergent opportunity might be useful to ameliorate/circumvent it?

Who or what might impose the constraint?

How might the constraint come about?

What constraint does the researcher face?

What is the researcher’s ideal goal?

Table 12.1 (continued)

426 12 How Do I Scope, Shape and Configure My Research Project?

12.1

Scoping and Shaping Your Research: Working Within Constraints

427

Of course, it would be preferable if you controlled all the scoping and shaping that occurs for your research, but that will not always be the case. Here are some very general questions you can ask yourself as you begin to scope and shape your research project: • Who has the power to influence what I do in my research, why do they have that power and how should I manage this? • What if I want to do A, but am denied access to or lack resources to do it? • What do I have to put in place to ensure that A works effectively in my research? • If I do A, are there likely to be any unintended consequences? • Should I try A out before I implement it fully, in order to see if it is feasible? • Am I aware of any outside influences, expectations or pressures on my research and, if so, what should I do about them? • Are there any institutional, cultural, ethical or legal requirements/expectations associated with my research that I should know and do something about? • If I have to do A, do I have the skills and resources to implement it? • Who else stands to gain or lose from the research I plan to conduct? • Who might be interested in the research I do and why? • What are the implications and trade-offs I need to worry about if I do C instead of A?; • Can I compensate for problems I anticipate in implementing A by including an additional stage in my research where I do B (i.e., can a pluralist approach help)? and/or • Who do I need to convince with respect to what I learn through my research and how can I convince them that my research is of high quality? … As a postgraduate student, you will typically have a relatively short time period within which to complete your research and relatively few resources that you can access. If you are studying by distance education, or are an employed and/or married student, you may also have family, community and/or job-related constraints to contend with. This means that research configurations involving multiple stages, in-depth ethnographic participation or long-period longitudinal components will often be less feasible for your purposes, especially at the master’s level. Pluralist approaches to your research, while often desirable, must be carefully planned in the context of your time and resource constraints. Research guided by interpretivist/constructivist patterns of assumptions will generally take longer to carry out (and data analysis will generally take longer to complete) compared to research conducted under the positivist pattern. Research guided by the positivist pattern of assumptions, on the other hand, requires much more up-front and in-depth planning, which reduces the amount of time for study execution. Big budget projects requiring multiple researchers or research assistants are not generally feasible for postgraduate research because the research must be your own, for assessment purposes. Very large sample sizes are probably also out of reach in most circumstances. You may have to settle for convenience samples (see Chap. 19 ) to gather the data you require, and you will just have to wear the limitations this will impose on your capacity to generalise your results, if that is your intention.

428

12 How Do I Scope, Shape and Configure My Research Project?

Appropriately scoping and shaping your research project means that: your research must be feasible to complete as an academic or professional doctorate exercise within the rules of your university; it must fit within your life space in the time you have to complete the project, and it must be something that you are committed and motivated to complete. Your job is to work within those constraints to configure and execute the most convincing study you can. One thing that can help you build convincingness is to use your research journal to record the trade-offs you make along the way (and why) in order to make your research project feasible to complete, while retaining the highest quality possible, within the constraints you have to work under. This record can then help you write the story of your research journey in such a way that the reader is very clear as to how and why you made the trade-offs you did and why you limited your research scope in the ways that you have. If there is one important lesson at this stage, it is to learn when to let go. Your research problem is like a garden—you want to nurture it to achieve its full potential, but you need to be careful with what you do and don’t plant, and you need to control its size and weed and prune it carefully—otherwise you either just grow an increasingly entangled mess or it doesn’t grow at all. Your supervisor(s) can help you tend this garden.

12.1.1 Research Scoping and Shaping Choices There are a range of concrete scoping and shaping choices you can make that will help appropriately focus your research activities on your research problem and emergent research questions/hypotheses more sharply. Making these choices earlier on in the planning of your research journey will help you to make more sensible and feasible choices downstream when you are deciding upon how best to configure your research and when choosing data gathering strategies and data sources. If you devote conscious attention to each choice and record your thinking in your research journal, this will help you not only to plan your research more effectively and understand the demands, opportunities and constraints created by each choice but also to defend your choices and their associated implications when producing and disseminating your research outcome(s). In many instances, these scoping and shaping considerations will cross-influence/inform each other, meaning that making one choice may limit or free up other choices. Furthermore, certain scoping and shaping choices may create implications for your research configuration. What data type(s) should I gather? This choice concerns the type(s) of data you think will be most relevant, desirable or workable to have access to: quantitative data (counts, indicators, indices, measurements, instrument readings) or qualitative data (which includes written, spoken, recorded or symbolised text, symbols, images, multi-media files, web-pages) or both. As we argued in Chap. 9, contrary to popular thinking, the choice of data type does not necessarily dictate or reflect your pattern of guiding assumptions. While

12.1

Scoping and Shaping Your Research: Working Within Constraints

429

many researchers believe that positivist assumptions demand quantitative data and interpretivist/constructivist patterns of guiding assumptions demand qualitative data, these are false equivalences. Remember that quantitative and/or qualitative data may be useful irrespective of one’s guiding assumptions; it is how you deal with and makes sense of such data that is influenced by your guiding assumptions. In positivist research, quantitative data are required in order to utilise the tools of statistical analysis to build meaning from the data; accordingly; qualitative data can be coded and categorised so as to facilitate their use in statistical analyses. In research guided by interpretivist/constructivist patterns of guiding assumptions, qualitative data are used to help construct meaning and understanding; quantitative data can be used, typically in count rather than measurement form, to reveal patterns of emphasis and meaning. Importantly, as we have argued earlier, the logic of mixed methods research, as commonly implemented, encompasses configurations of data gathering strategies that must yield both quantitative and qualitative data, but this is, in itself, an artificial and misleading constraint (in this sense, ‘mixed data type’ research would be a more accurate label than ‘mixed methods’ research). What is important is that you deliberately choose which type(s) of data you will need to be most convincing in your research; don’t be constrained by a label imposed by others to classify such research. In this light, you can envisage pluralist research where different data gathering strategies are used to collect data all of one type (either qualitative or quantitative), different data gathering strategies are used to collect data of different types or the same data gathering strategy is used to collect both types of data. Any combination is possible if it can lead to a convincing research story. What is the genesis of the data I need: primary, secondary or created? This choice concerns whether you will actively collect data, generate/create data or use pre-existing data gathered by someone else. This choice goes to the original genesis of the data to be gathered for your research (note we are using data gathering in the broadest sense to reflect how you obtain the data needed). If you choose to actively collect your data, you put in place the methodological steps needed to gather primary data directly from your data sources. If you choose to generate or create data, you put in place the methodological steps needed to create primary data from scratch using parameters and assumptions you specify (as with a mathematical model or simulation). If you choose to use pre-existing data, you put in place the methodological steps needed to obtain primary data that have been actively collected or produced by someone else, usually for other purposes. For your own research purposes, this means you will be using secondary data. The active data collecting and use of pre-existing data choices do not necessarily signal or reflect a specific pattern of guiding assumptions and may encompass quantitative and/or qualitative data in the context of almost any research frame. However, data generation using mathematical models or simulations is most consistent with the positivist pattern of guiding assumptions and always produces quantitative data; thus being most relevant to the Explanatory Research frame. Technically, this is data creation, rather than data gathering, but in the broadest sense, data generation is

430

12 How Do I Scope, Shape and Configure My Research Project?

simply another way for you to assemble the data needed to carry out your research. The critically important thing to note about this choice is that when you pursue a data generation approach (for example, with Monte Carlo research, systems dynamics research or agent-based modelling research; see the Generative data-shaping strategy in Chap. 14), you must be absolutely open and crystal clear about all aspects of your model building and data generation processes. If all aspects of the model building and data creation processes are clearly presented along with all the assumptions that underpinned their generation, then your research is ethically above board and can lead to convincing outcomes. Note that in research frames where qualitative data have been gathered via recorded interviews, you may transcribe the interview into a written format prior to analysis; this transcript then technically constitutes ‘created’ data (although ‘transformed’ data might be more technically appropriate than ‘created’ data). In this case, the data creation/ transformation process lies intermediate between primary data gathering (i.e., the recorded interview) and the final transcript that serves as the form of data used in analyses. If all aspects of the transcription process are made clear, this too becomes an unproblematic issue. If, however, you are not open and clear about your model and data generation/ creation processes in whatever research outcome you disseminate, you leave yourself open to accusations offalsifying your data. In many cases, this lack of clarity will be an oversight on your part, one that is certainly correctable, rather than deliberate falsification of data. In other cases, data falsification may be intentional and a number of researchers have been caught out doing just this, which constitutes a serious and potentially career-destroying violation of researcher ethical responsibilities (for example, visit http://retractionwatch.com/ and http://blogs.discovermagazine.com/ neuroskeptic/2015/01/20/how-diederik-stapel-became-fraud/#.VjayA7crLVZ to see some stories about researcher fraud involving faking of data and results). Am I pursuing externalised causal inferences? This choice concerns whether you intend to exert some or complete control over the research context as a pathway for pursuing externalised causal inferences, that is, inferences about generalisable cause and effects relationships that are external to the specific data sources sampled. This scoping and shaping choice is explicitly linked to and reflective of the positivist pattern of guiding assumptions. However, it may also be consistent with the critical realist pattern of guiding assumptions. Interest in causation is more likely to be translated into convincing outcomes within the Explanatory research frame. Interest in external causation creates ripple effects throughout your research journey in that you need to worry about and put into place mechanisms for control over extraneous causes. This will have implications for how you will appropriately manipulate causal constructs so that you can measure effects, your measurement processes, your sampling processes, your research configuration and for your choices of analytical techniques. How central is time to the questions I want to address? This choice concerns whether mapping the passage of time or specific time periods against data patterns and relationships is critically important for you to achieve.

12.1

Scoping and Shaping Your Research: Working Within Constraints

431

Often, this choice will be associated with making external causation inferences. If your interest is in examining change, evolution or development over time or the impact of an event or intervention through time (e.g., comparing the situation before it happened to the situation after it happened) or some other variation on this focus, then the passage of time becomes a central point of reference for the research. This choice does not signal or reflect any specific pattern of guiding assumptions and we will see that there are specific ways you can configure your research if time is central to your focus. For certain research frames (e.g., Action research, Evaluation research or Developmental Evaluation research), change over time is essential to the research goals within the frame. In other frames (e.g., Case Study research, Survey research or Explanatory research), mapping change over time may or may not be relevant to certain research questions/hypotheses within that frame. We need to be clear here that we are not talking about the passage of time being the causative factor in whatever is observed to happen; what is important is to map changes that are observed onto specific points or periods in time. For example, if we use a pre-test-post-test type of research configuration, we do not say that the pre-test causes what we observe on the post-test occasion, it is whatever happens between the pre-test and post-test occasions that is important to understand and map. Time is just the vehicle onto which change is ‘piggybacked’. The upshot of this scoping and shaping choice is that, if time is indeed central to your research, a longitudinal research configuration will be the most viable option to consider. If you do research where, for example, it takes 6 months to conduct all of the interviews you plan or you conduct your research in distinct phases (as in a sequential research configuration), this does not mean that time is central to the research. In the end, if your research story will involve talking about what happened before, during and after or earlier compared to later or this year relative to last year and so on, then time is central. How should I resolve the breadth versus depth issue? This choice concerns the degree to which you intend to pursue breadth of learning and/or depth of learning in your research. Breadth of learning tends to be more-surface level in focus and often implicates access to large samples of data sources. Depth tends to involve a longer and more intense connection/relationship with your research context and/or each of your data sources, which tends to mitigate against using larger samples. Often breadth and depth trade-off against each other in that to pursue one, you tend to sacrifice the other. It is possible, however, to pursue both breadth and depth in research, but this will generally mean a longer and more involving research process, a pluralist approach and a more complicated research configuration. While there is no necessary link between the breadth/depth choice and pattern of guiding assumptions, there is somewhat of a tendency for positivist research to have breadth as a focus (particularly within the Survey research, Descriptive research and Explanatory research frames) and interpretivist/ constructivist research to have depth as a focus (particularly in the Exploratory research, Explanatory research and Indigenous research frames).

432

12 How Do I Scope, Shape and Configure My Research Project?

What do I intend for the extensional reach of my conclusions? This choice concerns your intention for what is learned in the specific circumstances of your research to have meaning/relevance beyond the boundaries of the specific research context (and its associated data sources and data gathering activities) or to have meaning/relevance just within the boundaries of that context. The meta-criterion, extensional reasoning, is therefore explicitly relevant to this scoping and shaping question. This choice is closely linked to different patterns of guiding assumptions. If your intention is to generalise your findings to a larger population of data sources or to other related tasks and activities that data sources might experience in other contexts, this is more consistent with the positivist pattern of guiding assumptions. However, if your intention is to show how what you learn in one context may or may not be transported (i.e., have meaning for/relevance) to other contexts, this is more consistent with patterns of interpretivist/constructivist guiding assumptions. Extensional reach may be attempted in a number of directions: • toward other people who were not participants in the research; • toward other groups within an organisation, community or society; • toward other organisation, community or society contexts outside of the research context; • toward other times or occasions besides those relevant to the research context; • toward other cultures besides the culture of the research participants; • toward other tasks besides those undertaken as part of research participation; and/or • toward other experiences of the same or similar event or circumstance. Extensional reach is an aspirational as well as a strategic choice in any research frame: aspirational because you may entertain hopes that your results have relevance to, usefulness for and/or meaning beyond the specific research context or participants; strategic because realising extensional reach aspirations requires a cascade of subsequent and specific methodological choices, in terms of sampling, contextualisation and configuration. By its very nature, extensional reach is a specific kind of forecasting argument; one that may or may not be directly supported by the research evidence. You can configure your research to enhance the chances of successfully arguing the extensional reach of what is learned (e.g., by using multiple case studies, rather than a single case study; by using larger more representative samples in positivist research, by following participants through time), but, in the end, it is the research user who will judge the credibility of such arguments as an aspect of convincingness. Do I need technological support? This choice concerns whether it is important for you to employ technological systems to support specific aspects of your research. Technological support for research encompasses physical devices as well as computer software and hardware.

12.1

Scoping and Shaping Your Research: Working Within Constraints

433

• Devices for recording data in various contexts and for various purposes: – digital recorders/software for recording interviews, field notes and research journals; for example: LiveScribe (https://www.livescribe.com/au/) for recording talk while writing notes Evernote (https://evernote.com/) for general journaling purposes – recording devices for observational research; for example: Noldus (http://www.noldus.com/), which offers a range of audio/visual data recording software systems for behavioural research (e.g., Observer XT, FaceReader) – video cameras for audio-visual recording; and/or – instruments for recording physiological or physical data. • Technology for secure and robust data storage, including password-controlled cloud storage platforms; for example: – Google Drive (https://www.google.com/drive/); – DropBox (https://www.dropbox.com/); – Microsoft OneDrive (https://onedrive.live.com/about/en-au/). • Computer software to support quantitative data analysis activities; for example: – SPSS (https://www-01.ibm.com/software/au/analytics/spss/) for general and comprehensive statistical analysis and graphics; – SAS http://www.sas.com/en_au/home.html); – AMOS (http://www-03.ibm.com/software/products/en/spss-amos) for structural equation modelling; – NCSS (https://www.ncss.com/) for general and comprehensive statistical analysis and graphics – SYSTAT (https://systatsoftware.com/) for general and comprehensive statistical analysis and graphics; – Mplus (https://www.statmodel.com/) for building and testing multivariate models; – Statistica (http://www.statsoft.com/Products/STATISTICA-Features) for general and comprehensive statistical analysis and graphics; the company that produces this software package also offers a publicly-available electronic statistics textbook (http://www.statsoft.com/Textbook); – Stata (http://www.stata.com/) for general and comprehensive statistical analysis and graphics;

434

12 How Do I Scope, Shape and Configure My Research Project?

– R (https://www.r-project.org/) for general and comprehensive statistical analysis and graphics on an open-source (free) platform; – eViews (http://www.eviews.com/home.html) for general and comprehensive statistical analysis and graphics, oriented toward econometrics and financial research; – Excel (https://products.office.com/en/excel) for general spreadsheet capabilities along with some graphical and data analysis capabilities; XLStat (https:// www.xlstat.com/en/) is an add-on statistical package for Excel that can be purchased). • Computer software to support qualitative data analysis activities; for example: – MAXQDA (http://www.maxqda.com/) for general and comprehensive qualitative data preparation, analysis, visualisation and integrated ‘mixed methods’ tools for qualitative/quantitative analysis (multimedia data, images and recordings can also be handled); – NVivo (http://www.qsrinternational.com/) for general and comprehensive qualitative data analysis, visualisation and integrated ‘mixed methods’ tools for qualitative/quantitative analysis (multimedia data, images and recordings can also be handled); – Dedoose (http://www.dedoose.com/home/features) web-based system for general and comprehensive qualitative data analysis and visualisation (multimedia data, images and recordings can also be handled); – Atlas.ti (http://atlasti.com/) for general and comprehensive qualitative data analysis and visualisation (multimedia data, images and recordings can also be handled); – The Ethnograph (http://www.qualisresearch.com/) for general and comprehensive qualitative data analysis and visualisation; – HyperResearch (http://www.researchware.com/products/hyperresearch.html) for general and comprehensive qualitative data analysis and visualisation. • Computer software for producing/generating graphics and diagrams; for example: – PowerPoint (https://products.office.com/en-au/office-365-personal#) for general diagramming purposes; – Inspiration (http://www.inspiration.com/) for mindmapping, concept mapping and flowcharting; – NetDraw (https://sites.google.com/site/netdrawsoftware/home) for drawing social network diagrams; – SmartDraw (https://www.smartdraw.com/) for general diagramming purposes. • Computer software for producing text, presentations and displays for research outcomes; for example:

12.1

Scoping and Shaping Your Research: Working Within Constraints

435

– Word/Excel/PowerPoint (https://products.office.com/en-au/office-365personal#) for all authoring and presentation needs; – Origin (http://www.originlab.com/Origin) for extensive graphing capabilities; – Open Office (https://www.openoffice.org/) for all authoring and presentation needs; – Prezi (https://prezi.com/) for creating presentations. • Hardware/software/app platforms for managing automated or manual data collection: – online software for conducting web-based surveys, e.g. SurveyMonkey (https://www.surveymonkey.com/) Qualtrics (https://www.qualtrics.com/) – apps for collecting data via mobile phones and tablets, e.g. FastField Forms (http://www.fastfieldforms.com/); Fulcrum (http://www.fulcrumapp.com/); – software for conducting online and computer-based experiments, e.g. Sawtooth (http://www.sawtoothsoftware.com/) for designing and conducting online surveys and conjoint measurement experiments; PsyToolkit (http://www.psytoolkit.org/) for designing and conducting general online experiments. • For certain data gathering strategies where data must be generated (e.g., simulation and games, computational modelling), technological support, in the form of computer software, will be necessary to facilitate data generation and analysis processes; for example: – NetLogo (https://ccl.northwestern.edu/netlogo/) for agent-based modelling capabilities; – Insight Maker (https://insightmaker.com/) for free, online, open-source dynamic systems simulations and agent-based modelling research; – iThink (http://www.iseesystems.com/store/products/ithink.aspx) for dynamic systems simulation research; – Stella (http://www.iseesystems.com/store/products/ithink.aspx) for dynamic systems simulation research; – Vensim (http://vensim.com/) for dynamic systems simulation research; – Certain statistical platforms (e.g., SPSS, SYSTAT, Stata, eViews) offer Monte Carlo statistical simulation procedures.

436

12 How Do I Scope, Shape and Configure My Research Project?

• The Internet has emerged as an important technological tool for researchers but is a two-edged sword to be wielded carefully as the information accessed may vary greatly in quality and credibility. As we have seen, the Internet with associated online software, can be used to search for and provide access to various forms of published and grey literature, as well as access to various online databases. Social media platforms (e.g., Facebook, YouTube, Google+ , LinkedIn, SnapChat and Twitter, learning management systems (e.g., Moodle, Blackboard, Desire2Learn) as well as blogs and discussion boards set up by users can provide useful qualitative data for research. Whether you need to use technological support may depend upon your adopted pattern of guiding assumptions, your data gathering strategies and type of data gathered. For example, if you gather quantitative data under the positivist pattern of guiding assumptions, you will most likely need some sort of computer software to support your data entry and analysis activities. You may also need devices to help record observations. If you are gathering qualitative data under an interpretivist/ constructivist pattern of guiding assumptions, you may need a digital recorder to capture interview data and software support for qualitative analysis. Technological support choices will almost always create additional resourcing needs for you. Resources required may be financial in that the desired technology may need to be purchased or a subscription commenced or time/training-oriented in that time and energy must be expended in learning how to competently use the technology. When deciding upon the adoption of a specific research support technology, you should consider: robustness, reliability and security of the technology, reputability and available support from the vendor or company (including what other researchers’ experiences have been), operating and computer system requirements (for software-based technology), possible institutional licensing and access issues and ease of use. For computer software, it is always worthwhile downloading a trial version to see if the software will meet your specific needs. For online data storage platforms, how secure your data will be and how access to them will be controlled become important questions to address. Am I replicating/extending previous research or breaking new ground (greenfield research)? Whether to explicitly build upon what others have done, that is, give a strong primacy role to positioning that emerges from the literature or investigate something new and fresh (so-called ‘greenfield’ or discovery-based research) is a strategic choice on your part. The adopted research frame and guiding assumptions may influence this choice as will your own positioning as researcher. Replication or extension takes a specific set of prior research largely as a given and carries out essentially the same research process with a new sample in a different context (replication) or with some improvements in focus or process (extension). Your research will largely be constrained by what previous researchers have done, which can make downstream decision making somewhat easier but also path-dependent in the sense that large variations from what has been done previously may change the

12.1

Scoping and Shaping Your Research: Working Within Constraints

437

essential character of your research, thereby defeating the purpose of replication or extension. Another type of extension involves you focus on filling a perceived gap in the literature or knowledge, in which case, you may take your cue from future directions for research proposed by others in their research outcomes or from your own reading of the literature. Here, you are less constrained by what others have done, because you are extending their research in new directions/new ways. Note that for postgraduate researchers, exact or nearly exact replication of someone else’s research is generally discouraged as it will not provide a sufficient enough platform for examiners to judge the convincingness of your achievements as an independent researcher. Breaking new ground in greenfield research takes existing literature as a starting point but does not allow that research to overly constrain what you do. You seek, instead, to learn something new, something that was previously unknown, usually in an area that has received little or no research attention in the past. On the one hand, this orientation may free you from previous constraints, but, on the other hand, it may also make the research configuration process rather more difficult to navigate, because you are essentially starting with a clean slate. This can make contextualisation and trialling of new strategies and data gathering processes much more important to incorporate into your research. In instances where you wish to test new or additional theoretical propositions or measurement processes in a previously existing area of research, the distinction between extension and greenfield may become rather blurred. Will my research be feasible/practical with the resources I have available? Many choices you make during your research journey will be constrained by available resources. Resource constraints create the ultimate test question for you when making scoping and shaping as well as configuration choices, namely “is what I want/plan to do in terms of navigating the ‘Data Triangle’ (sampling, data gathering, data analysis) feasible and practical given my resource constraints?”. Feasibility and practicality considerations can influence your choices made with respect to all other scoping and shaping questions. For example, lack of feasibility may mean that your desired extensional reach cannot be achieved (e.g., sampling or contextual limitations), that interest in external causation cannot be realised (e.g., lack of sufficient control over context) or that breadth may need to be sacrificed in favour of depth. It is important to realise that, in some cases and contexts, you will be aware of your desired pattern of guiding assumptions prior to making at least some of these scoping and shaping choices, and can therefore allow the pattern of guiding assumptions to influence your choices. However, in other cases, you may make these scoping and shaping choices first to help you settle upon the most desirable pattern of guiding assumptions. In this latter set of circumstances, guiding assumptions emerge from the pattern of scoping and shaping choices. Which approach is right for you cannot be prescribed beforehand; it will depend upon your own positioning as a researcher as well as upon the influences of relevant stakeholders, situational constraints and opportunities (time; money; gatekeeper expectations which govern the feasibility of access to data sources; ethical constraints) and contextual constraints (access to

438

12 How Do I Scope, Shape and Configure My Research Project?

voices, events, relevant information, data sources, expectations of those who control access to/use of research contexts). As we have discussed earlier, you, as a researcher, will not be the sole ‘scoper and shaper’ of your own research. A myriad of external influences must be balanced with your own positioning and needs, and this tension is what drives the necessity for making trade-offs that move you away from what you would ideally like to achieve toward what is practical and feasible for you to achieve. One useful strategy for research scoping and shaping is to work through some thought ‘experiment’ scenarios. This involves trying to anticipate what could happen if you make certain choices. Another aspect of this exercise would involve thinking about what you might do if certain of your choices are thwarted by things that emerge that are out of your control (e.g., organisations decline to participate; you don’t get ethical permission for your sampling plan, your questionnaire items create adverse effects for participants from a specific cultural background). As you work through these scenarios, you could record your thinking in your research journal and discuss your thinking with your supervisors. This will provide you with additional material for defending your choices when it comes time to write up your research. You may even find it useful for you and your supervisor(s) to arrange a scenario brainstorming session where you work together in trying to anticipate possibilities while recording thoughts on a whiteboard. This would allow you to draw on your supervisors’ expertise and experience while, at the same time, allowing your own creative thinking and awareness of the constraints you face to come to the fore. Scenario thinking can also help you to figure out what a Plan B might look like for your study—a plan you would use if critical or insurmountable obstacles are encountered to interfere with your initial or ideal choices.

12.2

Potential Configurations of MUs—A Unifying Framework

It is productive to think more systemically and holistically about how you can configure your research activities. This has the benefit of unifying what is normally considered to be a diverse set of paradigmatically-differentiated research ‘designs’. Our unifying approach builds upon, expands and extends the typology of core designs set out in Creswell and Plano Clark (2018, Chap. 3; however, we also draw upon the typology as set out by Creswell & Plano-Clark, 2011, in their previous edition). While that typology provides a useful platform to build upon and appear in our unifying framework in some form, the systemic restrictions imposed on them by ‘mixed methods’ logic, as traditionally depicted in the literature (e.g., Azorín & Cameron, 2010; Creswell & Plano Clark, 2018; Tashakkori & Teddlie, 2010) have been rejected. Therefore, our unifying approach is not artificially constrained to just mixing quantitative and qualitative data types in a research investigation, as required by mixed methods logic. Instead, we can use the unifying framework to show how particular choices of any data gathering strategies, data types and data sources may be combined/connected in specific patterns to create a complete research plan.

12.2

Potential Configurations of MUs—A Unifying Framework

439

From this point forward, we will avoid all reference to ‘research design’, which has been the traditional way to refer to patterns of connected data gathering activities. We avoid using the term ’design’ because it has positivist connotations surrounding the exclusivity of research control over context and, since our goal is to invite pluralist thinking with respect to patterns of guiding assumptions, this is too limiting. Instead, we use the more neutral term ‘research configuration’ to refer to the pattern and associated connections between activities undertaken when navigating the ‘Data Triangle’. Thus, configuring research activities means working out the pattern in which they are implemented. Depending upon the research frame and associated pattern of guiding assumptions, the configuration pattern may implicate all three facets of the data triangle: access/connect with your data sources, implement data gathering strategies and build meaning from data.

12.2.1 The Method Unit (MU) In our unifying framework, the Method Unit (MU) provides the fundamental building block for any research configuration. A MU is a conceptual entity comprising one data gathering strategy (e.g., self-report questionnaire, semi-structured interviews, participant observation, paradigm-determined pairing as with an experiment/quasi-experiment coupled with a self-report measurement scale—all to be discussed further in Chap. 14), one type of data (quantitative or qualitative), and one type of data source (undergraduate students, postgraduate students, employees, teachers, webpages, documents, cartoons, patients, nurses, secondary database, performances, shoppers, managers, children, community members …). Every research configuration can be conceptualised as a specific amalgam or patterning of MUs and those basic patterns can be classified into distinct configuration categories, each having their own advantages and disadvantages. It is also possible to create and implement hybrid combinations of different basic configurations. Research frames, various contextualisations, positionings and patterns of guiding assumptions have dynamic influences on choices of MUs and their configuration, which may help shape and delimit your available choices. The Single category of configuration is simply one MU, comprised however you choose. We symbolise the Single configuration as shown in Fig. 12.1 with its definition clearly emerging. This is the simplest most basic research investigation that can be conceived; what some researchers have referred to as ‘monomethod’

Fig. 12.1 The symbol for a single MU research configuration (with definition expanded), which will be used as the building block for all other research configurations

440

12 How Do I Scope, Shape and Configure My Research Project?

research (see, for example, Johnson & Onwuegbuzie, 2004; Spector, 2006). It is thus a non-pluralist configuration. Examples could include in-depth semi-structured qualitative interviews with a sample of teachers from metropolitan high schools, quantitative questionnaires administered to a sample of shoppers in a shopping mall or structured quantitative observations of children on a playground at a preschool. The biggest advantages of a single MU investigation are its simplicity and, in most cases, its resource (time, effort and money) efficiency. The biggest disadvantage is that convincingness, in the form of a well-rounded story, is more difficult to achieve. Too many questions are often left unresolved in single MU research. Consequently, it is becoming harder to defend the use of the single MU configuration in most research frames, even for postgraduate research. However, it should be noted that you are somewhat more likely to see the single MU configuration used in the Survey research frame, especially if the data gathering strategy is being used to simply gather one type of information from a specific constituency, often for very applied purposes such as decision- or policy-making, but occasionally for postgraduate research. For example, a local city council may use a short questionnaire to gather quantitative ratings of satisfaction with a range of council-provided services from members of the community. This information may then be used to help inform certain resource allocation decisions. Some types of laboratory experiment investigations, conducted within the Explanatory research frame, may use a single MU configuration to gather quantitative measurements from a sample of undergraduate student participants under specific experimental conditions. However, in more recent years, quantitative monomethod research guided by the positivist pattern of guiding assumptions (which historically was the prototypical approach) has come in for criticism, due to what has been termed ‘common method variance’, or bias contaminating research outcomes because the same strategy has been used to gather all quantitative measurements (see Spector, 2006).

12.2.2 Simultaneous Configuration The basic Simultaneous MU configuration involves implementing different MUs at the same time or nearly the same time. This configuration is sometimes classified as a ‘mixed methods’ approach, if you explicitly include multiple types of data or as a ‘triangulation’ (or ‘convergent’) approach if you use multiple data gathering strategies coupled with multiple types of data and/or types of data sources (see discussion in Creswell & Plano-Clark, 2018, Chap. 3). Generally, you carry out the data triangle activities within each MU independently of the other MUs (i.e., cross-influences between simultaneous MUs are minimised), so the focused stories arising from each MU have a chance to emerge. Some advantages of the Simultaneous configuration include: • resource (particularly time) efficiency in sampling, data gathering and analysis activities;

12.2

Potential Configurations of MUs—A Unifying Framework

441

• the opportunity to access a deeper or a broader level of learning in a short space of time; • the opportunity to pursue convergence in the stories arising from the different MUs; and/or • the opportunity to achieve cross-sectional (a ‘slice of life’ emphasis) learning, where appropriate for your research goals (especially where you have adopted the Survey or Exploratory research frame). Some disadvantages of the Simultaneous configuration include: • the configuration is much more challenging for a single researcher to implement if different patterns of guiding assumptions are used for each MU (a much broader skill set would be required of a single researcher); • convergence in the stories that emerge from each MU may not occur, leaving you with a more challenging integration to achieve; • the learning focus is episodic and cross-sectional in nature (focusing on a single time period (a ‘slice of life’ emphasis), rather than longitudinal), so this is not the most convincing configuration to use if change/growth over time is important to understand/measure; and • reduced usefulness if finding evidence for external causality is a goal of the research since temporal priority of cause before effect is much harder to arrange/ ensure/argue for in the Simultaneous MU configuration. The Simultaneous configuration can prove useful in: • the Survey, Descriptive or Exploratory research frames as a pathway to assembling a time-efficient cross-sectional story, through employing more diverse data gathering strategies and/or types of data and data sources. • the Case-Study research frame as a pathway to enhancing learning in the context of a single visit to the case context through employing more diverse data gathering strategies and/or types of data and data sources; or • the Cross-Cultural (Comparative) research frame, where it can be useful in facilitating an efficient comparison of samples from different cultural contexts using the same data gathering strategy (say, a questionnaire designed to have culturally equivalent forms). There are two variants of Simultaneous configuration possible: Single and Multiple. The choice of variant to use is generally informed by your resourcing/ feasibility constraints and research goals as well as by your choices on the breadth versus depth and extensional reach scoping and shaping questions. The Single Simultaneous configuration is more efficient in terms of resource demands, but favours breadth over depth of learning and offers limited extensional reach for generalisations. The Multiple Simultaneous configuration is more demanding in terms of your time, effort and money resources, but favours depth over breadth of learning and offers you the opportunity to achieve a more convergent and convincing story that has broader contextual/extensional reach in conclusions. For

442

12 How Do I Scope, Shape and Configure My Research Project?

either variant, implementing more than two MUs at the same time would be incredibly onerous and generally infeasible. Single Simultaneous configuration. The Single Simultaneous configuration involves using a single data gathering strategy in conjunction with multiple types of data sources and/or data types (see Fig. 12.2). Thus, it is basically a monomethod approach gathering quantitative and/or qualitative data from one or more types of data source. Three examples would be (a) where you administer a quantitative questionnaire to a sample of managers in a large organisation (MU1) and, at the same time, administer a similar quantitative survey to a sample of non-management employees from the same organisation (MU2); (b) where you conduct semi-structured qualitative interviews with teachers sampled from high schools in rural/regional Australia (MU1) and, at the same time, conduct semi-structured qualitative interviews with the parents of students sampled from high schools in rural/regional Australia (MU2) or (c) where you use a document-based archival sample of government reports to gather quantitative financial data (MU1) and, as well, use the same document-based archival sample of government reports to gather qualitative data, focusing on the language used in the reports (MU2). Multiple Simultaneous configuration. The Multiple Simultaneous configuration involves using multiple data gathering strategies in conjunction with multiple data sources and/or data types (see Fig. 12.3). Triangulation of different data gathering strategies (generally considered to be a stronger as well as more popular form of triangulation, see Jick’s, 1979, discussion of ‘method triangulation’; see also Flick, 2018) is one distinct advantage of this particular configuration. Two examples would be: (a) where you administer a quantitative questionnaire to a sample of employees in the manufacturing industry (MU1) and, in the same time period, conduct qualitative semi-structured interviews with a sample of senior managers within that manufacturing industry (MU2) or (b) where you conduct a study of emergency room procedures in a training simulation context using a sample of nurse practitioners to gather quantified behavioural observations (MU1) and, at the same, conduct qualitative focused interviews with a sample of emergency room doctors reflecting on specific issues associated with trauma management procedures (MU2).

Fig. 12.2 Single simultaneous configuration

MU1 MU2

Fig. 12.3 Multiple simultaneous configuration

MU1 MU2

12.2

Potential Configurations of MUs—A Unifying Framework

443

12.2.3 Sequential Configuration The basic Sequential MU configuration involves sequentially implementing different MUs but where mapping data patterns/meanings against the passage of time is not of explicit interest in your research. This configuration is also sometimes classified as a ‘mixed methods’ approach, if you explicitly incorporate both quantitative and qualitative data (Creswell & Plano Clark, 2018; Teddlie & Tashakkori, 2011). In this configuration, each MU stage involves a complete progression through the data triangle from sampling to data gathering to analyses. The outcomes from the analyses in the previous MU(s) then provide input into the next MU stage. Some advantages of this configuration include: • learning from earlier research stages can provide feedforward to later stages to help shape and refine what is done and perhaps how; • different stages may be guided by different patterns of guiding assumptions, allowing the weaknesses of one pattern to be offset or mitigated by the strengths of another pattern; and • a single researcher can feasibly cope with a sequential MU configuration. Some disadvantages of this configuration include: • it is more time and resource-intensive; • generally, analysis of data gathered in the first MU stage must be completed before the second MU stage commences (if learning from the first MU is to help in shaping what is done in the second MU) adding to the timeline for your research; and • integrating stories between stages can be more challenging, especially where patterns of guiding assumptions differ for each stage. The Sequential configuration may be useful in a range of research frames: • the Explanatory research frame where you are explicitly interested in demonstrating theorised external causation or where grounded theory is to be constructed under an interpretivist/constructivist pattern of guiding assumptions; • the Exploratory research frame where earlier exploratory MU phases set the stage for subsequent deeper exploratory phases; • the Survey research frame where a quantitative questionnaire MU stage is followed by semi-structured qualitative interviews; • the Action research frame where each MU phase embodies an aspect of the cycle of action learning intended to change or transform practices/behaviours in a specific context; • the Indigenous research frame where early MU stages can provide you with the foundations for building trust to pursue deeper learning in later MU stages; or • the Transdisciplinary research frame or Developmental Evaluation research frame where MUs at different stages target learning at different levels from

444

12 How Do I Scope, Shape and Configure My Research Project?

different constituencies or perhaps from different parts of a chain of innovation (from Research & Development to innovation adoption). There are four variations of Sequential configuration: Deep Learning, Explanatory, Exploratory and Programmatic/Accumulative. The choice of variant to use is generally informed by your resourcing/feasibility constraints, research goals and research frame as well as by your choices on the breadth vs depth and extensional reach scoping and shaping questions. Note that Sequential MU configurations need not be restricted to just two MUs, but each additional MU will demand more time and resources. Deeper Learning Sequential configuration. In the Deeper Learning Sequential configuration, the goal is to extend, expand and enhance learning achieved from one MU to the next, irrespective of the types of data involved (see Fig. 12.4). Such learning may influence how subsequent stages are carried out, what questions are asked, what data sources are sampled and what contexts/events should be observed (in short, how movement through the data triangle in MU2 should unfold to enhance learning). In terms of scoping and shaping, the Deeper Learning configuration seeks depth of learning through the sequencing of the two or more MUs. Two examples would be: (a) where you facilitate qualitative focus groups with product consumers (MU1) followed by qualitative interviews with specific product development managers and advertisers (MU2), or (b) where you carry out unstructured qualitative interviews with primary curriculum development specialists and academic subject matter experts (MU1) followed by semi-structured qualitative interviews with teachers sampled from primary schools (MU2). Explanatory Sequential configuration. In the Explanatory Sequential configuration (see Fig. 12.5), the goal is to gather qualitative data in the second MU to help explain, qualify or further characterise patterns and relationships uncovered in the quantitative data gathered in the first MU (see Creswell & Plano Clark, 2018, pp. 75–83). This is one of only two MU configurations where the type of data gathered in each MU is specified. Essentially, in this configuration, the different types of data are gathered strategically to achieve a specific research goal. Here, your goal is retrospective in focus, namely MU2 is undertaken to help flesh out/ unpack/qualify/amplify/further understand the conclusions, especially anomalous, confusing or unexpected findings, from MU1. One benefit of the Explanatory Sequential configuration is your ability to seek breadth of learning in MU1, then

Fig. 12.4 Deeper learning sequential configuration

MU1

MU2

Fig. 12.5 Explanatory sequential configuration

quant

qual

MU1

MU2

12.2

Potential Configurations of MUs—A Unifying Framework

445

depth of learning in MU2. Two examples could include: (a) where you administer a quantitative questionnaire to a stratified random sample of community members (MU1) followed by conducting a series of qualitative interviews (MU2) with community members sampled from the participants who completed the questionnaire in MU1 or (b) where you employ systematic observations of assembly line workers to obtain quantitative measurements of work efficiency and safety in a large manufacturing firm (MU1) followed by qualitative focus group interviews (MU2) with groups of employees meeting to discuss general problems and issues discovered in MU1. Exploratory Sequential configuration. In the Exploratory Sequential configuration (see Fig. 12.6), your goal is to use learning from qualitative data in MU1 to identify and inform the constructs, patterns and relationships of interest in the quantitative MU2 (see Creswell & Plano Clark, 2018, pp. 84–93). This is the second of the two MU configurations where the type of data gathered in each MU is specified. Here, your goal is prospective in focus, where MU1 is undertaken to produce learning intended to shape the focus in MU2. Two examples could include: (a) where you conduct qualitative focus group interviews with consumers interested in purchasing organic and sustainably produced foods (MU1) followed by a quantitative questionnaire, designed using input from what was learned from the focus groups in MU1, administered to a random sample of shoppers at various supermarkets and specialist food shops in shopping malls in a metropolitan city (MU2) or (b) where you examine a sample of historical text documents providing qualitative commentary and reflections on major decision mistakes made by CEOs, politicians and world leaders (MU1) followed by a laboratory experiment (MU2), conducted using postgraduate student volunteers as participants, designed to yield quantitative measurements of decision making behaviours under conditions arranged to simulate those encountered in specific historical instances explored in MU1. Programmatic/Accumulative Sequential configuration. The Programmatic/ Accumulative Sequential configuration (see Fig. 12.7) is a pattern of MUs that reflects a planned series of investigations which successively vary specific characteristics of a research context in order to further tease out, qualify or extend meanings, patterns and relationships that are being explored (similar to the ‘multiphase design’ in Creswell & Plano Clark, 2011, pp. 100–104). Here is a configuration that is purpose-built for systematically accumulating knowledge through a series of linked investigations. Under the positivist pattern of guiding assumptions,

Fig. 12.6 Exploratory sequential configuration

Fig. 12.7 Programmatic/ accumulative sequential configuration

MU1

qual

quant

MU1

MU2

MU2

MU3

446

12 How Do I Scope, Shape and Configure My Research Project?

this configuration might comprise a series of incremental investigations where specific causal variables are systematically manipulated and tested in each MU in order to carry out more well-rounded and thorough tests of theoretical propositions. Under an interpretivist/constructivist pattern of guiding assumptions, this configuration might embody a ‘grounded theory’ approach, where a new theoretical perspective is built up using a series of MUs. Two examples would be: (a) where you employ a general quantitative questionnaire (Survey research frame) to tease out some general theoretical propositions that can then be further tested (MU1) followed by a more formal laboratory experiment (Explanatory research frame) designed to test those emergent propositions using undergraduate university students (MU2) followed then by a quasi-experiment (Explanatory frame again) designed to test the theoretical propositions in a field setting (MU3) or (b) where you conduct a series of qualitative semi-structured interviews with dairy farmers about their views on innovation needs in their industry (MU1) followed by qualitative semi-structured interviews with research and development scientists in a major farming innovation company working on a new dairy farming innovation to meet dairy farmer needs (MU2) followed by qualitative focus groups (MU3) with potential and actual dairy farm innovation adopters, evaluating (e.g., would it work? does it work? meets your needs? cost effective? changes needed? etc.) an innovation once it has been released into the market for testing after incorporating the feedback and learning from both MU1 and MU2, followed by follow-up focus groups with actual dairy farm innovation adopters once the company had further modified the innovation in response to what was learned during MU3 (MU4; note that this entire sequence of investigations could be a pattern of research done within the Developmental Evaluation research frame). The Programmatic/Accumulative Sequential configuration is also quite useful in the Action research frame where deliberate change planning, implementation and evaluation for learning purposes unfold in a series of MUs; learning from each MU feeds into the next MU in the sequence.

12.2.4 Hierarchical Configuration The basic Hierarchical MU configuration involves one MU being defined as primary, major or dominant and other MUs as secondary, minor, supportive or subordinate. The basic configuration was set out by Creswell and Plano Clark (2011, pp. 90–96) and its goal is to support what is to be learned/what is being learned/ what has been learned in the primary MU with data gathering using a strategically chosen second (or more) MU. This helps you to round out the research story you are constructing in such a way as to enhance convincingness. In Creswell and Plano Clark’s (2011) intentions for this configuration, the primary MU focuses on quantitative data and the secondary MU focuses on qualitative data or vice versa. Here, this constraint does not exist; the configuration can be just as viable using two

12.2

Potential Configurations of MUs—A Unifying Framework

447

MUs that gather the same type of data, but using different data gathering strategies and/or focusing on different types of data sources. Some advantages of this configuration include: • it is resource efficient in terms of time, effort and finance, in that all data are gathered simultaneously or nearly simultaneously (i.e., the secondary MU can be implemented before, during or after the primary MU); • it seeks to provide triangulated stories through diversifying MUs, thus extending what might be achieved with a Simultaneous configuration; and • it may encourage enhanced participation and participation retention through deeper involvement of participants in the data gathering processes that unfold in what amounts to a single data gathering stage. Some disadvantages of this configuration include: • it is not an effective configuration for drawing inferences about externalised cause and effect, since temporal precedence of cause before effects cannot be assured; • since primary and secondary MUs differ in emphasis, the weaknesses of one cannot be fully offset by strengths in the other; • this configuration would be challenging to implement using different patterns of guiding assumptions for each MU; and • there may be incommensurabilities between what is learned from the dominant MU and what is learned from the subordinate MU, which may create ambiguities that are difficult to resolve. The Hierarchical configuration may be useful in a range of research frames: • the Survey research frame, embedding one type of questionnaire (e.g., qualitative open-ended questions) inside of another (quantitative rating scales) or vice versa in order to provide supportive triangulation data; • the Descriptive or Exploratory research frames, where a secondary MU is employed to tap into supplemental, often textual or other documentary data sources beyond those involved in the primary MU in order to achieve a more well-rounded story; • the Indigenous or Feminist research frame, where a secondary MU is used to implement an alternative data gathering strategy to that used in the primary MU in order to better connect with certain sub-groups in a sample (e.g., children, elders); or • the Transdisciplinary or Developmental Evaluation research frame, where a secondary MU uses semi-structured interviews to more deeply probe the views of a specific sub-group of participants, such as community leaders or research scientists, in support of the primary MU of undertaking structured interviews with samples from different community contexts (e.g., business, farming and agricultural, public transport).

448

12 How Do I Scope, Shape and Configure My Research Project?

There are two basic variations of Hierarchical configuration: Embedded and Supplemental. The choice of variant to use is generally informed by resourcing/ feasibility constraints and research goals as well as by your choices on the breadth vs depth and extensional reach scoping and shaping questions. In general, with respect to resourcing issues, the Hierarchical configuration implements an intentional imbalance in resourcing: more resources to the primary MU, fewer to the secondary MU. The choice of secondary MU is then made in terms of achieving maximum value in supportive learning within the context of fewer allocated resources. Embedded configuration. The Embedded configuration is a pattern of MUs where a secondary MU is embedded or nested within the primary MU (see Fig. 12.8). The secondary MU may be implemented before, during or after the primary MU (usually contiguously), along the lines of arguments presented by Creswell and Plano Clark (2011, p. 75). The Embedded configuration is generally oriented toward achievement of depth rather than breadth of learning, allowing you to dig deeper within the context of the primary MU and is somewhat more resource-efficient relative to the Supplemental configuration because of its contiguous data gathering focus. Three examples include: (a) where you employ a quantitative questionnaire (MU1) that also contains a small number of qualitative open-ended questions (MU2) (a common practice within the Survey research frame, which often produces a side-benefit of enhancing response rates due to inviting participants to share their own views); (b) where you conduct an experiment gathering quantitative measurements (MU1) and closely follow up those measurements with qualitative focused interviews focusing on participants’ perceptions of their experiences in the experiment (MU2) (useful in the Explanatory research frame, for instance); or (c) where you conduct a qualitative participant observation study (MU1) in which you also conduct qualitative interviews with specific participants (MU2) (useful in a grounded theory investigation conducted within the Explanatory research frame, for instance). Supplemental configuration. The Supplemental configuration is a pattern of MUs where a secondary MU is not embedded within the primary MU; rather it is implemented separately to it (see Fig. 12.9). The Supplemental configuration is generally oriented toward achievement of greater breadth rather than greater depth of learning, allowing you to gather data outside the context of the primary MU. Fig. 12.8 Embedded configuration

MU1 MU2

Fig. 12.9 Supplemental configuration

MU1 MU2

12.2

Potential Configurations of MUs—A Unifying Framework

449

Two examples would be: (a) where you administer a short quantitative questionnaire within a specific workplace (MU2) as a prelude to selecting employees to participate in the main study involving semi-structured qualitative interviews about their workplace experiences (MU1) (useful in the Exploratory research frame, for instance); or (b) where you gather text evidence in the form of annual reports, media releases and other documents relevant to a Board of Directors and its decisions within a specific company (MU2) in addition to carrying out a participant observation study of meetings of that Board of Directors (useful in a Exploratory research frame, for instance).

12.2.5 Case-Based Configuration The basic Case-based MU configuration focuses on a defined entity or event (the ‘case’, which may be a group, an organisation or institution, a community or a focal event such as a natural (e.g., earthquake, flood or fire) or man-made (e.g., plane or spacecraft accident, military invasion or incursion, terrorist action) disaster, sporting or community event or economic crisis) in its natural context. In this configuration, contextualisation of data gathering is of primary importance—the case provides the focal context and the MU focuses on data gathering within and with an understanding of that context. As Yin (2014) discussed, the focus of data gathering may reside at the level of the case as a whole, yielding a ‘holistic’ case study, or at the level of units of analysis contained with the case context, yielding an ‘embedded’ case study. Embedded units of analysis may comprise one or more different groups (e.g., different classrooms, different departments, enrolments in different courses, work teams, social groups), different levels within an organisational hierarchy (e.g., senior executives, managers, teachers, students, nurses, doctors, line workers, union officials) and/or different events (e.g., team meetings, emergencies, new staff inductions/indoctrination, ritual events such as retirements, graduations). In an embedded case study, the case context forms a backdrop against which data for the different units of analysis may be gathered and interpreted; what is learned from the different units of analysis is therefore foregrounded against the background of the general case context. In a holistic case study, some aspects of data gathering may focus on contextual influences outside of the case context itself, lending a wider systems focus to your research. For example, a holistic case study of a specific school may gather data within the school that speaks to wider systemic concerns (e.g., government funding, school system regulations and policies, demographic and economic conditions of the geographic area in which the school resides and so on). Irrespective of whether you are conducting a holistic or embedded case study, one important implication of the Case-based configuration is that you are obligated to gather contextual background information about the case in parallel with or prior to implementing the MU. This contextual learning and backgrounding may be achieved through activities such as reading case-related documents (e.g., historical documents, annual reports, minutes of public meetings, media stories), accessing

450

12 How Do I Scope, Shape and Configure My Research Project?

websites (perhaps even staff biographies) associated with the case context, informal discussions with key people in the case context. In some instances, what you learn from this contextual background information may help shape what you do in the MU and, perhaps, who important data sources would be. [Note that if the contextual backgrounding activities are planned and undertaken as more formalised and structured data gathering activities, rather than as the more informal learning activities implied here, they become their own MU, meaning that the Case-based configuration becomes hybridised with multiple MUs in play (see discussions of the Case-based Simultaneous, Case-based Sequential and the Case-based Hierarchical/Supplemental configurations below).] Some advantages of the Case-based configuration include: • it is explicitly oriented toward providing depth of contextualised learning; • it facilitates the interpretation of data patterns through reflecting on those patterns in light of contextual circumstances in the case; and • it is particularly well-suited for research guided by an interpretivist/ constructivist pattern of assumptions as you, as the researcher, often become embedded in the case during the research journey (however, this does not mean you cannot adopt the positivist pattern of guiding assumptions in the Casebased configuration; it is just less common). Some disadvantages of the Case-based configuration include: • it is generally a more resource-intensive and challenging form of research, especially with respect to your effort and time; • writing the story of a case study investigation in a research outcome is generally more challenging because contextualisation needs to be thorough and sensitively utilised to make sense of data patterns and interpretations; and • unless you study multiple cases, transportability of learning/meaning to other contexts (under an interpretivist/constructivist pattern of assumptions) or generalisability of relational patterns to a larger population or other contexts (under the positivist pattern of guiding assumptions) is hard to convincingly argue (combatting this disadvantage is what gives rise to the Multiple Case-based configuration variation to be discussed below). The Case-based configuration may be useful in a range of research frames: • most centrally, of course, is the Case Study research frame, where your research goal critically depends upon your awareness, understanding and use of contextual nuances in the focal case; • the Descriptive or Exploratory research frame, where you may be seeking greater depth of learning in a specific context, but with an orientation toward providing stepping off points for further research; • the Evaluation research frame, where you might be evaluating a specific program in the context of one or more selected case organisations; • the Developmental Evaluation research frame, where you might seek to develop, modify and systematically evaluate an innovation in a specific case context (e.g.,

12.2

Potential Configurations of MUs—A Unifying Framework

451

in a hospital or other public or private organisation, community of farmers or an industry); • the Indigenous research frame, where you might seek to achieve an understanding of life within a specific Indigenous community; • the Cross-Cultural Research (cross-cultural learning orientation) research frame, where you might seek to understand life within a specific cultural context/ community, that is outside your own cultural context; and • the Explanatory research frame, where you pursue case-contextualised theory building or testing (as with grounded theory, for example, or where the case is a specific event in which the applicability of a theory might be tested). There are two basic variations of Case-based configuration: Single and Multiple. The choice of variant to use is generally informed by resourcing/feasibility constraints and research goals as well as by your choices with respect to the extensional reach scoping and shaping question. Extensional reach is really the pivotal question: do you intend or hope to point to transportable meanings/social constructions and/or make generalising statements about the relevance of learning to or in other contexts/situations? If the answer is no, then you would prefer the Single Casebased configuration; if yes, then you would prefer the Multiple Case-based configuration. The choice will primarily be a strategic one for you, but once made, will also have resourcing implications associated with it. Single Case-based configuration. The Single Case-based configuration involves a MU that focuses on a single case entity or event (see Fig. 12.10), where the yellow oval surrounding the MU signifies the contextual learning and backgrounding that you need to achieve as part of your research journey. This configuration is generally more feasible, from a resourcing perspective, especially where your research time is limited (as in a postgraduate degree). The drawback, of course, is that any transportable conclusions or generalising statements can only be logically and speculatively argued for (very challenging but not impossible), rather than empirically demonstrated. In many instances, making such statements will not be your intention; your goal instead would be achieving a deep understanding of that one particular case. Two examples would be: (a) where you administer an organisation-wide survey, measuring organisational climate and job satisfaction, to a sample of medical and technical staff employed in a large public hospital (positivist guiding assumptions would be reflected here); or (b) where you conduct qualitative semi-structured interviews of managers (guided by interpretivist assumptions) in a particular city council under threat of amalgamation with other city councils as a result of changes to state government policy (where the topical landscape of the interviews is informed by your contextualisation learning and backgrounding with respect to the city council’s context).

Fig. 12.10 Single case-based configuration

Case MU

452

12 How Do I Scope, Shape and Configure My Research Project?

Fig. 12.11 Multiple casebased configuration

Case1

Case2

MU

MU

Multiple Case-based configuration. The Multiple Case-based configuration involves a MU that focuses on multiple case entities or events (see Fig. 12.11), where each yellow oval signifies contextual learning and backgrounding that you must achieve for each distinct case context as part of the research journey. The resource demands for the Multiple Case-based configuration multiply with each case considered. Not only must data be gathered and analysed within each case context, but you will also need to make cross-case comparison analyses as well as carry out contextual backgrounding for each case context. A single researcher can effectively manage such a research project, including managing all data triangle activities, but the timeline for your project will be greatly lengthened and the drain on your personal resources (energy, motivation, cognition) will generally be considerable. If you take appropriate steps to ensure that the commonalities and uniquenesses of all case contexts are well-understood, you may be able to advance empirically defensible generalising statements or transportations of meanings/ constructions/patterns, drawing upon those commonalities with caveats and qualifications reflected in the uniquenesses. Two examples would be: (a) where you undertake qualitative focused interviews about changes in/impacts on job demands and stress with sworn and unsworn officers sampled from two police organisations in regional New South Wales which have recently been forced to downsize due to reallocations of state and federal government resources; or (b) where you undertake participant observations of classroom teaching, focusing on classrooms embedded in three schools sampled from a particular school district in a metropolitan city.

12.2.6 Longitudinal Configuration The basic Longitudinal MU configuration involves mapping/tracking data movements/patterns/meanings against the passage of time. The centrality of time is a key scoping and shaping question that leads to choosing this research configuration and is what primarily distinguishes this configuration from the Sequential configuration. In this configuration, the passage of time, whether relative to some intervention, event or program or not, becomes a critical aspect of contextualisation. It allows you to assemble coherent and convincing ‘before-during-after’-type stories (where a focal intervention, event or program is involved) or stories about change or development over time with respect to some context or social or economic process of interest. In some instances, predictions of future occurrences beyond the time window of the research may be pursued (as in certain forms of time series research). The important thing to note here is that in the Longitudinal configuration, time itself does not play a causal role (in that the passage of time itself does not cause change), it merely provides a linear path and signposts against which

12.2

Potential Configurations of MUs—A Unifying Framework

453

to map trajectories and events of change. The unit of analysis for the Longitudinal configuration may be an individual economic, social or financial indicator, a person, a group, an organisation or institution, or a nation, with the basic requirement, in most cases, being that the same unit is observed on each data gathering occasion. Some advantages of the Longitudinal configuration include: • temporal precedence of cause and effect can be ensured by observing causal events/behaviours/situations or by you, as researcher, manipulating them to occur before any observations of outcomes (the effects to be attributed to the previously observed or manipulated causes) are made, an arrangement that is important if understanding and making inferences about external causation of events is an important research goal (normally associated with the positivist pattern of guiding assumptions); • patterns of movement, changes or developmental evolution over time can be explicitly tracked/understood/observed; and/or • you can achieve greater depth of contextual immersion through repeated episodes of data gathering (may be important under an interpretivist/constructivist pattern of guiding assumptions and for research approaches such as grounded theory or data gathering strategies such as participant observation). Some disadvantages of the Longitudinal configuration include: • it is, potentially, a time and resource-intensive configuration to implement, especially where many data gathering episodes are involved or where repeated contextual immersion is involved (except in instances where you can access the time-based measurements or other data from a secondary database); • where repeated contextual immersion is involved for research guided by an interpretivist/constructivist pattern of assumptions, there is a risk you may lose your ‘researcher’ attitude and begin to adopt/reflect the perspectives of those you are sharing the context with (so-called ‘going native’; an especially potent risk when implementing the participant observation data gathering strategy); • depending upon the spacing of data gathering activities between occasions, you may not be able to monitor what goes on between the episodes of data gathering (which may result in obscuring potentially important aspects of learning about the phenomena being researched, especially where you are interested in external causal inferences under the positivist or critical realist pattern of guiding assumptions); • in cases where you wish to pursue external causal inferences (again under the positivist or critical realist pattern of guiding assumptions), it may not be possible for you to arrange or manipulate the intended causal conditions, which means that causal inferences may be more difficult to defend (for the experimental/quasi-experimental data gathering strategy, this problem is what produces a quasi-experiment); • where human participants are involved, there is a greater risk of participant dropout (i.e., ‘mortality’) from the research due to fatigue, boredom, changes in

454

12 How Do I Scope, Shape and Configure My Research Project?

life, work or health circumstances (the longer the timeline or the more data gathering episodes along the timeline, the greater the risk); and/or • in research that depends upon time-based data from secondary databases (e.g., a financial or economic database), there may be gaps in the data for specific time periods for specific units of observation (e.g., data for one year for a company or a school might be missing), which you will need to accommodate and deal with. The Longitudinal configuration may be useful in a range of research frames: • the Explanatory research frame, where you explicitly pursue an external causation story (under the positivist or critical realist pattern of guiding assumptions, giving rise to a configuration labelled as a repeated measures or time series design) or where deeper contextual immersion is intended to yield a theoretical account for what you have observed (e.g., grounded theory under an interpretivist/constructivist pattern of guiding assumptions); • the Action research frame, where you work with a specific group or organisation through a cyclical research process to facilitate and evaluate localised contextual changes/improvements and where mapping those changes over time is critical for causally linking planned change interventions to observed improvements as well as to possible side effects; • the Survey research frame, where you gather data from the same sample of participants on several occasions (a so-called ‘panel study’, which creates a longitudinal Survey research frame as distinct from a cross-sectional Survey research frame) or where you gather data from approximately equivalent (with respect to one or more focal characteristics, such as age or school grade) samples from the same population on each data gathering occasion (a so-called ‘cohort study’); • the Evaluation research frame, where you set out to evaluate the effectiveness of an intervention, treatment or program using, at a minimum, a pre-test (before undertaking the program or experiencing the intervention) and a post-test (after undertaking the program or experiencing the intervention) data gathering occasion (intermediate or follow-up data gathering occasions may also occur during or after undertaking the program or experiencing the intervention); in psychological, educational, medical (e.g., clinical trials) and other types of social and behavioural research pursuing external causal inferences regarding the effectiveness of an intervention, treatment or program, a comparative or placebo control group (which does not receive or experience the undertaking the program or experiencing the intervention or treatment) may be used to enhance the convincingness of the causal claims; • the Developmental Evaluation research frame, where you are tracking and gathering contextualised data on the development, modification, adoption and effectiveness of an innovation that necessarily unfolds over time and which may be linked to the seasonal or periodic occurrences of events (see, for example, discussions in Patton, 2011). There are three basic variations of Longitudinal configuration: Time-Aligned, Intervention Time-Aligned and Developmental Time-Aligned. The term

12.2

Potential Configurations of MUs—A Unifying Framework

455

‘time-aligned’ is meant to signal that data gathering episodes occur at (i.e., are aligned with) specific points along the timeline. The choice of variant to use is generally informed by your resourcing/feasibility constraints and research goals as well as by your choices on the external causation scoping and shaping question. In terms of strength of external causal inferences, the Intervention Time-Aligned configuration offers the strongest opportunities for convincingness, where the intervention, program or treatment is intended as the causal event and data are gathered, at a minimum, both before and after that event. (Note that when paired with one or more control groups and random assignment of patients to a control or treatment group, this configuration can become the gold standard randomised clinical trial (RCT) in medical research, although not without some reservations, see, for example, Kaptchuk, 2001). The Time-Aligned configuration is commonly employed in institutional (e.g., tertiary education institution research), economic (e.g., labour and productivity studies) or financial (e.g., stock market and organisational profitability research) as well as in sociopolitical systems research (e.g., marketing and social policy research such as tracking policy impacts or impacts of social or advertising interventions). In this configuration, the power of the story comes from many occasions (e.g., days, weeks or years) of data gathering, however, external causal inferences are more difficult to defend and may be tied to the length of the series. The Developmental Time-Aligned configuration is less concerned with mapping external causation over time and more concerned with addressing the evolution of people, groups, institutions or innovations over time, taking account of input and feedback from many different stakeholders at various points along the innovation development-adoption timeline. In this configuration, learning is an especially important focus and early learning can influence and shape what is done and what is learned later in time. Time-Aligned configuration. The Time-Aligned configuration involves the implementations of a MU over multiple, sometimes quite numerous, occasions over time (see Fig. 12.12). If, under the positivist pattern of guiding assumptions (the most common pattern of guiding assumptions underpinning this specific configuration), many time-aligned quantitative data measurements are gathered, this gives rise to a variation of this configuration called a ‘time series’ design (see, e.g., Cryer & Chan, 2008) where the spacing between data gathering occasions is intended to be approximately equal (e.g., days or years). Two examples include: (a) where you use secondary database records to track car accident occurrences attributable to mobile phone use, drink driving, drug use or speeding over a period of many months or years to provide data to inform the creation of new government legislation for penalties; or (b) where you undertake systematic daily observations in a classroom over the course of a semester to record teacher-student interaction patterns over time.

Fig. 12.12 Time -aligned configuration

Time MU1

MU1

MU1

MU1

456

12 How Do I Scope, Shape and Configure My Research Project?

Fig. 12.13 Intervention timealigned configuration

Time MU1

I

MU1

I

MU1

Intervention Time-Aligned configuration. The Intervention Time-Aligned configuration involves implementations of a MU over one or more occasions in time before, during and after some intervention of interest has occurred (see Fig. 12.13). The ‘intervention’ in the label for this configuration is intended to be a generalist term, encompassing events or episodic changes of human origin (e.g., a training program, drug or other clinical treatment, acquisition of an organisation by a competitor, terrorist attack, assassination of a president or prime minister, drink driving advertising campaign) or natural origin (e.g., flood, earthquake, drought) in the research context. Two occasions (one pre-intervention and one post-intervention) are the minimum requirement for this configuration if external causal inferences are to have some chance of being convincing and defensible. The external causal inference being pursued in this configuration is that whatever happened during the intervention is what causes any changes you observe between the pre- and post-occasions. A variation of this configuration could involve additional implementations of the MU within the period of an intervention or program as well. Iincluding such intermediate observations during the intervention may enhance your causal inference capability (often done in formative evaluations, for example), but would be more resource-intensive. If, under the positivist or critical realist pattern of guiding assumptions, many time-aligned quantitative data measurements are gathered which both precede and follow an intervention of some kind, this yields a variation of this configuration called an ‘interrupted time series’ design (see, e.g., Glass, Willson, & Gottman, 2008). If only a few data gathering occasions are involved (e.g., a pre-test and a post-test), the configuration would be called a ‘repeated measures’ design. Two examples would be: (a) where you employ a quantitative survey obtaining specific employee performance and attitude measurements before and after a corporate takeover of a company; and (b) where you carry out focused qualitative interviews of individual group members before and after each group counselling session they participate in with a specific group counsellor. Developmental Time-Aligned configuration. The Developmental Time-Aligned configuration involves implementations of a MU over multiple occasions in time where the goal is to track evolution over time in the context of incremental changes introduced to or made within the research context (see Fig. 12.14). In many cases, the configuration could be used in formative evaluation research within the Evaluation research frame or to track the development and modification of an innovation or program to meet the needs of a specific community, industry or target

Fig. 12.14 Developmental time-aligned configuration

Time MU1

MU1

MU1

12.2

Potential Configurations of MUs—A Unifying Framework

457

group of users/potential adopters in the Developmental Evaluation research frame (Patton, 2011). An example would be where you conduct group-based focus group interviews with dairy farmers in various dairy-producing regions of a state where groups identify innovation needs, that information is then fed back to the Research & Development division of a major agricultural manufacturing company who create and provide an innovation solution to the members of the groups to meet their identified needs for a specified period of time, the focus groups then reconvene to reflect on and evaluate the innovation and suggest modifications which are then fed back to the R&D division and so on (the time taken from innovation idea to successful realisation becomes one aspect of learning important in this configuration).

12.2.7 Hybrid Configuration A hybrid configuration refers to any more complex configuration of MUs that combines features of two or more of the above basic configurations. Hybrid configurations are becoming increasingly more commonplace in modern day research as researchers, research users, stakeholders, funders and sponsors seek more convincing stories. While a basic MU configuration may work well in many instances, a hybrid configuration may add substantive value to your research investigation. Some advantages of a hybrid configuration include: • it can generally enhance convincingness relative to what any one of its component basic configurations could offer on their own; • it can harness synergies between different configurations which may assist in offsetting the weaknesses of one basic configuration with the strengths of another configuration, including situations where different patterns of guiding assumptions are employed for each basic configuration; and/or • it may assist in achieving both depth and breadth in research scope, where one basic configuration may be oriented toward achieving breadth of learning and another basic configuration oriented toward achieving depth of learning; Some disadvantages of a hybrid configuration include: • most hybrid configurations are more resource and effort intensive to plan and implement in that the research will almost always take more time to plan and complete than a basic configuration would; or • when a single researcher is involved, as in postgraduate research, implementation could be very challenging, especially where your expertise in a particular facet of a hybrid configuration is weakest or, at the very least, requires development. Seven different hybrid configurations are discussed below. Each offers its own advantages and disadvantages, and each can be used to achieve very different research purposes. However, it is important to note that these seven hybrid

458 Fig. 12.15 Case-based— simultaneous configuration

12 How Do I Scope, Shape and Configure My Research Project?

Case MU 1 MU 2 MU 3

configurations should not be considered exhaustive of the possibilities; there is ample room for creativity in hybridisation depending upon your goals, research frame, contextualisations and positionings and as well as on your available resources. Case-based-Simultaneous hybrid configuration. The Case-based-Simultaneous configuration involves the implementation of multiple data gathering strategies in the same time period within a single focal case (see Fig. 12.15). The purpose of using multiple data gathering strategies, multiple data types and/or multiple data sources is to achieve a deep, rich, triangulated and well-rounded understanding of the case context and its people. However, this hybrid configuration provides more of a ‘slice of life’ window on the case, rather than depth through time, meaning that this configuration would not be very useful in change scenarios. In many single case studies, the minimal configuration would involve you analysing documents and texts relevant to the case context to gain a deeper understanding of the case context coupled with qualitative interviews of some type with people embedded in that context, often during a single visit to the case context. A concrete example would be where you administer a quantitative survey to teachers, conduct qualitative semi-structured interviews with the principal, vice-principal and head teachers and observe a parent-group meeting carried out simultaneously within a selected high school during a week-long visit. A variant of this hybrid configuration could synergistically combine the Multiple Case-based configuration with the Simultaneous configuration. A concrete example of this variant would be where you prepare, pre-test and administer cross-culturally equivalent quantitative questionnaire to workers combined with semi-structured qualitative interviews of managers or executives in two culturally different contexts (e.g., two schools, organisations or communities; one in China and one in Australia). This hybrid configuration may be useful in the following research frames: • The Case Study research frame is the most obvious frame for this configuration, where you employ different data gathering strategies (e.g., interviews, questionnaires, document analysis, participant observation) to obtain information (qualitative, quantitative or both) from different data sources within a single case study context; • The Descriptive research frame, where you seek to develop a more well-rounded descriptive picture of a situation, event or sample, including contextual information;

12.2

Potential Configurations of MUs—A Unifying Framework

459

• The Exploratory research frame, where you seek to undertake a thorough exploration of a new or unfamiliar case context through gathering data from different sources, perhaps as a prelude to further research; • The Cross-Cultural (comparative orientation) research frame, where you implement the Multiple Case-based Simultaneous configuration within two culturally different case contexts as described above; • The Transdisciplinary research frame, where you connect with various stakeholders (e.g., community members, local government, business owners, agricultural producers) using various data gathering strategies, in order to establish strategies for managing policy or natural resource impacts in an applied learning case context (e.g., a local farming community); or • The Indigenous or Feminist research frames, where you connect with a variety of data sources, some of whom might be gatekeepers (e.g., tribal elders, senior management) for gaining access to other people in the case context (e.g., an Indigenous community, a public service organisation such as the police, a school) perhaps using different data gathering strategies, in order to construct a more complete perspective or learn as much as possible about relationships and issues. Case-based-Sequential hybrid configuration. The Case-based-Sequential hybrid configuration involves multiple data gathering strategies that are sequentially implemented within a focal case with subsequent MUs building on what is learned from previous MUs (see Fig. 12.16). This hybrid configuration uses multiple data gathering strategies, multiple data types and/or multiple data sources to build up a deep, rich, increasingly focused (through the sequencing of MUs) and well-rounded understanding of the case context and its people. An example would be where you implement an organisational document analysis (MU1) to learn about the case context and the recent background history of a failed takeover bid by a larger competitor, then conduct a focus group interview (MU2) with departmental managers in the case context, discussing the failed takeover bid and its impacts on the case organisation, then using what has been learned from the focus group, construct and administer a quantitative questionnaire focusing on job security perceptions to a representative sample of employees (MU3). A variant of this hybrid configuration could synergistically combine the Multiple Case-based configuration with the Sequential configuration. This could be illustrated in the above example if you add two more case study organisations, each surviving a recent takeover bid in order to build up a more transportable (i.e., cross-case) set of theoretical arguments to explain what happened in each case. The Case-based-Sequential hybrid configuration may be useful in the following research frames: Fig. 12.16 Case-based— sequential configuration

Case MU1

MU2

MU3

460

12 How Do I Scope, Shape and Configure My Research Project?

• The Case Study research frame illustrated in the failed takeover bid example presented earlier; • The Action research frame, where you effectively become a joint participant, in conjunction with relevant groups in a specific case context (e.g., a school, community group or business), in action learning cycles intended to improve some process or outcome, where there is a MU focused on specific stages in the cycle: diagnosing the problem/formulating a solution and implementing/ evaluating/reflecting on that solution; • The Explanatory research frame, where you wish to implement an explanatory sequential configuration within one or more case contexts; or • The Indigenous or Feminist research frames, where you might need to undertake a small-scale research investigation in order to build trust and good will for gaining deeper contextual access for further data gathering within a specific community or institution. Case-based Hierarchical/Supplemental configuration. The Case-based Hierarchical/Supplemental hybrid configuration where the supplemental MU2 focuses on data gathering from larger contexts surrounding and perhaps influencing/impacting the case context that provides the primary data gathering focus for the primary MU1 (see Fig. 12.17). This hybrid configuration employs multiple data gathering strategies, multiple data types and/or multiple data sources, but augments the research in such a way as to provide a larger, more systemic and contextualised understanding of the case, perhaps connecting with relevant stakeholders outside of the case context but who have an interest in what happens within the case context. An example would be where you conduct semi-structured interviews with managers within a specific case context focusing on new government regulatory requirements as well as accessing and analysing external documentation focusing on the new regulatory requirements and what their implications are for the case context. One variant of this hybrid configuration could synergistically combine the Multiple Case-based configuration with the Hierarchical/Supplemental configuration. This would be illustrated in the above example if you added another case study organisation, conducting semi-structured interviews in each as well as accessing external case-related documents. Such a configuration would facilitate making cross-case comparisons/contrasts with respect to the implications of the new regulatory requirements for each case. A second variant could blend the Single or Multiple Case-based-Simultaneous or Sequential configuration with one or more Hierarchical MU elements, yielding richer data both within the case context as well as from the larger contexts in which the case is embedded. Fig. 12.17 Case-based hierarchical/supplemental configuration

12.2

Potential Configurations of MUs—A Unifying Framework

461

The Case-based Hierarchical/Supplemental hybrid configuration may be useful in the following research frames: • The Case Study research frame illustrated in the new government regulatory requirements example presented earlier; • The Transdisciplinary research frame, where you conduct participatory action research within a case context, but also connect with key stakeholders outside the case context including those who may be in a position of power to influence events and perceptions within the case context (here is where the second variant described above could be useful); or • The Exploratory research frame, where you wish to round out as well as deepen the exploratory learning from a case study, by seeking out additional contextual information outside the case. Longitudinal—Simultaneous hybrid configuration. The Longitudinal— Simultaneous hybrid configuration emerges when simultaneous configurations of MUs are repeatedly implemented over time, perhaps to track the impact of an event or change occurring between or during implementation occasions (see Fig. 12.18). This hybrid configuration can be used for panel studies where a sample of people is followed through time to track development or change, but with the additional advantage of using multiple data gathering strategies, multiple types of data and/or multiple data sources on each occasion to enhance triangulation. An example would be where you administer a quantitative questionnaire to employees in a company and conduct qualitative interviews with managers in the company at four different times of the year (each after quarterly company reports have been submitted), to track changes in organisational climate and leadership perceptions following the appointment of a new CEO. The Longitudinal—Simultaneous hybrid configuration may be useful in the following research frames: • The Action research frame, where you participate in several cycles of action learning with one or more groups, each time using MUs that focus on different stages of the cycle (e.g., focus groups for the diagnosis and planning stages; participant observation for the implementation and evaluation stages), and where tracking how the change or solutions evolve over time is important; • The Evaluation research frame, where you are interested in conducting a formative evaluation of a program over time (while the program is being implemented), using different data gathering strategies and tapping into different data sources; • The Cross-Cultural (Comparative) research frame, where you implement MUs, focusing on different cultural groups as data sources on several occasions, Fig. 12.18 Longitudinal— simultaneous configuration

Time MU1

MU1

MU1

MU2

MU2

MU2

462

12 How Do I Scope, Shape and Configure My Research Project? Time

Fig. 12.19 Longitudinal— case-based configuration

Case

Case

Case

MU

MU

MU

perhaps as a way of studying the impact of a major historical event (e.g., such as a change in global economic conditions) on the different groups; or • The Developmental Evaluation research frame, where you tap into several different data sources (e.g., research and development scientists, community members and consumers, workers/producers) on multiple occasions along the evolutionary timeline for a specific product or social innovation. Longitudinal—Case-based hybrid configuration. The Longitudinal—Casebased hybrid configuration involves an MU being repeatedly implemented (perhaps even continuously implemented) in the context of a specific case (see Fig. 12.19). For example, you might undertake specific short periods of participant observation of a local city council over several occasions as the council works toward amalgamation with another local city council. It is quite straightforward to envisage a more complex form of this hybrid configuration where multiple simultaneous MUs are implemented. For instance, a simultaneous multiple MU variant applied to the local city council context example above might include some focused qualitative interviews with senior council managers as well as the amassing of documents and field notes on each occasion as separate sources of data. The Longitudinal—Case-based hybrid configuration may be useful in the following research frames: • The Case Study research frame, where you wish to track/understand changes over time within one or more specific cased contexts; • The Action research frame, where you participate in several cycles of action learning with a group in a specific case context, each time using MUs that focus on different stages of the cycle (e.g., focus groups for the diagnosis and planning stages; participant observation for the implementation and evaluation stages), and where tracking how the change or solution evolves over time is important; • The Developmental Evaluation research or Transdisciplinary research frame, where you wish to focus data gathering strategies on processes and stakeholders implicated in the development, evolution, adoption and use of one or more innovations within a specific case context (e.g., within a specific industry or community); • The Exploratory research frame, where you wish to explore a new case context and to achieve greater depth of understanding by following the case over time, perhaps as a way of exploring the evolution and dynamics of the case context; • The Explanatory research frame, where you plan to build up and/or test a set of theoretical accounts or explanations for how a case context develops/changes over time, perhaps because of or in parallel with events/changes in within the case context itself or in one or more larger context(s) within which the case is embedded; or

12.2

Potential Configurations of MUs—A Unifying Framework

463

Time

Fig. 12.20 Longitudinal— hierarchical configuration

MU1

MU1

MU1

MU2

MU2

MU2

• The Indigenous or Feminist research frames, where you wish to examine the dynamics over time in an indigenous case context, perhaps in concert with an increase in depth focus over time, or in dynamics over time within a case context, but from a critical feminist perspective. Longitudinal—Hierarchical hybrid configuration. The Longitudinal— Hierarchical hybrid configuration involves embedded (or supplemental) hierarchical MUs being repeatedly implemented across time (see Fig. 12.20). The advantage of this specific configuration is that it allows you to primarily map out/ understand changes over time or to tease out cause-effect relationships (by controlling for temporal priority in obtaining cause and effect information) while, at the same time, providing you with additional participant-focused contextual information to help round out or qualify the emerging story. For example, you might undertake a panel study using quantitative questionnaires for measuring attitudes and preferences for products and product attributes that also contain open-ended questions focusing on personal views about products and their attributes, administered on several occasions, to the same sample of consumers, with the intention of testing specific theoretical propositions by ensuring that potential causal factors are measured before any measurements of effects are obtained. The Longitudinal—Hierarchical hybrid configuration may be useful in the following research frames: • The Survey research frame, where you undertake a panel study as described above (a popular research approach taken by marketing and organisational behaviour researchers, for example); • The Evaluation research frame, where you are interested in conducting a formative evaluation of a program over time (while the program is being implemented) using a primary data gathering strategy (with a specific associated data type, quantitative or qualitative) and a secondary data gathering strategy (often in conjunction with the other data type, qualitative or quantitative); or • The Developmental Evaluation research frame, where you tap into several different data sources (e.g., research and development scientists, community members and consumers, workers/producers) on multiple occasions along the evolutionary timeline for a specific product or social innovation, but where some data gathering strategies and/or data sources have greater priority of focus for you.

464

12 How Do I Scope, Shape and Configure My Research Project?

Time

Fig. 12.21 Longitudinal— sequential configuration

MU1

MU2

MU3

MU1

Longitudinal—Sequential hybrid configuration. The Longitudinal—Sequential hybrid configuration involves different MU implementations at specific points along a timeline, perhaps to track the evolution of a change process with increasing depth or breadth of focus (see Fig. 12.21). The advantage of this hybrid configuration is that it facilitates the progressive inward (or outward) focusing on some phenomenon of interest, using what is learned from previous stages to influence what is sought after or undertaken in subsequent stages. Two examples could be (a) where you carry out a quantitative choice experiment with a sample of consumers to determine the most important features to include in a new product that a manufacturing company plans to develop followed by conducting focused interviews with R&D scientists who then develop that product (after receiving feedback on what was learned from the choice experiment) followed by administering a short quantitative questionnaire to a large sample of consumers to assess their satisfaction with the product and its features after its release; or (b) where you carry out general participant observations within a specific organisation followed by semi-structured interviews with key people within that organisation (where the focus/content of each interview is suggested by analyses of the earlier participant observations) followed by more focused observations of specific events and interpersonal interactions within the company (this is a common type of research trajectory for grounded theory research, guided by an interpretivist/constructivist pattern of assumptions). The Longitudinal—Sequential hybrid configuration may be useful in the following research frames: • The Action research frame, where you, in conjunction with a focal group or groups, employ a distinct MU for every stage of the action learning cycle to diagnose (e.g., focus group discussion), plan and implement (e.g., participant observations), evaluate (e.g., quasi-experiment) and learn (focus group discussion) about a change process or problem solution; • The Evaluation research frame, where you carry out a combined formative/ summative evaluation of a new training program, employing different MUs at different stages of the evaluation process (e.g., focused interviews with trainers and potential trainees prior to final program design and implementation; quantitative measurement of key desired outcomes for trainees from the program immediately prior to its commencement (pre-testing); measurement of milestone desired outcomes for trainees midway through the program (intermediate testing); measurement of final desired outcomes at conclusion of the program (post-testing); focused interviews with trainers and trainees reflecting on the program after its conclusion; and measurement of desired outcomes six months after program completion (follow-up testing to assess program impact duration);

12.2

Potential Configurations of MUs—A Unifying Framework

465

• The Developmental Evaluation research frame, where a researcher undertakes an innovation- or product-focused research trajectory like example (a) above; or • The Cross-Cultural Research (cross-cultural learning orientation) research frame, where you employ several different but linked MUs intended to facilitate a deeper understanding of a specific cultural context through a period of change and, with appropriate feedforward from earlier MUs, provide a foundation for one or more subsequent MUs (e.g., conducting qualitative semi-structured interviews with key people in the focal cultural group early in the change process where what is learned influences the nature and content of a quantitative questionnaire intended for a larger sample from that cultural context focusing on the change process itself and where what is learned from the questionnaires helps to identify which participants to interview and what to focus on in those interviews after completion of the change process). (Note that a similar configuration logic could be applied within the Indigenous or Feminist research frames.)

12.2.8 Evolutionary Configuration An evolutionary configuration is more complex and open-ended configuration that employs a planned initial MU during which, through which or as a consequence of, a critical choice point emerges for undertaking one or more additional unplanned MUs (Creswell & Plano-Clark, 2018, Chap. 3, refer to such configurations as ‘emergent’). The term ‘evolutionary’ is intended to suggest that the research configuration evolves as circumstances and complexities dictate, meaning that you maintain maximal flexibility and adaptability in tailoring the configuration to emergent research needs, constraints and opportunities. Some advantages of an evolutionary configuration include: • it provides maximal research flexibility and agility in that signals and new directions that emerge from the first planned stage can be followed up as appropriate in subsequent stages; • it can take advantage of emerging opportunities such as tapping into new data sources that become available during the course of the research; • it can adapt to emerging constraints or changing circumstances, such as an organisation or group withdrawing its participation part through your research; and/or • it is responsive to/dependent upon learning in that what has been learned during the first MU helps you to decide and plan how to scope and shape the next MU. Some disadvantages of an evolutionary configuration include: • it presents greater challenges and uncertainties associated with planning and coordinating the implementation of the MU components, since only the first MU is scoped, shaped and planned;

466

12 How Do I Scope, Shape and Configure My Research Project?

• it may create a greater sense of uncertainty for you, knowing that the entire research journey cannot be fully planned; • it creates a greater dependence upon systems thinking in order to more effectively conceptualise and capitalise upon a research pathway that will maximise your learning; and/or • it may be more challenging to integrate learning holistically across the evolutionary configuration, because learning from the first MU could overly narrow or constrain the focus for planning the next MU (i.e., premature convergence). There are two types of evolutionary configuration are discussed below: the Adaptive/Flexible configuration and the Developmental configuration. What influences the choice of configuration is the source of pressures that create the choice point, namely whether the choice point emerges predominantly as a result of external influences on the research journey or predominantly from learning from previous stages of the journey (i.e., internal influences). The former situation is reflected in the Adaptive/Flexible configuration and the latter situation is reflected in the Developmental configuration. Adaptive/Flexible configuration. The Adaptive/Flexible configuration involves an MU choice point being created as a consequence of an unanticipated opportunity, constraint, situation or event or new data source that you become aware is available (see Fig. 12.22). Here, predominantly external forces create the choice point which offers multiple optional pathways to travel down and you navigate that choice point by considering what would best fit your research needs and resource capabilities. Such flexibility is consistent with the concept of bricolage (see, e.g., Kincheloe, 2005) and embeds a pluralist complexity perspective. Three examples would be: (a) where you administer a quantitative questionnaire focusing on organisational climate to a sample of employees, but then an opportunity arises where you can access and interview some ex-employees about their perceptions of the climate of the organisation they have just left; (b) where you are conducting climate change management research within a community suffering from drought and after conducting initial focused interviews with community members, farmers, local government managers and research scientists, it becomes clear that you should engage in data gathering from farmers and their families (who are claiming that climate change is affecting their access to water) because their needs are most immediate and urgent to address; or (c) where you, while conducting preliminary qualitative interviews with staff in a school, become aware of two upcoming events in that school, a staff meeting and a student awards presentation evening and decide to conduct participant observations of the awards presentation evening, because it

Fig. 12.22 Adaptive/flexible configuration

MU1

? ?

MU2 MU3

12.2

Potential Configurations of MUs—A Unifying Framework

467

could potentially lead to a wider range of learning opportunities from teachers, students and parents. The Adaptive/Flexible evolutionary configuration may be useful in a several research frames: • The Exploratory research frame, where you, going into an unknown or little known area of research or situation, carry out an initial planned stage of data gathering, following which unanticipated opportunities emerge which create a critical choice point offering several possible pathways for you to take and explore next (here you must consider which option offers the greatest promise for expanding/extending the preliminary knowledge base being assembled as well as meeting stakeholder needs and try to avoid those pathways likely to be blind alleys, tangential or of minimal benefit); • The Developmental Evaluation research frame, where you implement an initial planned stage of data gathering, focusing on innovation in complex contexts and circumstances, and the resulting learning coupled with larger contextual dynamics (reflecting the confluence of internal and external forces) creates a complex choice point offering several pathways for where, how and on whom to focus next (this configuration is largely consistent with Patton’s, 2011, conceptualisation of developmental evaluation and is often the way things play out in practice, but this configuration is accompanied by a greater sense of uncertainty in terms of which pathway would be most productive to pursue and with whose interests such pursuit would be most congruent); • The Transdisciplinary research frame, where you implement an initial planned stage of data gathering with key stakeholders in a specific issue or problem and the complex nexus of circumstances and competing stakeholder expectations creates a critical choice point offering multiple possible pathways to follow with subsequent research activities (here you must attend to many voices and sources of input, while avoiding disciplinary constraints/expectations) in order to choose the most productive pathway to follow; or • The Indigenous research frame, where you implement a participatory focus group-type of data gathering process with community elders to learn about the important traditions, issues and concerns of the community and then negotiate with those elders to gain access to and carry out participant observations within the community to see how those traditions, issues and concerns play out over time, under the condition that a tribal elder must accompany you at all times while in their community gathering data. Developmental configuration. The Developmental configuration involves an MU choice point emerging as a consequence of what was learned from MU1 (see Fig. 12.23). Here, predominantly internal forces create the choice point and you navigate that choice point by considering what would best suit the evolving Fig. 12.23 Developmental configuration

MU1

?

MU2

468

12 How Do I Scope, Shape and Configure My Research Project?

trajectory of your research needs while providing the best opportunities for further learning. An example would be where you commence your investigation with participant observations in a community and, following on from your preliminary analysis and interpretations of those observations, decide to pursue a set of semi-structured qualitative interviews with key people, identified during the participant observation phase, where the intention of the interviews is (a) to test out some of the preliminary interpretations from the participant observation MU and (b) probe for additional information that would help to flesh out and extend those interpretations and provide guidance for subsequent data gathering activities (a grounded theory approach). The Developmental evolutionary configuration may be useful in a several research frames: • The Developmental Evaluation research frame, where you implement an initial planned stage of data gathering, focusing on innovation in complex contexts and circumstances, and the resulting learning creates a choice point for where, how and on whom to focus next (this configuration is most consistent with Patton’s, 2011, conceptualisation of developmental evaluation where downstream MU choices depend upon earlier MU choices and what they have yielded and in what contexts—building up a coherent chain of logic); • The Explanatory research frame, where you implement an initial planned MU with the intention of building up an explanatory theoretical account (i.e., a grounded theory approach) but refrain from planning subsequent research activities until preliminary interpretations from this initial MU have been achieved, whereupon you make a choice as to how best to proceed to build upon and extend that learning in terms of theoretical sampling and/or more selective or focused data gathering and context choices (Charmaz, 2014); or • The Action research frame, where what you have learned during the initial planned action learning cycle creates a critical choice point that motivates changes in direction, strategies and focus of the research for another action learning cycle (navigating this critical choice point usually involves a group decision-making process).

12.3

Making Trade-Offs

As we mentioned earlier in this chapter, scoping, shaping and configuring social and behavioural research is all about making trade-offs, where you give away, modify or augment some valued angle, aspect or component of your research in order to incorporate another angle, aspect or component that you value in a different way. Here, though, we refine and fine-tune that discussion, by allowing that some trade-off decisions may result in more convincing research outcomes, but generally not in the form that you initially envisaged. Many trade-offs (call them Loss trade-offs) are conscious decisions you make to give away or modify an aspect of

12.3

Making Trade-Offs

469

the research that you would ideally or initially like to implement but, because of constraints and emerging obstacles, becomes unrealistic or infeasible to accomplish. Therefore, Loss trade-offs move your research away from its initial ideal form toward a more feasible, practical and realistically achievable form, often (but not always) sacrificing some convincingness in the process. Other trade-offs (call them Gain trade-offs) involve conscious choices you make to evolve or augment your initial research contextualisation and/or configuration so that additional or different and perhaps more convincing set of valued outcomes might be achieved. When either a Loss or Gain trade-off is made, there will almost always be implications for research quality criteria and meta-criteria that you will need to acknowledge and manage; sometimes convincingness will be enhanced, sometimes it will be degraded. With any trade-off, you need to understand what you lose as well as what you gain by making the decision. You also need to consider whether potential downstream issues will be created by the trade-off and, if so, take steps to address those issues, where possible. The conditions that demand Loss trade-offs generally arise because, in social and behavioural research, humans (researcher role) study humans (participant role) or the artefacts and systems humans produce and inhabit and what happens to and with those humans is dependent upon larger circumstances, many of which cannot be predicted or anticipated. You might wonder why we talk about making Loss trade-offs when research is supposed to involve lofty and objective goals and doing anything that would detract from its ideal conduct would be harmful. The answer resides in the very humanness of research activity. Researchers must live, work, prosper, survive, have careers, have families and friends and so on as part of their everyday lives (the researcher role being just one of many roles you enact in your life) and research activities can be influenced by those other aspects of life as well as by aspects of the lives of others. Thus, your research cannot be fully compartmentalised from other aspects of your life and this is often creates the need to make Loss trade-offs. This type of trade-off tends to create or impose limitations on your initial contextualisation and/or configuration that you need to acknowledge. Gain trade-offs are typically made when you realise that your initial research contextualisation and/or configuration is missing some important quality, characteristic or opportunity that would make it more convincing and, as a consequence, you augment your research in such a way as to facilitate achievement of those new valued outcomes. Gain trade-offs most often lead to a more complex research contextualisation and/or configuration with attendant increases in time, effort and other resources that such increased complexity demands. As a consequence, however, they may also provide ways to overcome or mitigate limitations imposed by your initial research contextualisation and/or configuration. Loss or Gain trade-offs can be made at any point along your research journey (call these critical decision points), where you make changes along the way in order that your goals and valued outcomes can be achieved (even though those goals themselves may be modified as a consequence of what happens during the journey). No matter what type of trade-off you make, what drives your decision is your need to achieve valued outcomes that your current research contextualisation and/or

470

12 How Do I Scope, Shape and Configure My Research Project?

configuration cannot or is likely not to provide. For Loss trade-offs, your major value of interest is feasibility; for Gain trade-offs, your major value of interest is convincingness. It is important to realise that there are several types of Loss and Gain trade-offs you might make. Each type accomplishes a different purpose, produces a different valued outcome and the need for it may arise from different sets of circumstances. It is also important to understand that not all trade-off decisions may yield positive outcomes; some may create ethical dilemmas and other downstream problems for you, which may require you to make further trade-offs—a cascading effect. Since trade-off decisions affect the very nature of the research you are conducting, it is important to record all of your thinking about each trade-off in your research journal, including its implications for research quality criteria and meta-criteria, so that your logic can be reconstructed, if needed, when you write the research story or respond to reviews and criticisms. Each type of trade-off decision is discussed in more detail below. • Substitution: This Loss trade-off involves substituting a research angle, aspect or component that is more likely to work in your research context for another component or approach that is less likely to work or that has, in fact, not worked. For example, this decision would occur when you replace a data gathering strategy that you have found to be unviable (in a trialling phase, for example) with another data gathering strategy that is more viable. It would also be evident if you made a decision to use a less powerful statistical analysis technique that is more appropriate for handling anomalies in your quantitative data. The potential downside of this decision is that the substitution may require you to pull back from aspirations to generalise or infer external causality or may be accompanied by changes in your adopted pattern of guiding assumptions, which must then be accommodated. The substitution trade-off generally shifts the balance in emphasis between internal and external validity (and, possibly, statistical conclusion validity) in research guided by the positivist pattern of guiding assumptions or between authenticity and sufficiency for research guided by an interpretivist/constructivist pattern of guiding assumptions. In terms of the meta-criteria, a substitution trade-off may shift the balance in emphasis and character between internal coherence, analytical integrity and extensional reasoning and may create additional limitations that will need to be acknowledged in your research story. • Sacrifice: This Loss trade-off involves the deletion of a research angle, aspect or component, thus, removing a part of your research because it is no longer viable or its intent has become irrelevant. This decision can involve removal of a research question from your goals. The sacrifice trade-off is a more drastic decision and, as a result, can create real consequences for meeting the expectations of specific research quality criteria and meta-criteria. In many cases, this trade-off is forced on you by changing circumstances in your research context (such as a school pulling the plug on its participation in your research halfway through the data gathering process or a gatekeeper denying access to certain

12.3

Making Trade-Offs

471

desirable data sources). In other cases, you may come to realise that a particular research goal is not achievable within the resources constraints you confront. For example, you may intend to conduct multiple case studies in the context of your PhD candidature but then discover that you have insufficient time to gather and analyse the type and quantity of qualitative data beyond a single case study (thus, a Multiple Case-based configuration is traded away for a Single Casebased configuration, which means sacrificing capacity to draw strong conclusions about transportability of what is learned). In any case, the sacrifice trade-off will create limitations that you will need to address in your research story. Downstream, a sacrifice decision (e.g., losing access to a desired school) may lead to a substitution decision (e.g., selecting a lesser desirable but accessible school) to try to recover from some of the losses incurred by your sacrifice. • Opportunistic redirection: This Gain trade-off is one that can create unanticipated positive outcomes for you, because it is not stimulated by unviability of an angle, aspect or component of your research but by the potential viability of pursuing a new angle. Here, you decide to pursue a new research pathway within the same broad research context, which can involve a partial or a complete rethink of your research configuration (or even further back, a rethink of your research questions), because a new situation has presented itself or new resources have become available. Thus, an unanticipated opportunity in the research context (e.g., a shift in market forces, due to the emergence of new competitors, may amplify the possibilities for exploring strategic decision making in an organisation, when your original intention may have been to study decision making in performance reviews) provides the impetus to shift the research focus. Part of what may feed this decision is your interest and curiosity, stimulated by the new opportunity. However, the downside to this decision might be that the timeline for your research gets extended as you need to plan and reconfigure your research activities to follow the new direction. There may also be some consequences for research quality criteria and meta-criteria (such as the need for re-contextualising the research problem, need to become familiar with new areas of literature, etc.) that you need to address. • Complement: The complement trade-off is a Gain trade-off that involves adding an angle, aspect or component to your research contextualisation and/or configuration whose strengths can at least partially compensate for the weaknesses of another angle, aspect or component. Thus, you extend/augment your research through adding a complementary facet. This, of course, adds to the complexity and resource-intensiveness of your research, but with the potential to yield a more convincing story. In terms of research configuration, this trade-off decision would be signalled in a shift from a simultaneous MU configuration to a sequential MU configuration. The trade-off achieved by the complement decision trades away resource efficiency in exchange for enhanced research effectiveness. This particular trade-off decision actually makes your research stronger and more convincing (against certain research quality criteria and meta-criteria) than its original version (where resource efficiency is valued).

472

12 How Do I Scope, Shape and Configure My Research Project?

• Adaptation on the fly: This Loss trade-off involves a mid-stream correction, redirection or reconfiguration of your research and it reflects flexibility and nimbleness in action. In certain situations, circumstances may emerge during your data gathering process, for example, where you must do something different in order to proceed. For example, you may be using a cause mapping data gathering strategy in an interpretivist investigation, asking participants to produce a visualisation (i.e., a cause map) of their understanding of the causes of particular events they have experienced, but part way through data gathering, you encounter a participant who is not really comfortable (or good at) producing such a visualisation. An adaptation-on-the-fly trade-off might be to ask the participant to think aloud, i.e., verbally explaining their thinking, while they drew the visualisation the best they can (this example was inspired by Sandall’s, 2006, PhD research experience). The implications of this adaptive decision are, for this participant and any others encountered who were in the same boat, their data do not take the same form as the data produced by other participants, meaning that internal coherence and analytical integrity could be affected. Another example would be where you conduct econometric research using a secondary national database of institutional economic, financial and labour statistics, but upon commencing the sampling of the database, you realise there are gaps in the information that is stored, creating a missing data problem. This generates the need to adapt your research approach to either estimate the missing data values (use more complex statistical processes, creating implications for analytical integrity) or to remove any entity from the sample that has missing data and choosing a replacement entity (creating implications for extensional reasoning). The adaptation-on-the-fly trade-off decision is also a hallmark of the evolutionary MU configuration where subsequent MUs are not known or planned for until a critical choice point has been reached in your research journey. The trade-off being made here is giving away certainty about your research configuration and living with the uncertainties of what is to come. Flexibility is maximised, but at the cost of not knowing what downstream research activities will be and whether they will be viable until it comes time to make the choice (i.e., after the first MU has been completed). An example might be where you conduct qualitative semi-structured interviews about remuneration practices with a sample of employees of a large company and what is signaled in these interviews is a need to interview union officials outside the organisation, who had previously argued and won a case against the organisation for paying employees below award wages with the Australian Fair Work Commission, as well as senior managers within the organisation. • Satisfice: The satisfice decision is a Loss trade-off where a research contextualisation or configuration choice is made immediately if it looks like it will work satisfactorily. The term ‘satisfice’ was coined by Nobel laureate Herbert Simon (1957, cited in Anderson et al., 1981, pp. 214–215) to refer to making a decision by adopting the first solution to a problem that a person thinks will work. It is a hallmark of ‘bounded rationality’ (see Anderson et al., 1981, p. 30) associated with human decision making in complex and uncertain circumstances. It means

12.3

Making Trade-Offs

473

you make a decision without giving rational consideration to all possible options; instead you make the first workable, i.e., satisfactory, choice, where satisfactory means the choice will meet your minimum standards. Circumstances for satisficing tend to occur in times of high stress and where insufficient time and resources are available to make a fully rational choice. In terms of a research trade-off, this means deliberately giving away looking for an optimal research angle, aspect or component in favour of the first concrete research angle, aspect or component that you think will get the job done to a minimum set of specifications. This usually means that you do not consider the full range of options for configuring your research in order to choose the best option. An example of the satisfice trade-off in action would be where you adopt the Survey research frame and decide to administer a simple quantitative questionnaire to a convenience sample of undergraduate students because it is a cheap and fast way to get data from a large sample on a particular issue. A satisfice trade-off almost invariability means a reduction in the quality of your research and its capacity to convince, especially with respect to its internal coherence and extensional reasoning. It can sometimes be a hallmark of a novice researcher; it can be highly practical (especially where resources are very tight) yet represents the antithesis of thinking systemically. • Complete restart: This is the ultimate and most drastic Loss trade-off you can make. Quite simply, it means that, in light of emerging circumstances, constraints and obstacles, you completely abandon your current research problem and decide instead to pursue a completely new problem in a new context. The distinction between this trade-off and the opportunistic redirection trade-off is that the latter remains focused on the research context you commenced researching; with the complete restart trade-off decision, even the research context changes. This trade-off decision involves a complete rethink of your research questions and the associated research contextualisations, positionings and configuration. As an example, suppose you conduct a pilot study to fine-tune a quantitative questionnaire on clinical decision making using a small sample of medical practitioners. As a consequence of your pilot testing experience, you learn that medical practitioners were extremely reluctant to be queried about their clinical decision-making practices and outcomes, giving rise to an extremely low response rate of 4%. As a result, you abandon your intention to study clinical decision making and completely restart your research in another area of interest (this example was inspired by Ross’, 1999, PhD research experience). When a complete restart trade-off is made, your entire research process commences again, from scratch, a consequence that has obvious implications for the timeline and probably the resourcing of your research, as well as, at least in the short term, demoralising you! In terms of postgraduate candidature, you would have to decide if there is sufficient time left in your candidature to allow for a complete restart. • Strategic: A strategic trade-off is where you alter or add a research angle, aspect or component in order to create opportunities to achieve longer-term research goals (e.g., enhanced grant opportunity, create good will for future research in a

474

12 How Do I Scope, Shape and Configure My Research Project?

specific context). The strategic trade-off can be either Loss or Gain depending upon circumstances: Loss if your current research contextualisation and/or configuration is modified to achieve the strategic goal; Gain if your research contextualisation and/or configuration is augmented to achieve the strategic goal. This trade-off strategy can create new ethical concerns for you, if your valued goal becomes more personal than professional and/or can introduce biases into decisions you make while navigating the data triangle, if they run contrary to certain stakeholder expectations or sound research practices (in which case, it may backfire, and you may lose everything). For example, suppose you wish to conduct a longitudinal year-long study to implement and evaluate an innovative classroom teaching technology at a particular high school but are denied access by the principal. After some negotiation with the principal, you agree to first conduct a small-scale short-term study within the school and workshop the results with staff. If successful, the principal will agree to grant you permission to conduct your longer term, more involved, study in the school. • Political: A political trade-off is almost always Loss in nature and involves you adapting your planned research by adding a specific research angle, aspect or component in order to meet a specific stakeholder’s (or group of stakeholders) expectations. You might make this trade-off because it will help the stakeholder achieve a valued goal in their own context (such as gaining information that will help them to manage a change process within a community or evaluating a new innovation for use in a particular industry). Generally, you might consider making this trade-off if the stakeholder offered to provide resources or access (if they are also a gatekeeper) in exchange for adapting your research to meet their needs. In making this kind of trade-off, you must be very careful not to ‘sell your soul’ as it can potentially bring certain ethical concerns into the foreground. For example, the CEO of an organisation may be willing to provide you with access to their organisation for data gathering purposes as well as provide resources to support those data gathering activities as long as you are willing to share what you learn from every participant with them. This creates an ethical dilemma for you in the case where you need to promise full anonymity to every participant. The ethical decision would be to withdraw from pursuing your research in that organisation, i.e., to make a sacrifice trade-off instead. The practical decision would be to agree in general to the CEO’s request but negotiate strongly for doing it in such a way as to protect individual participant identities, i.e., a compromise political trade-off. • Comfort Zone: A comfort zone trade-off is often Loss in nature as well and almost always results in less-than-optimal research. As the name suggests, this trade-off seeks to keep you safely operating within your comfort zone, with respect to your research preferences, skills and abilities, and prior experiences. In short, it preserves your status quo as a researcher. It involves (a) incorporating a specific angle, aspect or component into the research plan that you prefer, think is most expedient or feel most comfortable/competent in carrying out or (b) ruling out an approach that you feel is least preferable, expedient or you feel least comfortable/competent in carrying out. The outcome associated with

12.3

Making Trade-Offs

475

(a) may reflect a philosophical bias such as adopting only the positivist pattern of guiding assumptions in your research (which itself may reflect disciplinary biases or selectiveness in prior research training). The outcome associated with (b) may reflect a skills deficit or a general bias against quantitative (e.g., you hate or can’t work with numbers/statistics) or qualitative (e.g., you think qualitative research is not credible) research approaches. This kind of trade-off is thus made to meet personal rather than professional needs, incurring sacrifices in research quality and convincingness, and has the effect of stifling thinking openly and systemically about your research. It substitutes expedience for quality and suitability. The way to negate the pressure to consider this trade-off is to take that pressure as a signal that you need to develop new skills and new thinking patterns before pursuing the research project. • Professional: The professional trade-off may be either Loss or Gain, depending upon your circumstances and intentions. A Loss professional trade-off is strictly utilitarian in nature and involves adding or modifying an angle, aspect or component of your research in order to enhance your chances for achieving a valued career-oriented goal, such as your postgraduate degree, a promotion, a publication, tenure, working with a valued colleague or mentor, more consulting work, more professional prestige, opening up a new job opportunity. The Loss version of the professional trade-off will generally tend to detract from convincingness as its tighter focus on achieving a professional gain will inhibit systems thinking about what is best for your research project itself. An example of a Loss professional trade-off would be where you aspire to get your research published in a well-respected marketing journal but learn that chances of acceptance are enhanced if structural equation modelling is used as the data analytic theory-testing approach. You may not have initially planned to use this procedure for your analysis but decide to use the approach in hopes of gaining an edge for the publication, even though your data are of insufficient quality to warrant the use of the procedure. The Gain version of the professional trade-off has a learning application focus and involves you adding or modifying an angle, aspect or component of your research so that you can open up an opportunity to create new knowledge or acquire/apply new skills (such as tapping into new and unexpected data sources, using a new data gathering or data analytic technique, enhancing the extensional reach of your research), in ways not envisaged in your initial research plan. This version of the trade-off thus seeks to add value and convincingness to your research. An example of the Gain version of this trade-off would be where you learn about a new analytical procedure and see value for enhancing analytical integrity as part of convincingness by using that new procedure in your current study, when you had planned to use other approaches initially. • Personal: The personal trade-off is not necessarily Loss or Gain in nature. Rather, it is an adaptive response to your circumstances and intentions. This trade-off explicitly connects with your larger life space and may involve adding or modifying an angle, aspect or component of your research in order to accommodate family and lifestyle needs (e.g., minimise time away from family,

476

12 How Do I Scope, Shape and Configure My Research Project?

minimising travel, avoiding conflicting work/life commitments) or to accommodate limitations imposed by physical impairments such as blindness, deafness or mobility restrictions. While the personal trade-off may create difficulties in terms of the convincingness of your research that need to be dealt with, its overall priority is to improve the degree to which your research project can fit within your larger life and circumstances—the epitome of trying to achieve a realistic and feasible project. For example, if you have a very young family and a working spouse, you may not wish to conduct your research away from your home town. This might lead you to make a personal trade-off to conduct your research using a local convenience sample of participants, rather than a geographically-dispersed (but more representative) sample, accepting the limitations on generalisation that this choice would impose. As another example, suppose you had a severe vision impairment such that you are legally blind. Such personal circumstances might lead you to adopt an interpretivist/ constructivist pattern of guiding assumptions and a semi-structured qualitative interview data gathering strategy rather than adopting the positivist pattern of guiding assumptions and using a more broad-reaching quantitative internet questionnaire (which a vision impaired person would find very challenging to design and construct). In the end, this could actually enhance the convincingness of your research because of your heightened listening skills, but would mean a shift from a breadth to a depth research scoping and shaping focus (see Smith-Ruig & Sheridan, 2012, for a more detailed case study exploring the research adaptation and accommodation experiences of a vision-impaired researcher, including an emergent tendency to favour electronic literature over paper-based printed literature because of how adaptive technologies for the blind functioned).

12.4

Harnessing Synergies

One goal of thinking more openly and systemically about social and behavioural research is that you can begin to see how synergies between different approaches might be harnessed and leveraged to yield more powerful and convincing research outcomes. This involves critically assessing the strengths and weaknesses of each aspect of research being considered. This thinking extends from contextualisation of your research, including settling upon one or more patterns of guiding assumptions, through to configuring your research and navigating the data triangle and should be recorded, as it unfolds, in your research journal. When looking for possible synergies, you are looking for extensions, combinations, adaptations and/ or modifications to your research contextualisations, positionings and configuration that will add value to what you are doing, help you to overcome constraints and/or take advantage of opportunities. Thus, harnessing synergies is all about enhancing the convincingness of your research, helping you to:

12.4

Harnessing Synergies

477

• overcome limits to your thinking about the research and what you intend to achieve by deliberately engaging systems thinking, asking “what else could I do to add value to my research processes and outcomes?”; • go beyond constraints imposed by specific patterns of guiding assumptions (overcoming paradigm blindness), move beyond traditional expectations associated with research in specific disciplines (transcending discipline silos) and/or remove boundaries and barriers that have typically been imposed on the type of research you are undertaking (breaking down conventional boundaries); • augment your research with an angle, facet or component whose strengths will help to compensate for the weaknesses of the angles, facets or components you have already settled upon; • gather data that will help you to convey a more well-rounded convincing research story and/or reach a wider research audience; • better meet research user/stakeholder expectations through research augmentations that more closely target their needs; • circumvent, negate or mitigate the impact of certain constraints on, or emergent obstacles to, your research through making appropriate modifications/ adaptations to your research process; and/or • take advantage of an opportunity that emerges during the course of your research journey through the extension of your research contextualisation or configuration. When you harness synergies, you are deliberately anticipating the weaker aspects of your research and their potential impact on your research story, then specifically targeting those aspects by augmenting your contextualisation and configuration strategies. For example, if you are aware that key stakeholders are expecting clear evidence of causality when conducting a program evaluation, this may lead you to augment your research approach to incorporate a longitudinal dimension so that changes over time, attributable to the program, can be explicitly assessed using both quantitative questionnaire data and qualitative interview data, e.g., changing from a Multiple Simultaneous configuration to a Longitudinal Intervention Time—Aligned configuration. As another example, suppose you intend to administer a quantitative questionnaire (measuring attitudes toward and preferences for council services) to a sample of community members (in the context of a Survey research frame), but are aware that including only pre-specified quantitative measures and rating scales on the instrument can turn many participants off (which could reduce response rate for the questionnaire) because their own point of view has not been sought. As a consequence, you incorporate some open-ended qualitative questions in your questionnaire in order gain insights into those participant perspectives or points of view. This then augments the research configuration from a Single MU configuration to a Hierarchical—Embedded configuration. As a third example, consider the situation where you are conducting research in a discipline area where the predominant guiding assumptions in the literature have been positivist and the predominant data gathering strategy has been experimental or quasi-experimental in nature. However, you want to go beyond this traditional way

478

12 How Do I Scope, Shape and Configure My Research Project?

of doing business by adding a component to your research which adopts an interpretivist/constructivist pattern of guiding assumptions. This could lead you to augment your research to incorporate a field-based qualitative interview component (which would extend your research context) as a front-end component of your research configuration, conducted prior to and with the intention of providing learning to inform the design and conduct of a formal laboratory experiment. This modification would move your research from a Single MU configuration to an Exploratory Sequential configuration and could help to enhance the realism of your laboratory experiment for participants, thereby enhancing your capacity to generalise beyond the boundaries of your laboratory.

12.5

Building a Conceptual/Theoretical Framework—Do I Need One?

One question that postgraduate students often ask, during their scoping, shaping and configuring activities is, ‘do I need a conceptual or theoretical framework for my thesis/dissertation/portfolio?’ Some supervisors will invariably say, ‘yes, you do’. Others will say that the need for a conceptual or theoretical framework depends upon the pattern of guiding assumptions that informs your research and, we would add, the research frame you have adopted. A conceptual or theoretical framework, in the traditional sense, is most useful and meaningful in positivist research having a quantitative emphasis and can lead you toward a specific MU configuration. You can see this in the language frequently used to describe components of such frameworks: constructs, variables of various types, relationships, causes and effects. However, more and more, conceptual frameworks, especially, can be seen to add value to research conducted under interpretivist/constructivist or other patterns of guiding assumptions. Irrespective of the direction from which you approach conceptual or theoretical frameworks, they can serve an important core function—to establish boundaries around your research problem while specifying those key points of focus, concepts and relationships relevant to that problem which are of interest to you. In short, a conceptual or theoretical framework may help you to both shape and scope your research as well as perhaps influencing your choice of MU configuration. You don’t always have to have one in your thesis, dissertation or portfolio, and for some kinds of research (invoking certain patterns of guiding assumptions), you should definitely not have one, at least early on. However, if it makes sense for you to build a conceptual or theoretical framework for your research, we would recommend that you do so, simply because of its shaping and scoping advantages. Many researchers use conceptual framework and theoretical framework as interchangeable terms. However, there is a subtle difference that can be important, depending upon your pattern of guiding assumptions. Conceptual and theoretical frameworks differ in terms of their intended precision and closeness to the actual data to be, or already, gathered. Conceptual frameworks are typically less precise and less closely tied to data, displaying more general relationships. Theoretical

12.5

Building a Conceptual/Theoretical Framework—Do I Need One?

479

frameworks are more precise and more closely tied to past research and to future data gathering, to the point where specific relationships are being depicted (e.g., externalised cause-effect relationships under the positivist pattern of guiding assumptions or processes, relationships and connections between concepts under an interpretivist/constructivist pattern of guiding assumptions).

12.5.1 Conceptual Frameworks A conceptual framework is basically an organising schema that displays how you understand various concepts or ideas to be linked together. In a sense, a conceptual framework is a mental map of your research landscape. Maxwell (2005) described four important sources you can draw upon for components of a conceptual framework: • experiential knowledge—where you draw upon your own experiences to help shape the framework; • existing theory and research—where you draw upon the existing literature, in various ways, to provide the focus for your framework; • your own pilot and exploratory research—where you employ a preliminary research investigation or pilot/trial study to identify concepts to incorporate into your framework (i.e., implementing a sequential MU configuration); and • thought experiments—where you run mental simulations of your planned research in order to anticipate how things might pan out and what concepts might be relevant to include. Retrospective conceptual framework A conceptual framework may be retrospective in focus if you employ it as a way of showing how research concepts link together or relate to each other in the literature you have reviewed, i.e., a mental map of the past research landscape. The retrospective conceptual framework is built up before you commence data gathering. It can be useful, irrespective of your intended guiding assumptions, because you are only using it as a device for organising and displaying what you have understood from your literature review as it helps you to scope and shape your own research. Figure 12.24 shows a retrospective conceptual framework from a PhD that explored the literature on organic food purchasing and organic/conventional food switching behaviour in consumers (Henryks, 2009). Prospective conceptual framework A conceptual framework may be prospective in focus if you employ it as a mapping of the concepts and linkages you intend to explore in your research (Fisher, 2007). A prospective conceptual framework is also built up before you commence data gathering and may represent a creative display where the “researcher’s interaction with the literature and the context of the research can produce a new way of linking

480

12 How Do I Scope, Shape and Configure My Research Project?

Fig. 12.24 Example of a retrospective conceptual framework (Henryks, 2009, p. 52)

concepts and creating new relationships” (Smyth & Maxwell, 2008, p. 26). It is more useful if you intend to adopt the positivist pattern of guiding assumptions, but not to the extent of making strong theoretical anticipations. For example, if you are conducting research within a Survey research frame, a conceptual framework could be the first step you would undertake in planning the content of your questionnaire. The framework would signal the construct domains for which questionnaire items must be written. A prospective conceptual framework can also serve as the springboard for detailing your research questions and/or hypotheses. Here, components of the framework are usually developed throughout the literature review and the actual framework you want to evaluate is set out late in the literature review chapter, followed by your research hypotheses. Ross (1999) presented a prospective conceptual framework (see Fig. 12.25) in his positivist PhD investigation of organisational commitment in academia where commitment was proposed to be predicted by a range of university and individual contextual factors (the constructs of interest). His framework was also used to identify domains (italicised) for developing questions for a broad-ranging questionnaire and gave rise to 10 specific hypotheses to be tested in his thesis. This conceptual framework was presented by Eugene toward the end of his literature review, just before he set out his specific hypotheses.

12.5

Building a Conceptual/Theoretical Framework—Do I Need One?

481

ACADEMICS’ COMMITMENT TO THE UNIVERSITY Organisational Commitment Loyalty commitment Attachment commitment No-choice commitment

CONTEXTUAL FACTORS WITHIN THE UNIVERSITY

CONTEXTUAL FACTORS WITHIN THE INDIVIDUAL ACADEMIC

Organisational culture factors Productivity focus Quality focus Creativity focus Cooperation focus

Gender

Work environment factors Workplace commitment Peer cohesion Supervisory support Autonomy Task orientation Work pressure Clarity of instructions Control by management Innovation orientation Physical comfort of work environment

Anxiety factors Trait anxiety State anxiety Self-esteem Role attitude factors Role conflict Role ambiguity

Fig. 12.25 A prospective conceptual framework showing hypothesised relationships between different categories of constructs and sub-constructs. Adapted from Ross (1999, p. 86)

Certain types of interpretivist/constructivist research may also usefully employ a prospective conceptual framework to show how you think certain concepts might be interrelated (see the discussion in Miles, Huberman, & Saldana, 1994, pp. 20– 25). However, you must be careful here. A more stringent set of interpretivist/ constructivist guiding assumptions, such as those required in a strict implementation of certain forms of grounded theory (what specialists term ‘Glaserian grounded theory’, see Charmaz, 2014; Glaser, 1992; Morse et al., 2009; Strauss & Corbin, 1990), would demand that apriori anticipation of the concepts you plan to look for or examine in your research should be avoided. This is because spelling out a conceptual framework before you collect the data risks violating a primary

482

12 How Do I Scope, Shape and Configure My Research Project?

assumption that you, as the researcher, should ‘bracket’ your preconceptions so that the participants’ views or views from other data sources such as documents can drive the story that emerges—the voices of the participants thus become privileged. So, presenting a prospective conceptual framework up front, in anything but a very loose form leading to more generally stated research questions, would not be consistent with the tenets of a strict view of the interpretivist/constructivist paradigm and would, therefore, not be expected. In this case, your conceptual framework should not emerge until after you have gathered and interpreted your data—a function more fully explored below. Interpretive conceptual framework A conceptual framework can serve as a mechanism for organising and displaying the interpretations of your data when drawing conclusions and implications, i.e., an interpretive focus (Fisher 2007). This may be an especially useful function in interpretivist/constructivist research where the framework presents, in an integrated pictorial form, the stories and meanings that have emerged from your data. This is most clearly seen in grounded theory research where you present a conceptual or theoretical framework you have developed after several sample-gather-analyse iterations around the ‘Data Triangle’ at the end of your thesis or dissertation (this can only occur if your theory is grounded in your data rather than driving the structure of the data as it would in a positivist investigation). In positivist research, an interpretive conceptual framework can serve as a way of displaying the quantitative relationships you have empirically established between specific variables (using correlations or statistical inference tests). Figure 12.26 shows a rich conceptual theoretical framework depicted by Keith Wolodko (2017) in his PhD where he carried out a grounded theory investigation of group dynamics through time. His theoretical overview emerged from 12 months of

Fig. 12.26 Example of an interpretive conceptual framework emerging from a grounded theory investigation (Wolodko, 2017, p. 246)

12.5

Building a Conceptual/Theoretical Framework—Do I Need One?

483

participant observations and incorporates a time dimension, flowing from left to right allowing the reader to track the trajectories of different leadership dynamics as they impacted on the group’s functioning over time and in response to specific events. How this relates to scoping, shaping and configuring is that Keith knew, going into his research, that he would need to end up producing some kind of theoretical picture of what happened in the group he observed, so he had to plan to gather sufficient contextualised data in order to develop and refine that picture. To that end, he treated his group as an embedded case study and employed a Longitudinal—Case-based MU configuration. Methodological conceptual framework There is another rather creative purpose for a conceptual framework that is especially useful for a pluralist study, namely a methodological conceptual framework. This type of framework can be useful in conveying the logic and flow of a complex research process. For example, Braund’s (2001) PhD investigated public perceptions of the police in Canberra using a pluralist approach. Michael’s study combined participant observation of police-public interactions in different locations with questionnaires administered to the police, members of the public as well as students. His study was intended to generate both qualitative (in terms of participant observation field notes) and quantitative data (from questionnaire rating scales), but as his methodology was complex, he chose to display the overall conceptual organisation of his methodology as a colour-coded process map that linked types of data sources and data gathering strategies (numbered circles) to his research questions and what he wanted to learn from his analyses. Figure 12.27 shows the DATA SOURCES

PUBLIC INSTRUMENTS

2

CARTOONS ACTIVITIES CIRCULAR GRAPH PROFILE DISCOMFORT WORD ASSOCIATION OCCUPATIONS’ FEELINGS

ANALYSIS

1 PUBLIC QUESTIONNAIRES

OBSERVER AS PARTICIPANT ALL SHIFTS ALL SEASONS FRONT DESK CELLS

POLICE

COMMUNITY

IMAGE

Research goal 2 Research questions 2.1

STUDENT INSTRUMENTS

3

OCCUPATIONS ACTIVITIES CIRCULAR GRAPH PROFILE

ANALYSIS

Public perceptions of police Police behaviour Support for police Police characteristics Police image

ANALYSIS

Police role Police image Support for police

STUDENT QUESTIONNAIRES OPINION OF POLICE WELFARE SERVICE

Research goal 2 Research questions 2.1

POLICE SERVICE RESPECT

Police perceptions of public perceptions of police Public perceptions of police Details of interactions, and variables for building up and testing a Turf Interaction model

POLICE INSTRUMENTS

4

CARTOONS ACTIVITIES CIRCULAR GRAPH PROFILE DISCOMFORT WORD ASSOCIATION

Research goal 1 Research questions 1.1 Research question 1.2

Police perceptions of public perceptions of police

ANALYSIS POLICE QUESTIONNAIRES

Research goal 2 Research questions 2.1

GENERAL QUESTIONS ABOUT POLICE

Used in the building up of a Turf Interaction model.

Fig. 12.27 Methodological conceptual framework with four types of data source (including himself as participant-observer) and several data gathering strategies. Adapted from Braund (2001, p. 124)

484

12 How Do I Scope, Shape and Configure My Research Project?

map he included in his Methodology chapter; you can see how it visually portrays the scope of his research.

12.5.2 Theoretical Framework Under the positivist pattern of guiding assumptions, a theoretical framework is basically a diagrammatic representation of your logically argued anticipation of how various constructs are linked together (Cavana, Delahaye, & Sekaran 2001). In a sense, a theoretical framework is a mental map of your hypothesised research predictions; one that purports to specify cause and effect links between the various constructs. Harrison (2003, p. 73) presented a relatively simple theoretical framework (see Fig. 12.28) for her research investigation into the relationships between organisation strategy and effectiveness. In this framework, four distinct constructs (Futurity, Product-Market Development, Market Scope and PEU (Perceived Environmental Uncertainty—at the tail of each arrow)) are hypothesised to be causally related to a fifth construct, Information Scope (at the head of each arrow). In her thesis, Jenny spelled out each pathway hypothesis. For example, in text form, the hypothesis labelled “H2c (+)” read as “H2c: Futurity positively affects information scope” (Harrison 2003, p. 72). The arcs with double-headed arrows show where Jenny hypothesised pairs of constructs to be correlated but not causally related. The theoretical framework shown in Fig. 12.28 was part of a larger framework that Jenny built up through reading the literature as well as through logical argument. Figure 12.29 (Muchiri, 2006, p. 48) shows a much more complex theoretical model. Michael’s framework was built up through an integration of constructs from three different leadership theories, coupled with theorised connections to a number of other organisational behaviour constructs which, he hypothesised, would be causally linked in specific ways. Each construct was measured by at least three specific indicator variables (in this case, items on a questionnaire). The structure of his framework was informed by reading the literature coupled with logical argument where gaps occurred. A structural model is a more formal and stylised representation of a theoretical framework, used only in research guided by the positivist pattern of assumptions. Generally speaking, a structural model is required as the first step to implementing a statistical structural equation model approach to theory testing and data analysis. Figure 12.30 displays a simplified hypothesised structural model for the theoretical framework shown in Fig. 12.28 (Harrison, 2003). In her structural model, she included specific objects that related directly to her research questions: exogenous variables (whose causes lie outside the boundaries of her research investigation, remaining unknown; i.e., Futurity, Product-Market Development, Market Scope and PEU = ovals); endogenous variables (whose causes are other exogenous variables in the model that she explicitly measured or other endogenous variables; i.e., Information Scope = rectangle); causal paths (symbolised by directional arrows

12.5

Building a Conceptual/Theoretical Framework—Do I Need One?

485

Futurity

H2c (+) Product Market Development

H2b (+) Information Scope H2d (+)

Market Scope H2a (+)

PEU

Controlling for Demographic Variables

Fig. 12.28 A simple theoretical framework showing five hypothesised constructs and how they are causally interrelated (each hypothesised cause points to its hypothesised effect). Adapted from Harrison (2003, p. 73)

Transformational -Transactional Leadership

Organisational Commitment

Substitutes for Leadership

Collective Efficacy/ Expectations

Social Processes of Leadership

Organisational Efficacy

Organisational Citizenship Behaviours

Performance Outcomes

Indicator Variables for Constructs

Fig. 12.29 A complex theoretical framework showing a number of hypothesised constructs all causally connected to each other Adapted from Muchiri (2006, p. 48)

that point from variables that Jenny theorised to be putative causes to those variables that Jenny theorised to be their effects; there were four hypothesised causal paths between constructs in the model); covariation paths (symbolised by

486

12 How Do I Scope, Shape and Configure My Research Project?

Futurity

Product Market Development

e1

Information Scope

Market Scope

PEU

Fig. 12.30 A structural model representation for Jenny Harrison’s theoretical framework from Fig. 12.28

double-headed arrows that connect two variables in a non-causal correlational sense; thus, Futurity and Product-Market Development were hypothesised to be correlated); and an error path (a path that reflects the causal influence of prediction (i.e., random) errors (=circle) on a specific endogenous variable; i.e., e1).

12.6

Trialling Your MU Research Activities for Navigating the ‘Data Triangle’

Irrespective of the particular MU configuration implemented in your research project, a key component of your research plan should involve trialling the research activities associated with each MU in your configuration. Trialling is essential because it provides you with early feedback about how well various research activities associated with navigating the ‘Data Triangle’ will work, where the pitfalls are likely to be, where important modifications and improvements might be made before you commence primary data gathering and, at a more holistic level, whether your planned MU configuration and associated data-related activities will even be viable. Where appropriate, trialling also gives you a chance to develop and hone your own skills needed in certain types of data gathering, such as interviewing and observing, in conjunction with the use of any supporting technologies (such as laptop computers, tablet computers or smart phones (perhaps using communication software such as Skype or FaceTime), digital recorders, video cameras or handheld

12.6

Trialling Your MU Research Activities for Navigating the ‘Data Triangle’

487

recording devices) and/or the skills of others who may be involved with data gathering. For research guided by the positivist, or possibly the critical realist, pattern of guiding assumptions, intending to gather quantitative data, trialling can focus on the following research activities. • Pilot testing of measurement systems. An essential aspect of any positivist research investigation gathering quantitative data involves pilot testing and validating the quantitative measures, instruments, structured interview protocols and/or observation rating scales used to gather those data, either by researcher-administered or self-administered means (see further discussions in Chap. 18). The goal here is to assess whether or not the measurement systems (either designed by you or by other researchers) to be used in the MU(s) will work (from the point of view of participants) in terms of the clarity of the expressed intent of the measurement system, clarity of the instructions for giving responses to questions, measurement system layout and formatting, clarity of the language used to write items, the time it takes to complete the measurement task, how difficult the measurement system is to use, the cultural relevance/ meaningfulness of the measurement system and the concepts it embodies and the face validity of the measurement system in terms of the constructs it purports to measure. You can also gather feedback on any ambiguities or emotional reactions to specific questions asked or items for rating as well as pre-test any contextual manipulations and controls if a Manipulative experience-focused data gathering strategy (see Chap. 14) is being employed. For observational research, you can pilot test your recording form for making structured observations and estimate the inter-rater reliability of the observational coding system. • Evaluating technological support systems. Nowadays, a good deal of positivist quantitative research depends upon the use of some form of technology to support your data gathering activities. Computers in various forms (e.g., desktops, laptops, tablets, smart phones), in particular, have become an important avenue for data gathering, but this, in turn, creates additional challenges for you. In order for a computer to be used effectively as a data gathering device (whether as a stand-alone device or connected to the internet), the interface between the human participant and the measurement system for gathering data (e.g., online questionnaires, web-based experiments (e.g., choice, marketing, decision making, perception or cognition experiments; see, for example, http://www. mouselabweb.org/; http://www.psytoolkit.org/; http://www.sawtoothsoftware. com/products/conjoint-choice-analysis/cbc)) must be designed and programmed. Once you have completed the initial programming, you should then evaluate the interface for functionality, errors, clarity and general ease of use from the point of view of the participant. This means trialling the interface with a sample of participants in ways that will allow you to assess the screen layout, use of colour, speed, readability, ease of use, including how participants can change their responses without losing all previously collected data, compatibility with various computing platforms, operating systems and web browsers, error

488

12 How Do I Scope, Shape and Configure My Research Project?

trapping to prevent undesirable responses, data recording accuracy and stability. You should also check the functionality of any audio and/or video recording devices in context (such as in an interview room); test out Skype or Facetime platforms to support the conduct of structured interviews; calibrate and test physiological and neurological measurement technology such as eye tracking devices for tracing eye movements while performing a task (e.g., GazePoint—see http://www.gazept.com/; iMotions—see https://imotions.com/), EEG devices for measuring and recording brain waves (see https://imotions.com/; also see EMOTIVE Brainware—https://www.emotiv.com/), Galvanic Skin Response (GSR) devices to measure electrical conductivity of the skin as an index of emotional arousal and stress (see https://imotions.com/). • Practicing data gathering procedures. It makes sense for you to devote time and effort to practicing the data gathering procedures you will be implementing. This would include ensuring clarity in the process for obtaining informed consent to participate; ensuring that any instructions that are provided to participants are clear and unambiguous; practicing the conduct of structured interviews (and monitoring associated interviewer behaviours to ensure consistency); trialling processes for content coding of documents (e.g., as might be done for a meta-analysis); checking that the training of interviewers has achieved a consistent standard; practicing/simulating the handling of difficult problems that may arise with interviewees; practicing making and recording observations in context (whether obtrusive or unobtrusive in nature) and ensuring that observations are consistent if multiple observers are used (i.e., checking inter-rater reliability). • Checking the viability of your sampling plan. It also makes sense for you to double-check your access to a desired population list (however defined or delimited) for your research where random or systematic sampling processes will be used. You should also test any algorithms for obtaining the sample by sampling participants for the trialling phase as well as evaluate your intended approaches for obtaining informed consent to participate in the research to ensure that participation rates will not be adversely affected by flaws in the informed consent process (especially important in the case of research conducted within the Cross-Cultural or Indigenous research frames). If you use a secondary database as a data source for your research, you should trial sampling from that database and check the accuracy and completeness of the information accessed (this could reveal if there is likely to be a missing data problem, for example). • Practicing data preparation and analytical procedures. It is important for you to try out your intended analytical procedures or a statistical software support package on a small scale in order to ensure that you understand all the required steps. Such trialling may constitute professional development for you in cases where you plan to use unfamiliar approaches or software. This aspect of trialling should also encompass learning the requirements of the software support system (s) to be used (e.g., SPSS, NCSS, SYSTAT, SAS, Stata, eViews, Statistica, R; see the discussion in Cooksey, 2014, Chap. 3) and acquiring facility in their use and

12.6

Trialling Your MU Research Activities for Navigating the ‘Data Triangle’

489

interpretation, including data formatting, input and storage. This aspect of trialling provides the most appropriate opportunity to devise and test out systems for quantitatively coding trialling sample responses to categorical questions (e.g., demographic questions on a questionnaire or in a structured interview). Where open-ended questions are used in a data gathering process, you should practice content coding qualitative responses from the trialling sample into categories suitable for quantitative analysis. For research guided by an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, intending to gather qualitative data, trialling can focus on the following research activities. • Practicing data gathering procedures. It is important to carry out dry runs of your intended data gathering strategies using a trialling sample of participants. Think of this process as a simulation of your actual data gathering activities, undertaken to ensure that you can manage the process and that you can anticipate, prepare for and effectively manage possible adverse events, such as a digital recorder failing or an interview going off the rails because of a lack of rapport between the interviewer and interviewee. This is especially important where semi-structured, focused or unstructured interviews are to be conducted. It takes time and effort for you to learn how to effectively conduct such interviews, especially if you have not done one before. Things to attend to would include learning not to dominate the interview, avoiding asking leading questions, asking sensitive questions sensitively, listening actively and managing the consistency of verbal and non-verbal aspects of your behaviour to avoid inadvertently sending the wrong signals to an interviewee (such as appearing judgmental, bored or disinterested). If you plan to employ participant observation as a data gathering strategy, it is important to practice participant observation techniques and processes, even down to the nitty gritty of where to place digital recorders, where to sit and how to behave if observing a group meeting and how to manage informal conversations with others in the research context. If data gathering is to involve participants keeping their own activity or reflective diary or drawing their own concept maps, these processes need to be trialled for viability and consistency as well. If these activities depend upon the use of technology to carry out, then training of participants in the use of the technology also needs to be trialled. • Practice connecting with participants. A very human aspect to any interaction-based data gathering strategy is learning how to connect with participants so that they feel comfortable allowing you access to their views and perspectives, i.e., the building of rapport. Participants must feel safe and protected in any type of qualitative interview before they will reveal their authentic perspectives instead of managed perspectives. How easy or difficult this connection is for you to make depends a good deal upon your own personality and social skills and preferences. It takes practice to become more natural in building rapport with participants, especially for more introverted researchers. Thus, conducting trial runs of qualitative interviews can provide a useful training program for you

490

12 How Do I Scope, Shape and Configure My Research Project?

as interviewer. Part of this process is learning what to do if rapport is threatened (and learning to be sensitive to potential sources of such threats that could arise during an interview) or never established, how to salvage a failing interview, how to make the interview flow as much like a natural conversation as possible and how to sensitively end an interview that is going nowhere. It is also important to realise that the approach you take for obtaining informed consent to participate in the research establishes the initial foundation upon which rapport is built (or fails to be built) and this process should be trialled as well. • Practice recording field notes. You should practice recording field notes and observational notes as well as self-reflective observations in your research journal, under as close to the same situational conditions as you will experience when primary data gathering commences. This means gaining facility with using a split-page recording system, either manually or using a computer support system (as illustrated in Chap. 3, Fig. 3.1a–e) in order to transparently segregate the recording of what you have heard, seen and experienced in a research context from the recording and management of your preconceptions, impressions, initial analytical ideas and self-reflections on the data gathering, analysis and interpretation processes. • Working with technological support systems. As part of the qualitative data gathering process, you may employ technological support such as digital recorders, cameras, video cameras, laptops and tablet computers or smart phones (perhaps using communication software such as Skype or FaceTime) to facilitate data gathering, the internet for locating and downloading documents, images, video files and social media content and LiveScribe, EverNote or other note-taking and memoing technology for recording field notes and observations. Such technology needs to be seamlessly integrated with the data gathering process it supports without its use being intrusive or distracting and potentially influencing the nature of the data gathered. Trialling can provide practice experiences to help ensure that this smooth integration and can give you the opportunity to troubleshoot any problems that arise before primary data gathering commences. • Practicing data preparation and analytical procedures. This aspect of trialling is often closely linked to one or more of the technology support systems implicated above and focuses on trying out analytical procedures/approaches or a software package on a small scale. Nowadays, there are many Computer-Assisted Qualitative Data AnalysiS (CAQDAS) systems available (e.g., MAXQDA, NVivo, dedoose, Atlas.ti, HyperResearch, The Ethnograph) and each has their own rules and expectations for data/document formats, operating, coding, making notations, generating analyses and displaying what has been learned. You need to learn the ins and outs, advantages and limitations of any system you intend to use, including how to prepare data for use within the system. A trial run is a good and safe place for such learning to occur. Often, qualitative data (e.g., recorded interviews, field notes) need to be prepared for analysis in the form of transcripts (either transcribed by hand or electronically/automatically using a technological support system such as Dragon Naturally-Speaking or QuickVoice Pro).

12.6

Trialling Your MU Research Activities for Navigating the ‘Data Triangle’

491

Preparing transcripts requires forethought as to what content you wish to include in the transcript and how to represent certain types of content (such as emotional reactions, speaking emphasis, stutters, interruptions and pauses, speaker identity, unclear words or phrases, distracting noises) in the transcript (see Chap. 20). A trial run of the research is a good way to fully prepare and evaluate the workability and adequacy of such transcription rules. If you are doing document analysis, you can do a trial run of downloading, preparing and commencing the analysis of the content or conversations in the documents. You do not need large samples to conduct trial runs of your research. This is because your goal is typically not to fully analyse the resulting data but to learn how well your intended data sampling, gathering and analysis processes will work. In short, trialling provides practice in navigating the ‘Data Triangle’ before your actual journey commences. However, you should try to include a representative group of participants in the trial run, similar to the types of participants that will be sampled for the actual study, and to conduct the trial under conditions as similar to those in the actual research context as possible. As part of the trialling process and irrespective of the assumptions adopted to guide the primary data gathering activities, you should also carry out a follow-up semi-structured interview with each trial participant. The purpose of this interview would be to gather feedback from the participant on their perceptions of all aspects of the data gathering processes and their participation in them; in short, discussing their participation experience with them. To maximise the learning value of such feedback so that appropriate modifications and fine-tuning of those processes can be accomplished, you should look for feedback that touches on the cognitive (e.g., were questionnaire items easy, hard or confusing?; if a computer interface was used, was it easy to use, comfortable to read and interact with and easy to fix mistakes within?; were instructions clear or ambiguous?; were things asked about that the participant shouldn’t have been expected to know), emotional (e.g., were some questions that were asked insensitive, offensive or culturally biased?; did the participant feel threatened by their participation or by what might be done with the information they provide?; did the participant feel like pulling the plug on their participation at any time?; did the use of technology create problems for them?) and practical (e.g., was the time demand for participation acceptable or excessive?; was the setting in which participation occurred comfortable and relatively stress-free?) aspects of their participation. It is also important that each participant be asked what they would change about the data gathering processes to make it a better experience. You should incorporate a trialling phase into your project management plan (recall Chap. 5) as the front-end component for any MU configuration. This, of course, means that the gathering of trial data must be factored into your ethics approval application as well. You can trial essential parts or components of MUs (such as measurement instruments or experimental manipulations, guided by positivist assumptions) or conduct a more holistic dry run of a complete MU, like a small-scale rehearsal (useful in research guided by an interpretivist/constructivist pattern of assumptions). The learning value of trialling research activities is

492

12 How Do I Scope, Shape and Configure My Research Project?

incalculable and can save you from disastrous mistakes. For example, it is far better to learn that some questionnaire items or interview questions are uncomfortable or offensive to participants during trial run whereupon revisions can be made, than to discover downstream that certain items or questions were indeed offensive and, as a consequence, contributed to a large participation withdrawal rate or created patterns of missing data. This would waste your resources through loss of valuable data, detract from convincingness through holes created in the evidence and damage good will with the participants themselves. You may find it useful to conduct your trial run as part of the process for preparing your research proposal so that what you learn can be built into the proposal process as refinements already in place. By conducting a sound trial run that provides good learning feedback, you can discover and deal with some of the unexpected obstacles that can emerge during your research journey, especially those obstacles that would interfere with smooth navigation of the data triangle and effective implementation of the MU configuration.

12.7

How Can You Reflect Research Scoping, Shaping and Configuring in Your Writing?

The best way to see how scoping, shaping and configuring might be reflected in writing up a postgraduate research project is by examining some concrete examples. Every thesis, dissertation or portfolio will be different and how the scoping, shaping and configuring stories or relevant aspects of them are told will depend upon how your project has evolved and the points you want to make about the pathways of that evolution. The more transparent your stories are in your thesis, dissertation or portfolio, the easier the reader will find it to follow your research planning logic; your choices of where in the document to convey such stories is a strategic decision. • In many cases, the Methodology (or equivalent) chapter is the place for your scoping, shaping and configuring stories (more will be said about this in Chap. 22). Here you can retrace the pathway of choices leading to your final research approach, including your choices of research frame, guiding assumptions; methodologies; sampling frames; ethical procedures; procedures for passing gatekeepers; desired analytical strategies and so on. • However, if you don’t want to make scoping, shaping and configuring stories a central feature of your Methodology (or equivalent) chapter, another choice that can work is to place relevant stories in your Introductory chapter. Here you foreshadow your scoping, shaping and configuring stories in a way that sketches what you thought about as you planned your research. This leaves the rest of the thesis, dissertation or portfolio for unfolding the story of your research itself as it was finally planned. • There may be situations where some research scoping and shaping is done in your Literature Review chapter. This is especially relevant where your literature review has examined several different theoretical positions or research contexts

12.7

How Can You Reflect Research Scoping, Shaping & Configuring …

493

and you must choose which will provide foci for your research. Here, for example, you need to display your thinking about which theories you have chosen to follow/test and why. • Some postgraduate students may choose to reflect on some of their scoping, shaping and configuring decisions in their final thesis, dissertation or portfolio chapter where implications and limitations are discussed. Here, you could structure the discussion of limitations along the lines of key choices you had to make to achieve a feasible research project. You are then clearly acknowledging that your choices were made in full knowledge of what was sacrificed (thus creating the limitation), with good reasons, and with the understanding of what the limitation imposes in terms of how far the reader should take your results. While you might think that you are shooting yourself in the foot by pointing out the flaws in your research, the reverse is the case. A postgraduate study is even more convincing if you ‘come clean’ about the choices you have made and the limitations those choices have created. It shows your critical reflection capacity, which is one hallmark of a truly independent researcher. • Finally, we should note that for most postgraduates, a hybrid combination of the above strategies is often the most workable. Different scoping, shaping and configuring choices can be positioned within your overall research story at the most sensible points along your journey. Thus, those choices that create limitations to interpretation and generalisation might best be placed in the final thesis, dissertation or portfolio chapter. Choices about such matters as guiding assumptions, methodologies, passing gatekeepers, handling ethical issues might best be positioned in the Methodology (or equivalent) chapter. Choices that reflect broader scoping and shaping of the research (e.g., narrowing your focus, defining your problem, deciding what is of interest and what isn’t) might be best placed in the Introductory chapter or perhaps the Literature Review chapter. There is even room for some flexibility and creativity here if some scoping and shaping of your research must be done ‘downstream’ during the conduct of the study. We know of one PhD student (Muchiri, 2006) who, after configuring his PhD study with a specific analytical approach in mind (structural equation modelling), discovered, during his data analysis activities in the ‘Data Triangle’, that the statistical approach was not working, and had to design a different approach on the fly (an emergent Plan B, if you will). He conveyed that specific story in his Results and Discussion chapter as a way of defending his use of a non-standard approach to analysing the data (he knew that structural equation modelling was the expected approach for his particular configuration and research questions). Table 12.2 provides some extracts from successful PhD theses and professional doctorate portfolios that show evidence of scoping, shaping and configuring and self-questioning that led to certain decisions being made. The stories that are shown sometimes give more indirect reflections of scoping, shaping and configuring but are important parts of the story nonetheless. Note the varied chapter locations for parts of each illustrative story.

Peter’s EdD focused on using secondary databases to develop new measures of institutional performance in the Australian TAFE sector and to explore patterns and predictors of effectiveness and efficiency at the institutional and sector levels. He had to wear an important constraint with these databases in that only public sector data were accessible, thereby limiting his extensional reach and breadth of inferences.

Areej’s PhD examined recruitment and selection experiences of female administrative officers in Saudi Arabian universities. She confronted and dealt with a major cultural constraint: she was a Muslim Saudi woman attempting to conduct research in gender-segregated universities in a male-dominated Muslim culture. She interviewed male and female administrators having to deal with this power differential but was assisted by a supportive male gatekeeper to gain access to a male campus for some interviews.

“The aim of this research portfolio is to develop a framework that enables comprehensive performance measurement in the Australian TAFE1 sector.” (resolves the breadth vs depth question in favour of breadth and implicates an Australia-wide extensional reasoning intent) (Chap. 1, Introduction, p. 3) “Firstly, it is hoped that this research helps to inform policy and practice. Effective evidence-based public administration requires data and tools to utilise the data in order to assess successes and failures of policy. This study will assess several aspects of the current state of public VET provision and defensibly quantify the performance of individual service providers relative to others in the Australian context.” (identifies key stakeholders consistent with the practice-based focus of a professional doctorate) (Chap. 1, Introduction, p. 5) “The study relies mostly on secondary data that were aggregated to the institutional level.” (addresses the data genesis scoping and shaping question) (Chap. 1, Introduction, p. 22) “This study only includes publicly funded VET providers. The analysis in the first two portfolio papers is based predominantly on TAFE institutes. This is largely due to the availability of data. While the Student Outcome Survey (SOS) does collect data from private and public providers of VET education, sample sizes and survey design are only sufficient to make statistically relevant statements about public institutions (private provider data is ordinarily aggregated to the state level). Any of the results presented in portfolio papers one and two can therefore not be extrapolated to the private Australian VET sector.” (addresses limits to extensional reasoning and reflects the impact of a sacrifice trade-off) (Chap. 5, Efficiency and Effectiveness in Australian TAFE Institutes—Implications of this Research: Portfolio Paper 4, p. 267) “how institutional effectiveness can be determined using data from various survey and administrative collections.” (signals a Multiple Simultaneous MU configuration for portfolio paper 1) (Chap. 1, Introduction, p. 6)

“It is important to note here that, particularly in Saudi culture, there are strong differential dynamics when females interact with one another compared with when females interact with males. (signalling the strong gender-based power dynamic in Saudi culture and the difficulties this could create for her research) This is because of the gender-segregated culture, which means that a Saudi female can easily meet and interview female “strangers” and unfold honest authentic viewpoints, but it is difficult for a Saudi female to interview, in an authentic way, non-relative males because of religious norms, and legal and cultural restrictions. As the researcher is a female, she could be seen as an intruder when interviewing male participants on a male campus and, as a result, male participants might give dishonest answers. (acknowledging that as a female, her power in the research context is negligible and this could have implications for her data quality; initiates a sacrifice trade-off to give away desire to interview on a male campus} However, one of the senior management male participants gave the researcher exceptional access when the rector, surprisingly, issued a memo to ban

Fieger (2015) EdD Professional Doctorate

Alfawaz (2015) PhD

(continued)

Comments

Quote (our observations that are not part of the quote) (chapter location)

PhD thesis/Professional doctorate portfolio

Table 12.2 Extracts from PhD theses and professional doctorate portfolios that reflect thinking associated with scoping, shaping and configuring research (specific comments on each quote are italicised)

494 12 How Do I Scope, Shape and Configure My Research Project?

Sandall (2006) PhD

PhD thesis/Professional doctorate portfolio

“For the purposes of this study it was recognised that with the time and resources available it would not be possible to conduct an in-depth analysis in relation to all ten catchment management regions. (acknowledging resource limitations will influence scope and shape; research cannot feasibly target all 10 catchment regions) This necessitated choosing a subset to analyse. After considering the trade-offs involved between depth and breadth of analysis a decision was made to seek coverage of two catchment regions in the case study. (sacrifice trade-off in favour of depth of learning to achieve feasibility) The option of two catchment

women from accessing all-male campuses. This male participant was willing to help because he was intrigued by and supportive of the research topic, which allowed the researcher to facilitate the interview more freely instead of having to look for ways to gain permission to ask questions. (opportunistic redirection trade-off allows male campus to be utilised as research context; key gatekeeper support) The willingness of the participants also suggested that the participants’ viewpoints were open and honest.” (Chapter 4, Research Methodology, p. 86) “For this investigation, the study adopted an exploratory sequential mixed methods design (Creswell & Clark, 2011; Creswell, 2013; Östlund et al., 2011) consisting of two stages … the first stage, which was an interpretive stage employing semi-structured interviews as the main data collection strategy. … The outcomes from the first stage assisted in the development of a questionnaire that was culturally and contextually appropriate to explore, in detail, the processes that are followed in choosing candidates for administrative roles in Saudi public sector universities. In other words, the revised conceptual framework focused the construction of the survey instrument, which became the second stage of the data collection design.” (signals an Exploratory Sequential MU configuration within a Survey research frame) (Chap. 4, Research Methodology, pp. 81–82) “The researcher selected the top five universities in terms of area of population density (see Table 4.5) and according to her available resources (i.e., time and money). (signals scoping and shaping to achieve feasibility) Although university age and size could be argued to be important, the researcher only partitioned her sample on the basis of region and did not address the age and size factors overtly in this exploratory study. This was because the researcher only selected the large and old main universities, where sufficient samples could be collected.” (signals a sacrifice tradeoff, giving away potentially important ways of partitioning her sample) (Chap. 4, Research Methodology, p. 108) “The fact that the R&S of non-Saudis are different from those of Saudis—that is, different sourcing, different salary scales, based on annual contracts and on work-sponsorship bases— limited the scope of the research to Saudi participants.” (acknowledges a contextual constraint that imposes a sacrifice trade-off, where non-Saudi participants had to be excluded) (Chap. 4, Research Methodology, p. 108)

Quote (our observations that are not part of the quote) (chapter location)

Table 12.2 (continued)

How Can You Reflect Research Scoping, Shaping & Configuring … (continued)

Jean’s thesis focused on the complexities associated with the strategic management of native vegetation policy from the perspectives of people occupying different roles in the policy context. Jean’s thesis provides excellent descriptions of trade-offs and other scoping and shaping decisions she made to make her research both feasible and able to produce the learning and perspectives she was chasing to an appropriate depth.

Comments

12.7 495

Gregson (2016) PhDI Professional Doctorate

PhD thesis/Professional doctorate portfolio

“DFES, I believe, was in such a state because it was a newly formed organisation with a new leader seeking to respond to a particular critical performance review, namely the report of the Perth Hills Bushfire Review (Keelty, 2011). The performance expectations changed the operational context and meant there was a significant need to explore new approaches to

regions was selected over the option of focussing on a single region simply to enhance the diversity of perspectives on the system.” (Chap. 5, Research Design, p. 153) “A case study approach was well suited to the purposes of this study as it gives emphasis to understanding the complexities of a bounded case (Yin 1989; Stake 2000). (identifies the Case Study research frame) This matched the overarching conceptual framework adopted in the study which was to conceptualise participants engaged in formulating and implementing native vegetation policy as part of a complex social system.” (Chap. 5, Research Design, p. 140) “Thus, the methods used to elicit, interpret and represent participants’ understandings of the policy system evolved through a highly iterative process. With each round of interviews I observed and reflected on how participants were describing the policy system. This provided insights that informed the interview design.” (in conjunction with first quote above, suggests a Multiple Case-based MU configuration yielding qualitative interview data) (Chap. 5, Research Design, p. 144) “The interview design was piloted with six people involved in natural resource management policy in New Zealand. (pilot testing the MU configuration in another similar case context) This process highlighted the importance of providing people with Post-It notes and connectors ranging in colours and size. It also helped the researcher develop some phrases for encouraging people to add to their physical representations. Further, the pilot process highlighted the value of asking participants if there was anything that struck them as particularly important when they thought about the various aspects they had described.” (showcases the learning accumulated from the pilot test to help further scope and shape her research processes) (Chap. 5, Research Design, p. 158) “A practical constraint on the degree to which it was possible to use the exact wording of participants in the items was imposed by the Decision Explorer software that was used to draw the causal maps (Banxia Software Limited 2005). This software limited the number of characters that could be represented in a single item to 159.” (noting use of a constraint associated with a technological support system) (Chap. 5, Research Design, p. 166) “The interviews were recorded on audio-tape where conditions were conducive to such recording and the participant gave their consent to have the interview recorded on tape. If conditions were not conducive to recording on tape, or the participant did not wish to have the interview recorded on tape, I took notes. In two cases participants did not wish notes to be taken during the interviews. In these cases I made notes after the interview.” (adaptation on the fly trade-off) (Chap. 5, Research Design, p. 158)

Quote (our observations that are not part of the quote) (chapter location)

Table 12.2 (continued)

(continued)

Wayne’s PhD I focused on a developmental evaluation of his innovation, Portal2Progress (P2P), an online platform for capturing innovative ideas from volunteers and members of the Western Australian Department of Fire and Emergency Services. Wayne was Commissioner of this Department throughout his research, hence his scoping and shaping awareness of the potential for his organisational role and researcher role to clash and create problems for

Comments

496 12 How Do I Scope, Shape and Configure My Research Project?

PhD thesis/Professional doctorate portfolio

Comments gathering high-quality data from participants. Much of his scoping and shaping involves wrestling with the power implications of this dynamic and his ultimate acknowledgement that participants and volunteers really had the power in his context to influence the success or failure of his innovation and that made their perspectives and feedback paramount to pursue.

Quote (our observations that are not part of the quote) (chapter location)

organisational reform and the evaluation of that reform. (scoping and shaping his research as breaking new ground) (Chap. 3, Research Configuration, p. 89) “This research framework was designed recognising that the participants were the fundamental source of knowledge about what occurred during the Innovation Portfolio Project. (acknowledging who possessed the knowledge he was pursuing, thereby identifying key data sources) Members of staff and volunteers were actively engaged in P2P’s establishment and utilisation and so had the first-hand experience that allowed them to make observations in real time. In this way they were a fundamental source of knowledge about P2P and the attendant agency dynamics. The participants’ ongoing experience, understanding and individual assessments were particularly valuable in evaluating the Innovation Portfolio Project and providing intimate feedback. This feedback, as captured through the research process, provided meaningful insights into the Innovation Portfolio Project, highlighting issues of concern and observations of effectiveness; thereby providing what Scharmer and Kaufer (2013) would describe as opportunities for future enhancement. (all these arguments acknowledge participants and organisational members as key stakeholders in the research and the P2P innovation itself) (Chap. 3, Research Configuration, p. 92) “Burns further asserts that the equivalencing of quantitative data with positivism and qualitative data with interpretivism continues to occur in the literature, but that this thinking limits the value that each data type might add to research under any guiding assumptions. This Innovation Portfolio Project makes complementary use of quantitative data in an Action Research framework, thereby not limiting its considerations just to qualitative material. (scoping and shaping arguments regarding data types) (Chap. 3, Research Configuration, p. 93) “In addition, having regard to the fact that this research was being conducted in a real work setting, I was aware that at times I had to trade tightness of control for ‘richness of reality’ (signals an explicit sacrifice trade-off toward depth focus and extensional reach within his organisation) (Mason et al., 1997, p. 308).” (Chap. 3, Research Configuration, pp. 100–101) “At all times I was conscious of not allowing my perspective as researcher, or indeed as Commissioner, to have any bearing on the interpretations I was drawing from observations made by participants. (acknowledging who has the power in the research context and noting that this could create problems) At times, I found this challenging and occasionally resorted to using peers as a sounding board to ensure an unbiased interpretation of the material (scoping and shaping to help manage perspectives by putting a procedure in place to ensure a more effective research process). (Chap. 3, Research Configuration, p. 103) “A cycle of data gathering, analysis, review and reflection revolved around the key elements of informal feedback, literature review, formal organisational P2P review and organisational surveys with a view to generating ongoing improvement in P2P and forming the fundamental approach to the research.” (Case-based Sequential MU configuration within an Action Research/Developmental Evaluation research frame) (Chap. 3, Research Configuration, p. 97;)

Table 12.2 (continued)

12.7 How Can You Reflect Research Scoping, Shaping & Configuring … 497

498

12.8

12 How Do I Scope, Shape and Configure My Research Project?

Key Recommendations

Scoping, shaping and configuring your research project will probably be the most difficult early task that you face in postgraduate research. It is vital that you engage with your supervisor(s) as frequently as possible during this stage. One of the key roles of supervisors is to help you to scope, shape and configure your project in a way that maintains research quality while ensuring feasibility within the time and resource constraints you face. Without proper scoping, shaping and configuring, you run the risk of having to cope with a runaway, chaotic, essentially unmanageable project, perhaps yielding much data, but with unclear purposes and focus. There are some important things we can recommend to help you through the scoping, shaping and configuring processes. • Scoping and shaping will often involve trimming down or pruning a project rather than scaling it up or adding bits on. Many postgraduate students have bigger ideas than are feasible to work with and tangents always tend to emerge when pulling together initial ideas for a research project—especially as you work through the literature. However, it is a good strategy to start big and complex so that you can be very clear as to what and why you trim off the project. Thus, our advice is to not scope and shape your project too early; rather you should ‘complexify’ before you simplify. It is always easier to scale a project down than it is to scale it up. • It is useful early on to begin thinking about the pattern(s) of guiding assumptions that you may end up adopting as this can provide boundaries for scoping and shaping your project. • It is also never too early to begin making contact with potential stakeholders to gain insights into their perspectives on your project. This is especially important if your research has an applied or participative slant to it. • When you do begin scoping your project, keep very clear notes as to what you are pruning, and why, in your research journal. Use scoping and shaping choice questions in Sect. 12.1 to help guide your discussions in conjunction with brainstorming sessions with your supervisor(s). • As you scope and shape, be aware of the constraints you face on all fronts, including the length of your degree program, the resources you have available, family and work commitments, guiding assumptions, available support systems, what others have learned in areas relevant to your project, supervisory interests, strengths and weaknesses, and the expectations of various stakeholders. • Remember that the scope and shape of your project when you commence and plan your research may not be the final scope and shape you end up with, and that some aspects of scoping and shaping may be out of your control. This is another reason to begin touching base with relevant stakeholders early on. • Configuring your research will require some cognitive effort on your part as you try to envisage which MU configuration will provide maximal opportunity to achieve your research goals/address your research questions, in light of your research contextualisations and positionings (recall Chap. 11) and scoping and

12.8

•

•

•

•

Key Recommendations

499

shaping choices. Remember that feasibility is a critical constraining concern here and will tend to drive you toward a MU configuration that may not be the absolute ideal but will be workable for your purposes. You will find that making MU configuring choices will automatically get you thinking about choosing data sources, data types and data gathering strategies (covered in Chaps. 14, 18 and 19) as well as anticipating at least some analytical activities (see Chaps. 20 and 21). Do not get hung up on ‘mixed method’ thinking that demands that both quantitative and qualitative types of data must be gathered. You can think more broadly and openly than that. Moving beyond the single MU configuration will automatically elevate your thinking to a pluralist level. An important part of scoping, shaping and configuring activity may be the generation of a retrospective conceptual framework. This can help you to organise the literature and provide an important reference point for making decisions. Such a framework can not only help you to organise what you have read, but it can assist the reader in understanding how you have pulled the literature together. If you are working in an area where little conceptual work has previously been done to integrate the literature into a coherent conceptual framework, you may have the basis for making an original contribution to the literature in the form of a published literature review that culminates in offering your retrospective conceptual framework. Some journals will be interested in publishing such reviews if your contribution is original, clear, well-argued and offers guidance to future researchers in the area. A methodological conceptual framework may also be useful to produce as part of your scoping, shaping and configuring considerations. It can provide a map of your methodological choices and the logic behind them. This would be especially critical if you are planning a pluralist research investigation. Whether or not you need to formulate a prospective conceptual framework, an interpretive conceptual framework or a theoretical framework, will need to be judged against your adopted pattern of guiding assumptions and usefulness for helping you to tell a convincing story. A structural model would only be used in cases where: (1) you have adopted positivist assumptions; (2) you have very strong theoretical reasons to propose and test the causal paths in the model; and (3) you have used a methodology yielding data that are appropriate for use in statistically testing the structural model.

References Alfawaz, A. (2015). Recruitment and selection practices for female administrative officers in Saudi public sector universities. Unpublished Ph.D. thesis, UNE Business School, University of New England, Armidale, NSW

500

12 How Do I Scope, Shape and Configure My Research Project?

Anderson, B. F., Deane, D. H., Hammond, K. R., McClelland, G. H., & Shanteau, J. C. (1981). Concepts in judgement and decision making: Definitions, sources, interrelations, comments. New York: Praeger Publishers. Azorín, J. M., & Cameron, R. (2010). The application of mixed methods in organisational research: A literature review. Electronic Journal of Business Research Methods, 8(2), 95–105. Braund, M. (2001). Understanding public perceptions of police in the ACT through observations of police-public turf interactions and surveys of the public. Unpublished Ph.D. thesis, Department of Marketing and Management, University of New England, Armidale, NSW. Cavana, R. Y., Delahaye, B. L., & Sekaran, U. (2001). Applied business research: Qualitative and quantitative approaches. Milton, QLD, Australia: Wiley. Charmaz, K. (2014). Constructing grounded theory (2nd ed.). Los Angeles: Sage Publications. Cooksey, R. W. (2014). Illustrating statistical procedures: Finding meaning in quantitative data (2nd ed.). Prahran, VIC: Tilde University Press. Creswell, J. W., & Plano Clark, L. (2011). Designing and conducting mixed methods research (2nd ed.). Los Angeles: Sage Publications. Creswell, J. W., & Plano Clark, L. (2018). Designing and conducting mixed methods research (3rd ed.). Thousand Oaks: Sage Publications. Cryer, J. D., & Chan, K.-S. (2008). Time series analysis: With applications in R. New York: Springer. Fieger, P. (2015). Efficiency and effectiveness in the Australian Technical and Further Education system. unpublished Ed.D. portfolio, UNE Business School, University of New England, NSW. Fisher, C. (2007). Researching and writing a dissertation: A guidebook for business students (2nd ed.). Essex, UK: Pearson Education. Flick, U. (2018). Triangulation. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (5th ed., pp. 444–461). Los Angeles: Sage Publications. Glaser, B. (1992). The basics of grounded theory analysis. Mill Valley, CA: Sociology Press. Glass, G. V., Willson, V. L., & Gottman, J. N. M. (2008). Design and analysis of time-series experiments. Charlotte, NC: Information Age Publishing. Gregson, W. (2016). Harnessing sources of innovation, useful knowledge and leadership within a complex public sector agency network: A reflective practice perspective. Unpublished Ph.D.I portfolio, UNE Business School, University of New England, Armidale, NSW. Harrison, J. (2003). Information scope in small service firms: A comparison of universalistic, contingency and configurational theoretical approaches. Unpublished Ph.D. thesis, University of New England, Armidale, NSW. Henryks, J. (2009). Organic foods, choice and consumer context: An exploration of switching behavior. Unpublished Ph.D. thesis, School of Business, Economics and Public Policy, University of New England, Armidale, NSW. Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24(4), 602–611. Johnson, R. B., & Onwuegbuzie, A. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. Kaptchuk, T. J. (2001). The double-blind, randomized, placebo-controlled trial: gold standard or golden calf? Journal of Clinical Epidemiology, 54(6), 541–549. Kincheloe, J. L. (2005). On to the next level: Continuing the conceptualization of the bricolage. Qualitative Inquiry, 11(3), 323–350. Maxwell, J. A. (2005). Qualitative research design: An interactive approach (2nd ed.). London: Sage Publications. Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis: An expanded sourcebook (3rd ed.). Los Angeles: Sage Publications. Morse, J. M., Stern, P. N., Corbin, J., Bowers, B., Charmaz, K., & Clarke, A. E. (2009). Developing grounded theory: The second generation. Walnut Creek, CA: Left Coast Press. Muchiri, M. (2006). Transformational leader behaviours, social processes of leadership and substitutes for leadership and their relationships with employee commitment, organisational

References

501

efficacy and citizenship behaviours. Unpublished Ph.D. thesis, New England Business School, University of New England, Armidale, NSW. Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: The Guilford Press. Ross, E. (1999). Academics’ perceptions of university culture and the factors that facilitate or inhibit commitment to it. Unpublished Ph.D. thesis, Department of Marketing and Management, University of New England, Armidale, NSW. Sandall, J. (2006). Navigating pathways through complex systems of interacting problems: Strategic management of native vegetation policy. Unpublished Ph.D. thesis, New England Business School, University of New England, Armidale, NSW. Smyth, R., & Maxwell, T. W. (2008). The research matrix: An approach to supervision of higher degree research. Milperra, NSW: Higher Education Research and Development Society of Australia. Spector, P. E. (2006). Method variance in organizational research truth or urban legend? Organizational Research Methods, 9(2), 221–232. Strauss, A., & Corbin, J. (1990). The basics of qualitative research: Grounded theory procedures and techniques. London: Sage Publications. Tashakkori, A., & Teddlie, C. (Eds.). (2010). Sage handbook of mixed methods in social & behavioral research (2nd ed.). Thousand Oaks, CA: Sage Publications. Teddlie, C., & Tashakkori, A. (2011). Mixed methods research: Contemporary issues in an emerging field. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (pp. 285–299). Thousand Oaks, CA: Sage Publications. Wolodko, K. (2017). The emergence of group dynamics from contextualised social processes: A complexity-oriented grounded-theory approach. Unpublished Ph.D. thesis, UNE Business School, University of New England, Armidale, NSW. Yin, R. K. (2014). Case study research: Design and methods (5th ed.). Los Angeles: Sage Publications.

Chapter 13

How Should I Select, Read and Review the Literature?

13.1

The Literature Review

Creating the literature review can be one of the most interesting and rewarding components of postgraduate research. However, with the vast array of information available both online and in hard copy from a diverse range of sources potentially relevant to your topic and related disciplines, it can also be intimidating. As part of the proposal approval process, a postgraduate student will already have had some exposure to the more obvious literature in the field but, like an iceberg, this is usually just the tip of the mass of material available and doesn’t yet include the literature that will be produced in the years prior to the final submission of your thesis/dissertation/portfolio. Most students find undertaking a literature review rather daunting. There is a constant fear that something may be missed out, as well as the sheer burden of encapsulating a considerable volume of material into a meaningful and coherent chapter that meets the criteria for critical evaluation. While the main domain of research may be well-known, it is often the related and impinging discipline discussions that cause the greatest consternation. How much further should you go into the additional areas? What constitutes critical evaluation? When do you stop? Do I only look at books and journal articles or do I need to read further afield (e.g., professional and trade journals, government, industry and organisational/institutional reports, media stories)? These and other questions can be vexing in the preparation of a literature review.

© Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_13

503

504

13

How Should I Select, Read and Review the Literature?

13.1.1 I Just Want to Get on with My Study So Why Can’t I Do the Literature Review Concurrently with Data Gathering? Some students are so anxious to get on with their research journey that they rush into data gathering thinking it is the heart of their research. Then, one day, when reading a paper, they realise that what they were planning on doing has been done before. A devastating statement that you hopefully will not receive from your supervisor, or the reviewer of a journal or conference paper, is the simple phrase, “There is existing literature on that”. This means that topic has already been thoroughly studied by many people and you have clearly failed to do your homework to discover this. Also, “you have just wasted a chunk of your life re-inventing the wheel” (Petre & Rugg, 2010, p. 22). The more time you spend reading widely at the beginning, the stronger the foundation upon which your research will be built, and the more currency it will have. It will also be more informed, be based on prior research outcomes, and will have considered a better selection of research theories, concepts, patterns of relationships, constructs and/or variables, as you will also have had exposure to a large array from which to choose and further develop your ideas. Furthermore, you will have been exposed to a range of methodological approaches that others have employed, something that can help in the development of your own methodological approach. Working effectively with the literature is so important that a distinct meta-criterion, Juxtapositioning with other research, is devoted to it. One important starting point in research is an understanding of the body of knowledge relevant to your topic. This relates both to the theoretical base and to prior research findings in the field. The aim of the literature review is to document all this material in a meaningful way. Although most of your material will be drawn from peer-reviewed academic journals, dissertations, research books and textbooks, additional material may be sought from other sources, including grey literature such as working papers, trade, professional and practitioner publications and government or industry reports. Your supervisor(s) should know the seminal papers and leading researchers in the field and will be able to provide guidance. Whatever the source, there will always be ongoing concern regarding the credibility and relevance of the material you gather. For credibility, consideration needs to be given to the source and intent of the material, that is, the organisation from which it is derived, the target audience(s) of the material, the reputation of the authors, the purpose(s) for which it has been produced and the recency of the material. Relevance is also an issue, particularly when there is a vast pool of information available to you. Sometimes students are so intent on gathering information, they have little time or inclination to distil what is important, what is not, and what relevance it has to the specific research project on hand. Consequently, postgraduate students can become passive pursuers rather than active critiquers of knowledge. You have to learn to see the trees of most interest and value in the forest of literature you will encounter.

13.1

The Literature Review

505

13.1.2 So, What Is the Purpose of a Literature Review? One of the common criteria for awarding a postgraduate degree is that a student is able to demonstrate a knowledge of the literature relevant to their subject and the field or fields to which the subject belongs, and the ability to exercise critical and analytical judgment of it (The University of Auckland. Doctoral Policies and Guidelines, 2018). Another criterion relates to the need to make an original contribution. To be able to make an original contribution within your postgraduate research, you need to have a good grasp of what has already been written. It is that grasp that allows you to appropriately contextualise and position your own research within relevant fields and disciplines (recall our preliminary discussion in Chap. 10, Sect. 10.5). Collecting an idea from one piece of literature and connecting it to another idea in another paper can create the basis of your thinking for your research questions and for the concepts you will be exploring. Originality emerges when you identify a unique combination of ideas that you are using and examine them in ways and/or contexts not previously done. Gaining a wide appreciation of the relevant literature can be challenging but is a prerequisite to building the intellectual foundation to support you later on. It would be virtually unforgivable to have a seminal paper pointed out to you while presenting at a workshop or conference, or by an examiner. Literature reviews serve a variety of functions such as: • prompting the identification of research gaps and questions that could be addressed; • identifying relevant theories that are of value to your research; • recognising/developing various classifications or schema for existing literature; • providing a critique of prior research and what some of the limitations of prior studies might be; • suggesting methodological approaches that have been used in the past, including potential research instruments; • suggesting alternative data analysis strategies and ways of presenting analysis outcomes; • demonstrating the building of theory from data; • providing an indication of potential applications of prior research; and • indicating potential concentrations of research interest in specific publications which may be of interest to you for your own publication intentions. Taking a slightly different perspective, Thody (2006, pp. 91–93) indicates that the purposes of literature reviews are to: • provide a general overview of the area of your research; • justify your research by showing that others have not already addressed your topic, or researched it in the same way; • pay homage to those who have gone before you and acknowledge whose work has influenced your thinking;

506

13

How Should I Select, Read and Review the Literature?

• demonstrate your analytical and critical skills; • establish the credentials of your research which may be important where others have investigated the same general area; • reveal a current understanding of your topic so you can more easily demonstrate that you have added to it in your research; • facilitate the emergence of your research topic and data-gathering methods; and • show how you generated your conceptual framework. We should add to this list another important purpose for the literature review, which is, to critically evaluate previously used methodologies and patterns of guiding assumptions, research contexts, procedures, measures and instruments with an eye toward identifying successful strategies to pursue as well as blind alleys to avoid. While there is clearly the external benefit of reassuring examiners that you are familiar with the body of knowledge, in our view, for a postgraduate student, the personal and principal benefits of doing a literature review are three-fold and relate to topic generation, advising on methodology, and theoretical development. Reading material in your discipline areas can stimulate and guide your topic generation. Ideas for potential research topics can often be found not in the introduction of a research paper, but in the conclusions and implications where the authors discuss future research suggestions, i.e., what they would do next given the findings they have discovered. These suggestions for future research could be used as input into your consideration and search for a unique topic. The literature can further help to define and limit the problem you are working on (a key role of contextualisation) and can guard against unnecessary duplication of research. Finding out what some of the unresolved questions or areas are that have not been explored can be exciting, and your topic and research questions will be enhanced by looking at previously constructed conceptual frameworks, avoiding already well-trodden paths and finding new ground. A review of the literature enables you to evaluate promising research methods, particularly in regard to where and how the data have been accessed in the past, what the common and uncommon data gathering strategies used were, what typical participation rates were, what the limitations of these studies were that potentially weakened their research (which you can avoid), and how data were analysed and presented. Existing theory will provide a conceptual foundation for your thinking at the start of your study but will also be of value when you interpret your own findings. Initially, consider what theories were used to guide projects like your own once you have data on how your findings differ from the outcomes of other studies. From this comparative work, you will be able to contemplate further modifications to existing theory, develop new theory and ascertain what the strategic implications of your research learning might be.

13.1

The Literature Review

507

13.1.3 What Are the Common Problems Associated with Literature Reviews? It is true that literature reviews are not without their problems. Finn (2005, p. 105) has identified some common pitfalls: • The objectives and scope of a review are not well-defined. This could result in a superficial, broad and shallow approach rather than a more focused, narrow and deep approach. • There is too much emphasis on summary with insufficient attempts either to critically evaluate the research material or provide synthesis. Remember that your own understanding and evaluation should be evident throughout. • Important conceptual developments are either not referred to or they are explained incorrectly. • There is limited scope in the reading material with over-reliance on a limited range (and/or quality) of references. Older seminal papers and recent important research are not referred to. • There is an over-reliance on websites and general textbooks, although this is less common in postgraduate research. In increasing order of priority, reviews should typically focus on peer-reviewed journal articles, primary sources, theses and dissertations. For professional doctorates, grey literature (e.g., government, industry, organisational and professional reports and working papers) can be especially important to access and deal with. • There are numerous, obvious mistakes that indicate inadequate proof-reading, for example, typographical errors, poor grammar, repeated sentences, improperly cited or quoted material, a paragraph that has been pasted more than once, or references in the text are absent from the bibliography and vice versa. An addition to this list could be the use of suspect references, for example, Wikipedia or unpublished work on an individual’s website. A recurring problem is a lack of closure to the arguments presented and/or reliance on uncritically considered or injudiciously selected anecdotes and opinions. This problem amounts to not answering the ‘So what?’ question. A valuable literature review argues very clearly for the implications of what has been learned for providing foundations for your own research as well as implications for future research, theory and/or practice.

13.1.4 How Do I Go About Creating a Good Literature Review? To assist you in the creation of your literature review, we recommend a six-step process, as follows:

508

13

How Should I Select, Read and Review the Literature?

1. Preparation—this involves preparation for the search process and, notably, the creation of a list of key words. The key words will naturally emerge from your initial defining of the research topic; you will further add to, refine and focus your list along the way. 2. Searching—investigating and obtaining from library and electronic sources material relevant to your study, including meta-analyses, meta-syntheses and other published analyses of literature. This is a good time to consult with a subject librarian at your university. Bibliographic databases will also be very helpful for locating not only relevant papers but also the leading academics around the world in your subject area. 3. Reading—getting through the mountain of material in a meaningful way. This involves skimming and scanning as well as active and critical reading, focusing on contexts, concepts, theories, methods and ideas that researchers have studied, applied and/or examined before. It also includes reading research published across as well as within paradigms/patterns of guiding assumptions and actively looking for gaps or weaknesses that could inform your research problem/ questions. 4. Taking notes—appropriate recording of your references into a bibliographic file and summarising key observations. This is possible in bibliographic software such as EndNote. If it suits your mental style, you may find mindmapping to be a useful tool for summarising and conceptually linking and integrating what you read in the literature. Key observations, thoughts and ideas that emerge during your reading should also be recorded in your research journal where they can be connected to other aspects of your research journey. 5. Writing—where you structure and critically evaluate what you have read as well as the appropriate referencing of material you have written about and build arguments focusing on any gaps, weaknesses andf research opportunities you have identified. 6. Revision—undertaking the necessary and on-going updating and revision of your literature review to ensure that it is current and relevant to your study. You will do this right up until the time you prepare your final draft for submission. For more in-depth exploration of issues and processes associated with preparing a literature review, see, for example, Booth and Sutton (2016), Cooper (1984, 1988), Cooper and Hedges (1994), Gash (2000), Hart (2001), Pautasso (2013), Polonsky and Waller (2015), Randolph (2009) and Ridley (2012).

13.2

Preparation

Given the variety of topics and the subsequent material generated, there is a range of potential circumstances that a postgraduate student may encounter: • No research has been undertaken on the problem. This depicts a ‘greenfields’ scenario. This is unlikely and can make the literature review simple but

13.2

Preparation

509

awkward to write. You have to depend much more upon logical and conceptual argumentation in such a situation, precisely because you are breaking new ground where literature guidance is thin. • Some research has been done on the problem; therefore, you will be expected to cover most or even the entirety of what has already been published. • An abundance of research has been produced. The difficulty here would be in trying to cover the ground, appropriately showing depth and breadth in a conceptually coherent way. • There is an abundance of related literature. Here the pressure is to not only provide cover but also to provide categorisation of the research. Hence there is a stronger need to create identifiable themes in the literature (Kilbourn, 2006, pp. 554–555). The material you are sourcing for your literature review can be drawn from several locations, typically, journal papers, both online and hard copy, books, conference proceedings, media (although valued less), published reports, government, industry and professional documentation, theses, dissertations and portfolios. In a research project guided by the positivist pattern of guiding assumptions, the literature review is usually located after the introduction and before the methodology, thus informing the reader that the author has become fully acquainted with the body of knowledge before commencing their own study. This is not to say that elements of the literature review will not occur in other sections of your thesis/ dissertation or portfolio. For example, literature items may appear in your methodology chapter, where you may be developing your own methodological approach through comparing and contrasting the advantages and disadvantages associated with previous approaches in the literature. Related literature may also appear in your conclusions and implications, where your findings may be compared with the outcome of prior studies. So, when speaking of the literature review, we are not necessarily referring to one specific chapter. For example, in many current PhD theses, one sees literature reviews at the beginning of several chapters or integrated into several chapters instead of having one literature review for the whole project. This is a change from the orthodox PhD thesis structure, but it is increasingly common, especially so for professional doctorates and for PhDs by publication. Many students find reviewing the literature to be a somewhat tedious process because of the volume of what seems to be unrelated material. The good news is that everything you can discard brings you closer to identifying what is relevant. Think of it like following a line of breadcrumbs to your ultimate destination. It’s a path you need to follow. Some papers will be exciting and right on the topic, others will seem to be superficial and only tangentially related. However, the boundary is very fuzzy and when they fall near the boundary, things can get tricky. Often, at the start of your research, you just don’t know whether a specific paper is important or not and it is only when you have been thoroughly immersed in the literature that you will become a better judge. So, for the first six months, you may have a lot more literature in the relevant than in the irrelevant pile but, as you start to firm up

510

13

How Should I Select, Read and Review the Literature?

your research area and key dimensions of the study, your improving critical eye may shift literature you previously thought was relevant over to the irrelevant pile and vice versa.

13.2.1 What Strategies Do You Recommend for the Preparation Phase of the Research? In order to have a systematic approach, Hart (2001, p. 23) has suggested the following steps when undertaking literature search preparation and planning: • Define the topic—write down the main topic and which disciplines relate to that topic. Generate your list of key words. Build some themes from your literature scan and develop key words associated with those themes. Keep the list of words handy or in your research journal and keep adding to it. • Think about the limits of your topic—limit your search by placing parameters around, possibly, the timeframe, dates, languages, contexts, places and populations. • Identify the main reference tools for your discipline—learn which indexes, abstracts and databases may apply and cover your topic. These tools may need to be more diversified if your research is multi- or transdisciplinary. • Plan the sources to be searched and start your search—list the sources you intend to search and the order in which you intend to search them. Keep a record of searches you’ve undertaken and the key words and their combinations in your research journal. For the more electronically able, saved lists is a customised feature of many searchable databases. You could also utilise email alerts and RSS feeds of searches. • Think about housekeeping—how are you going to record what you find? Here again your bibliographic reference database such as EndNote and your research journal may prove useful.

13.2.2 What Is the Best Way of Generating Key Words? Here are some useful strategies for developing key words for your search of databases, the library and the internet: • Create a search strategy document, identifying the main concepts and topics to search, in conjunction with your library liaison or subject librarian and keep adding new concepts as your research progresses. Consider alternative spellings (e.g., ‘judgment’ as well as ‘judgement’), variant forms of words and synonyms, as well as the various subject headings, descriptors or categories. Look at how concepts are combined in an advanced search and how you can expand or limit

13.2

•

• • •

•

•

Preparation

511

your searches. Use a thesaurus to build up your vocabulary on the topic by identifying words that are similar to those you are using. In some instances, databases may have a built-in thesaurus which you will be able to use. Check descriptors or subject headings pertaining to your topic and use them for broad searches or combine them with your key words. Check the Database Help for this. For online assistance, https://www.thesaurus.com/ provides a thesaurus, dictionary and writing resources and https://www.yourdictionary.com/ offers multi-language dictionaries. Read extensively. Your list of key words will continue to be enhanced as you read more literature and new words come into your vocabulary in relation to the topic. Enhancements will involve more fine-grained distinguishing words, useful in narrowing down the size of resulting search lists. For example, if your topic is ‘decision making’, using those two words as key words for a search will likely net you many thousands of results. To pare these possibilities down, you will need more precise terms that may target: a theory (e.g., ‘image theory’, ‘social judgment theory’), an author (e.g., ‘Daniel Kahneman’, ‘Ken Hammond’, ‘Lee Roy Beach’), a concept (e.g., ‘anchoring and adjustment heuristic’, ‘groupthink’ or ‘intuition’), a context (e.g., ‘strategic decision making’, ‘judgment under time pressure’, ‘disease diagnosing’), a research approach (e.g., ‘judgment analysis’, ‘process tracing’) or a tool or application (e.g., ‘analytic hierarchy process’, ‘brainstorming’, ‘decision conferencing’). Many journal articles now list key words which are usually located straight after the abstract. Incorporate those key words into your list for your future literature searches. Look up the synonyms relevant to your core key words. Try word stemming—some words have the same stem/beginning. For example, feminism, feminist and feminine all have the same stem femin, so the suggestion is to use ‘femin?’ or whatever symbol is used by that database for truncation. By using the truncated symbol of ‘?’ or ‘*’, you will retrieve all variants (suffixes) related to that stem. The downside of this approach is that you may receive too many results from the search, but it is worth doing a broad search and then narrowing it down (Hart, 2001, p. 143). When undertaking a literature search, you need to be using current as well as historical terminology. Frequently, a new phrase or terminology enters the lexicon of a discipline and is adopted by a group of researchers to describe their work. Keep a watchful eye out for a new spin on an old topic as new words come into vogue quite quickly. It is appropriate to learn what these terms are if you are to capture the more recent literature. Some of the literature may be in another language. Try searching the key words in other languages that may be relevant. Not only look at topics but also look at authors. Most academics tend to concentrate in a narrow area of activity; therefore, doing searches of authors’ and co-authors’ names can lead you into a new paper trail which could be quite beneficial. You can find and track key authors in the Scopus and Web of Knowledge bibliographic citation databases.

512

13

How Should I Select, Read and Review the Literature?

• Search citation databases to map out research, authors and journals forward in time, setting up your update alerts as you go. • Keep a list of the key words and terms that you have generated. Also, as you run database searches, keep a note on a piece of paper beside you of what the search was, otherwise you will find that in a long session you are starting to repeat yourself. So, record the different combinations of key words or phrases that you have used.

13.3

Searching

13.3.1 How Do I Become Acquainted with Everything the Library Can Provide? In the initial stages of your literature review, digital databases will be your primary sources, but many students miss out on all that the physical library can provide, especially the support that librarians can provide given their significant knowledge of data search, recording of literature and the setting up of alerts. Make yourself fully acquainted with the resources available to you. In relation to the physical aspects of the library (as opposed to the electronic sources through your computer), we suggest you take advantage of any library tours or courses that are being conducted. The aim of attending a library course is usually to: • • • • •

become familiar with the library catalogue; learn how to use inter-loan library facilities; be able to search journals and identify databases relevant to your area; search journals and electronic databases to find data; and learn how to build your private reference database, how to store and manage your references, and how to generate bibliographies and citations (e.g., in Microsoft Word using EndNote, EndNote Web, Zotero or other programs).

Libraries run regular information skills classes and have expert librarians who specialise in a particular range of disciplines. Discover which expert librarian is assigned to your school and get to know them; they will be an invaluable source of information and support during your candidature. Library websites contain important information about research information gathering and dissemination, and often have excellent online tutorials and frequently asked questions. Find out about the library’s holdings in your discipline. Your home library, the library of your university, may not, in fact, be the best library for you. Most libraries carry out an evaluation of the strength of their collection in subject areas. This is based on the Conspectus method for evaluating libraries. The system enables a library to evaluate and record by each subject its current holdings and the strength of its collection. This information is obviously useful for informing library users of collection policies and where a collection may require further building. It is also

13.3

Searching

513

used as a basis upon which collaborative relationships can be formed with other libraries. Library resources are usually divided into 24 broad subject divisions and, within those divisions, each subject is further delineated in accordance with the Dewey Decimal Classification System. In Conspectus, each subject area is graded using a code of 0 to 5, with 5 indicating the more comprehensive holdings and zero indicating that the library does not collect on the subject. This is referred to as ‘out of scope’. Investigate by asking a senior librarian what the Conspectus rating is in your subject area(s) and for other libraries.

13.3.2 What Questions Should I Ask? • Does the library provide any special support or services for postgraduate research students? • What training does the library provide, e.g., groups only or one-to-one, and is there a limit to the number of sessions you can attend? • Do they help with literature searches? • How long can books be borrowed for, and how many times can they be renewed? If I have to recall a book I want that has been borrowed by someone else, what is the typical waiting period? • If I need to obtain books or journals via inter-library loan for my topic, do I have to pay and is there an institutional or departmental limit on support for inter-library loans? • Can the library post material to me and, if so, are there costs involved? • When is the library open, e.g., are there reduced hours over summer which could affect some people? • What does the library hold in my subject area (books, journals, databases)? How much depth does the collection have (e.g., is locating historical material a problem)? • Can I access the catalogue and databases online from home? • Can I get support from the library for reference management software, such as EndNote? • Can I get specialist library support from home for my research topic? • What facilities are available in the library, e.g., individual study space? Is there a special area for postgraduates? • Does the library have access to archives (depends a bit on the subject area)? Today, most students find trawling through hard copies of journals and books a bit old-fashioned but, as some databases do not contain historical material, this will be required. More commonly though, an extensive amount of literature can be obtained from a variety of databases for which your library will have a subscription, and which will form the basis of your search activity. Researchers now have access to an unprecedented wealth of online information tools and services. Borgman

514

13

How Should I Select, Read and Review the Literature?

(2007) describes the role that information technology plays at every stage in the life-cycle of a research project. There is a difference between digital and internet searching. Internet searching is for sources on the internet; digital searching is using online indexes, abstracts and databases to source additional, more credible material.

13.3.3 What Advice Is There for Web Searching for a Researcher? There is a variety of information that can be obtained from the internet, these being electronic journals, citation indexes, news, blogs, data archives and sources, subject resources, bibliographies, relevant reference sources, statistical and factual information, research papers and articles, library catalogues, chat forums and digital collections (Hart, 2001, p. 129). As well, you may be able to access various types of grey literature from the internet. There are different ways to search the web: • Meta search engines—search engines which allow you to search across a range of engines at once, for example, http://www.search.com/ or http://www.dogpile. com, http://www.windweaver.com/searchlinks.htm. The downside is that you will create quite a volume of material and, because different search engines are searching using different rules, it will be difficult to do more precise searches. • Search engines—search engines are more focused and enable you to search using key words, for example, https://www.google.com/. Specifically, Google Scholar http://www.googlescholar.com/, is good for searching academic research outcomes; LexisNexis http://www.lexisnexis.com provides searchable access to legal and business reports, or you may just want to take a more general sweep of homepages. If you wish to learn more about search engines, investigate http://www. searchengineshowdown.com/, which is the site that reviews the search engines and their strategies. The search engine Colossus has links to search engines from 148 countries (http://searchenginecolossus.com). For help in developing your online search skills, see: • http://guides.lib.berkeley.edu/evaluating-resources, • https://www.scholastic.com/teachers/articles/teaching-content/6-onlineresearch-skills-your-students-need/, or • https://www.elearningindustry.com/7-tips-enhance-online-research-skillselearning. For general development of research and searching skills, see: • https://www.futurelearn.com/courses/searching-and-researching. • https://www.class-central.com/tag/research-skills.

13.3

Searching

515

The web can provide not only literature but also relevant secondary data that could be of value to the research student. Internet searches can be improved by using more effective search strategies, such as: • + symbol: use of the ‘+’ symbol enables you to add terms together in your search. For example, if you are looking at a specific geographic area, or a industry in relation to your topic, you would use, e.g., +business+ethics+IT. Consequently, only the web pages that contain these words will be retrieved. • − symbol: the ‘−’ symbol subtracts terms from the search to give more precision. You may have found that when you undertook a search there were several irrelevant additions that were included in your search. You can delete these by using the ‘−’ symbol. • Phrase searching—this is where you would use multiple terms for increasing the search by using quotation marks. You can restrict your search to web pages which have the correct sequence of words, for example, if you searched business +ethics, you would probably get far too many items in your search. However, by putting in “business ethics”, those two words must come together. Only put in word combinations in the order in which they are likely to be used. Each database will have a Help guide which will assist in fine-tuning your search strings. • Combining the search symbols—involves using Boolean searching methods. With Boolean searching one combines the English language with algebra to narrow the searches. This can create a very precise search, for example, “business ethics”+Asia-Taiwan-Japan. • Geographic focus—if you want to focus on a particular country, you can use the following search frame, e.g., “business ethics”+host:UK. The suffix can be changed if you are just looking at educational sites, e.g., +host:edu or to retrieve only Government websites, +host:gov.uk (Hart, 2001, pp. 142–143).

13.3.4 Can Material on the Internet Be Trusted? The internet is not without issues of concern regarding the legitimisation of data on the net and observers have even gone so far as to say, “The web is a treacherous, unreliable and unusually amateurish source of information, misinformation and downright lies” (Petre & Rugg, 2010, p. 28). The concern is that the internet has virtually no controls regarding the quality of the sites accessible through it observing “If you find an interesting site relating to your chosen area, it may have been written by a major authority or it could just as easily have been put together by someone who believes they are being controlled by devices put in their brain by aliens” (Petre & Rugg, 2010, p. 60). Another concern for postgraduate students and textbook writers is the issue of stability/availability over time. Essentially, is the information going to be around for a while?

516

13

How Should I Select, Read and Review the Literature?

For a discussion on evaluating the quality of material on the web, see: • https://www.library.georgetown.edu/tutorials/research-guides/evaluatinginternet-content, • https://peopledevelopmentmagazine.com/2016/07/10/information-internet/, or • https://library.nmu.edu/guides/userguides/webeval.htm. Finally, many students ask about using Wikipedia as a source of information for literature reviews. Debate is on-going about the reliability and accuracy of Wikipedia relative to other reference materials (see, e.g., https://www.quora.com/ How-reliable-is-Wikipedia-as-a-source-of-information-and-why). You should not rely on Wikipedia as a primary literature source (not least because the author(s) of Wikipedia contributions may not be known). However, it could be useful as a back-referencing aid, pointing the way to more authoritative sources. Wikipedia would probably rank ahead of newspaper and other types of media reports, perhaps on a par with textbooks, but well behind peer-reviewed journals in credibility.

13.3.5 Digital Searching With online searching, you could be looking at indexes and abstracts as well as full text databases. Given that you will want to source the entire paper; you will be spending more time with databases such as: • ISI Web of Knowledge; • Scopus (abstract and citation database); • Business Source Premier, which provides full text for more than 1,000 peer-reviewed titles; • Academic Search Premier, which is a multi-disciplinary database of nearly 4,000 peer-reviewed titles and files that go back to 1975; • EconLit which is the American Economic Association database; • EBSCO Host which can also be located under EJS E-Journals, which concentrates on e-journals; • ProQuest which sources from over 9,000 publishers; • ThomsonGale, an e-research and educational repository; • Computer Source which focuses on information technology and IS with full text for 300 publications; • Media Sources, which has full text for US national newspapers, television and radio; • SocIndex is a sociology database; • PsychInfo for psychological and general behavioural science material; and • Emerald full text, a full text database for Emerald journals. Databases regularly get bought out and merged, so watch for name changes. Not all institution libraries will have the same databases, and you may need to go to more than one library to access relevant information. The British Library (http://

13.3

Searching

517

catalogue.bl.uk) apparently holds the best catalogue if you are trying to locate specific textbooks from the UK. In the US, the equivalent is the Library of Congress (http://www.loc.gov). Check with your library how to obtain remote access to your library’s databases. This can certainly speed up the research process for you, because you could be accessing material from home at any time, rather than having to come into the library. Commonly, you will only need your student ID; however, you may require an additional password when accessing full text versions from your institution’s databases. When searching databases, break down your research statement and combinations of key words and check your spelling. There may be variations in the search symbols used for each database, so you are advised to spend a bit of time looking at what the search requirements are for the database you are using. Generally, librarians who deal with these databases daily will be up-to-date and conversant with the symbols if you prefer to talk to someone. Alternatively, your university may run a training program on database searches, and you would be well-advised to attend early on in your studies. Many online databases use Boolean logic to facilitate more complex patterns of searching. Common Boolean terms are as follows: AND

words on both sides of this operator must be present somewhere in the document for it to be identified by the computer as a result, i.e., X AND Y. OR words on either side of this operator are sufficient to be identified as a result, i.e., X OR Y. AND NOT documents containing the term after this operator are rejected from the results, i.e., X AND NOT Y. NEAR terms must be within a specified word distance from one another to be identified as a result, i.e., X NEAR Y (therefore like AND). BEFORE only the first term before this operator (within a specified word distance) must be present for the computer to identify it as a result, i.e., X BEFORE Y. AFTER only the first term after this operator must be present for the computer to identify it as a result, i.e., X AFTER Y. Phrases a set of words that must be adjacent to one another in order for the computer to identify the document as a result, i.e., X Y. Wildcards the root of words (sometimes called stem) that the computer must find for the document to be identified as a result, e.g., wom? = women and woman (but may also bring in completely unrelated words, e.g., wombat, so care is needed). Quotation marks A way of grouping words or phrases within a statement so that only exact matches are identified, e.g., ‘literature reviewing’ (Hart, 2001, p. 153). When using multiple concepts in your research, it is tempting to add a whole stream into your search; however, it is probably better to keep them down to a manageable size of, say, three concepts.

518

13

How Should I Select, Read and Review the Literature?

13.3.6 What Is Back-Referencing? Although it is not commonly discussed, there is a very useful literature search tactic you can use to gain a quick overview of an area of research. This is a technique called ‘back-referencing’ (Phelps, Fisher, & Ellis, 2007, p. 130, briefly mention, but do not elaborate on, a related technique they called ‘citation chaining’). It simply involves identifying a key or seminal research paper or text in the area (or, even better, a published meta-analysis). Once you have identified such a paper, search through the list of references that the author(s) have cited. This will allow you to see what literature the author(s) thought was important to work from to build up their own arguments. By exploring some of these identified references, you could then back-reference through them as well, building up a cadre of references in relatively short order (think of this type of search pattern as building up a branching tree of references that works back through time). Ensure that you check all referenced articles and have sighted the primary source. Failure to do this can result in reference inaccuracies and misunderstanding of content, which are then passed on. This type of back referencing search can quickly point you to other references to follow up. In fact, back-referencing can give you a nice way to achieve a bit of focus since you are relying upon the judgments of other authors to signpost key references. However, you should also realise that such reliance is a two-edged sword. You should always use back-referencing with a critical eye because, even in peer-reviewed published work, authors may have been selective in the literature they have cited. Generally, this could mean either that the author(s) sacrificed breadth of literature covered in favour of depth in selected areas to meet the length constraints of a journal, or that the author(s) were actually biased in their choices of literature to examined (a peer-reviewed process can help to reduce this risk, but not eliminate it). Make sure you keep a balanced view, which means you should only use this technique in conjunction with other searching methods so that you are not over-reliant on the judgements of others in targeting which literature to read. Perhaps the best time to employ the back-referencing technique is early in your exploration of the literature where you want to get ‘your toe in the water’ to explore and evaluate an area of research before you commit to diving head-long into it.

13.3.7 What Is Cited Reference Searching? Cited reference searching allows you to follow research forward in time. You can follow how an article is cited in other articles after it was published. Seminal articles can be identified in this manner, assuming that the larger the number of citations, the greater the popularity of the article (as long as the citation count is not heavily biased by self-citations). Using this method, you can also trace the citation counts of authors and identify the active researchers and research teams in a narrow field of research. You will, through this method, get a feel for the papers which are more

13.3

Searching

519

popular and gain more attention. Setting up alerts on articles, authors and journals is possible through some of the databases so that you can monitor publications going forward. One useful feature of Google Scholar is that it automatically reports the number of times each research item it identifies in a search has been cited by others. Furthermore, you can view the list of other researchers who have cited the work, simply by clicking on ‘Cited by’ link below each returned search item.

13.3.8 Finding Relevant Grey Literature? Grey literature can be defined as that “which is produced on all levels of government, academics, business and industry in print and electronic forms, but which is not controlled by commercial publishers” (Schöpfel et al., 2005, cited in Farace & Schöpfel, 2010, p. 1). It encompasses such material as theses and dissertations, reports from all levels of society (government, business, community, academic), working papers, discussion papers, conference proceedings (if not produced by a commercial publisher), patents, court opinions, newsletters and preprints of articles (see http://www.greynet.org/greysourceindex/documenttypes.html for more comprehensive details). Grey literature is not typically not consistently captured in digital reference databases, so you need be more targeted in your searches. In fact, you may need to know pretty much what you are looking for, at least in terms of type of document, topic and/or document source. Some examples are: • a thesis or dissertation (e.g., https://oatd.org/, a searchable open access web resource); • an annual report for a company (e.g., http://www.annualreports.com/, a searchable web resource); • a government report on national minimum wage, water use or some other issue of public interest (e.g., https://www.australia.gov.au/about-government/ publications/annual-reports; https://www.archives.gov/research/alic/reference/ govt-docs.html; https://www.gov.uk/government/publications?official_ document_status=command_and_act_papers); • an institutional position statement on climate change (e.g., https://sciencepolicy. agu.org/agu-position-statements/); • a report on investigations into nurse-patient ratios (e.g., https://www.health.qld. gov.au/ocnmo/nursing/nurse-to-patient-ratios); • a royal commission investigation into trade union governance and corruption (e.g., https://www.aph.gov.au/About_Parliament/Parliamentary_Departments/ Parliamentary_Library/Browse_by_Topic/law/royalcommissions); • a policy discussion paper (e.g., http://www.acas.org.uk/index.aspx?articleid= 2988); and • a research working paper (e.g., many university departments offer a series of working papers reporting on research by their staff members before that research

520

13

How Should I Select, Read and Review the Literature?

has been peer-reviewed and published, and they may offer a website that provides access to such papers; for example, https://melbourneinstitute.unimelb. edu.au/publications/working-papers; https://www.chicagobooth.edu/faculty/ working-papers). There are some useful library websites that can assist in providing you with tools and guides for searching grey literature, some general, some more discipline-specific: • General resources, not strictly discipline-bound: – – – –

https://guides.lib.monash.edu/grey-literature/greysources; http://www.greynet.org/greysourceindex.html; https://guides.mclibrary.duke.edu/sysreview/greylit; http://libguides.newcastle.edu.au/social-sciences-grey-lit/databases.

• Health sciences-related: https://www.ucl.ac.uk/child-health/support-services/ library/resources-z/search-tools-grey-literature. • Psychology-related: https://libguides.adelaide.edu.au/c.php?g=165080&p= 3546983. In postgraduate research, grey literature should not be ignored as it can provide information critical to research contextualisation and framing, especially, but not exclusively for, professional doctorate research. Grey literature is particularly important in applied research in a number of social and behavioural science disciplines such as management and business, accounting and finance, economics, health and medicine, political science and public policy, psychology, agribusiness, ethics, innovation and education. It can highlight contextualised issues, potential areas for development, early research outcomes and directional signals and raise relevant considerations for positioning your own research as well as potential research participants. Benzies, Premji, Hayden, and Serrett (2006) and Mahood, van Eerd, and Irvin (2014) explore the benefits and challenges of using grey literature and both are worth a look. Haddaway, Collins, Coughlin, and Kirk (2015) discuss issues associated with using Google Scholar to search for grey literature. Hopewell, Clarke, and Mallett (2005) argue that including grey literature in any systematic literature review (including a meta-analysis) is important to help reduce some of the problems associated with publication bias (created, for example, by an overly selective focus on formally published materials that tend to emphasise statistically significant findings rather than non-significant findings, which will tend to remain buried in grey literature). One major drawback of using grey literature is that its quality and credibility can be much harder to judge, because it has typically not been peer-reviewed or otherwise vetted. Thus, you must make your own judgment as to the convincingness and usefulness of each instance. At a minimum, we would suggest that when you look at an instance of grey literature, you ensure that you get a clear understanding of the purposes of the document or report and the context in which it was produced—this can help gauge the credibility and transparency of the piece and

13.3

Searching

521

whether it is pushing a specific agenda or point-of-view or a balanced perspective. If the document is reporting on a research investigation or provide a summary of data and other information, the meta-criteria can be useful in helping you to gauge convincingness (more about this below). If the document is describing or providing advice or opinion on an issue, consider carefully the motivations of the author (even if that ‘author’ is an institution).

13.3.9 How Do I Judge the Quality of the Literature I Read? If you want to get an indication of the perceived quality of journal, you may wish to refer to a number of the publicly available journal ranking lists (for example, see http://www.harzing.com and https://www.scimagojr.com/journalrank.php? category=3321). The ranking will give you an indication of the quality of the journal as it is perceived by the academic community and its funding bodies. Citation reports provide a ‘systematic, objective means to evaluate the world’s leading research journals’ (Nimbekar et al. 2012, p. 508). Journal citation reports as provided by different citation databases show: • • • • •

the most frequently cited journals in a field; the popular journals in a field; the highest impact journals in a field; leading journals in a field; and most published articles in a field.

Journal citation report data come from over 7,500 journals representing more than 3,300 publishers in 200 disciplines. For a broad look at citation indexes, go to http://mjl.clarivate.com/index.html. The Social Sciences Citation Index (SSCI) covers approximately 1,700 journals in a variety of disciplines in the Social Sciences. Further important sources of citation data can be found in ISI Web of Knowledge and Scopus. As they are updated on a weekly basis, it will give you a good indication of what the current publications are. You merely need to search by key word or author to get an informative picture of the research conducted in a domain. You will, however, need to specify the dates of the search, the document type and the language. A citation will only indicate that the author has been cited, it will not indicate whether the paper has been positively or negatively received by scholars in the field. In relation to specific articles, an indication of the reputation of the paper can be obtained by looking at the paper-specific citation report, which is the number of times that paper has been cited in the literature. Once again, the ISI Web of Knowledge Journal Citation Reports can assist with this (https://clarivate. com/products/web-of-science/). Citation rates are related to other data such as publication frequency and produce various metrics which are applied to the evaluation of scholarly quality. The Journal Impact Factor metric of ISI Web of Knowledge or the Trend Line metric in Scopus

522

13

How Should I Select, Read and Review the Literature?

are metrics which allow the comparison of journals. The impact factor for a journal paper identifies the frequency with which an average article from a journal is cited in a year. The impact factor trend graph will give the impact factor for the journal over three years or over the last five years. Loosely, the higher the impact factor of the journal or paper, the more credibility it has. One of the benefits of citation indexes is that you can start to build up a picture of the main players in the field. If you have found a paper that is particularly useful to your study, look at who they have cited in the paper and who has cited the paper after it was published. By then looking at those researchers, and who they subsequently cite, a network can be developed that will indicate those who are working in this field. Obviously, the more citations, the more prominent the researchers tend to be. On the other hand, a newly published article could provide a totally fresh insight into your research field. Knowing the main journals in your field of research is imperative. Experienced researchers acquire this information through networking and attending conferences. You can source relevant information from your supervisor(s) or from other staff in your school. For someone who is new to a field, journal metrics are a powerful tool in identifying the best performing journals in a field/discipline within a few hours. Citation counts metrics are also used by academic and government administrators to identify the research output performance of an individual institution’s staff and departments or of an entire research institution. Librarians use this information for collection management and development. When looking at specific papers, another common preliminary measure of quality is the question of whether the paper has been through a peer review process. With peer review, the paper has been evaluated by experts in the field (typically a blind review where the authors don’t know who the reviewers are), critiqued and revised prior to publication. The notes on submission to the journal will give you an indication of whether the paper has been through this process. Alternatively, there may be a notation at the end of the paper indicating when it was revised and accepted. Additional elements to look for when judging the importance of the literature are: • Authority—is the material published by a reputable publisher? Are the theses/ dissertations/ portfolios from a reputable university? • Seminal—are the works regarded as a significant contributor to the historical development of the topic? • Currency—did the research findings originate in the last five years? • Relevance—is the material of relevance to your specific topic or project? • Citations—is the work considered to have merit (i.e., is sufficiently convincing) by others and has it been cited. (Hart, 2001, p. 26) When searching for relevant literature, some additional suggestions are: • Read other postgraduate theses/dissertations/portfolios. For a multinational database of theses and dissertations, check out https://oatd.org/. To review a list of British and Irish theses, go to https://www.proquest.com/products-services/

13.3

Searching

523

pqdtglobal.html. For American and North American theses, go to http://www. ndltd.org, which features theses and dissertations in the Networked Digital Library of Theses and Dissertations. The ProQuest Dissertations and Theses database contains many doctoral theses to keep you busy, and most libraries will hold a copy of the thesis or dissertation of students who have graduated from their institution. So, if you know where a person attained their postgraduate research degree, you should be able to track down a copy. Older outcomes that are not on a database are usually recorded on old technology (e.g., microfilm), so check whether your library has the equipment to read them. • Talk to your supervisor. They may possibly have material which is not in the general domain. If your supervisor is a regular attendee at international conferences, we suggest you ask him/her for the conference proceedings for the last two years in order that you may review the conference material and pull out appropriate papers. • Try alerting functionalities in relevant databases or from the web pages of journals. These services are a way of getting updates emailed to you on the most recent literature which has been entered into a database. The downside is that you might get an immense amount of information but, if you are good at culling quickly, it could be useful. Essentially, what you do is create a profile which runs automatically when the database is uploaded, and you will receive notifications matching your profile criteria. Most people use a key words profile, so that when specific words appear in paper titles, you will be notified.

13.3.10

How Can I Ensure that I Don’t Miss Something During the Search Process?

Relax; it is unlikely that you will, in fact, cover all the material. However, it is important that you achieve a good coverage and have not omitted any significant papers. Undertaking the search component of your literature review is a bit like following a path in an elaborate garden. Try this back-referencing approach to allay your fears. As you are reading one paper or book, look at the bibliography and note the references you would like to pursue. Having looked up these papers and read them, repeat the process, marking the next set of papers in the bibliography that you would like to look at next. This process keeps going until the point where you realise you are noting fewer papers to look up as you have already accessed and read most of the papers being referred to and there is only the odd one that you haven’t heard of. When you move into a new variable or dimension of your study, repeat this process. The searching and reading process takes time and most supervisors will advise you to allocate many months to that early stage of your research journey. Note though that you will continue to read relevant and emergent literature throughout your journey in order to ensure you keep your knowledge current.

524

13

13.3.11

How Should I Select, Read and Review the Literature?

When Do I Call a Halt to Searching the Literature?

With the wealth of information available, knowing when to stop is often quite problematic. Reassure yourself that it is unlikely you will ever cover all the material, but it is important that you are able to demonstrate a good knowledge base. Also, comfort yourself with knowing you can (and should) continue to add to the literature review as you progress through your study, and as relevant material comes to your attention. Create alerts for seminal papers, key authors and key journals, as well as for successful searches in relevant databases. Know the major conferences in your field and, if possible, attend some of them to get the hottest unpublished research in your area. If you cannot attend, ask your supervisor or his colleagues for the program or find it yourself on the web and follow up with authors you are most interested in. It has been recognised that “many students drown in the literature and a good supervisor is alert to that danger” (Delamont, Atkinson, & Parry, 2000, p. 6). A reasonable literature search very much depends on the topic and the research question(s) you want to address. If you find there is a limited amount of literature in the field, you may be taking too narrow an approach to your topic. A dip into other related disciplines could prove to be quite valuable. Discuss with your supervisor(s) which other disciplines might provide input into your research area. A brainstorming session would be quite useful when considering where else you might look for material. Note that a pluralist investigation or research done within the Developmental Evaluation or Transdisciplinary research frame may create an imperative to read broadly across several disciplines as well as deeply within specific disciplines and this should influence your planning timeline.

13.4

Reading

13.4.1 How Do I Cope with the Sheer Volume of Reading? Academic reading has been likened to reading a foreign language. But it is not only the complexity but also the volume which is problematic. Handling the data deluge is a problem that most postgraduate students experience, likening it to drinking from a fire hydrant! Dealing with the volume of material, determining what is of primary and secondary importance and summarising the information all add complexity to the task. In addition, students are often anxious that they are going to discard material which is potentially of value. Don’t worry; if you have entered it into your bibliographic referencing system, you can always retrieve it. Reading for academic success has been described as ‘casing the joint’ (Craswell, 2005, p. 130). Craswell (2005) also suggested the ‘4 S’ system: (1) Search, look for structure, the title, abstract, headings, subheadings, hover over the paper; (2) Skim, gain the overall look of the paper, re-read the abstract, look for the key words;

13.4

Reading

525

(3) Select key material that you wish to delve further into, phrase your questions; and (4) Study the key material in depth. If at point 2, the skimming phase, you feel that the paper is not potentially relevant for you, either discard or record the paper. What is being alluded to with the ‘4 S’ system is that you do not have to read every paper in full detail. John Witherspoon (1723–1794) observed “Some books are to be tasted, others to be swallowed, and some few to be chewed and digested: that is, some books are to be read only in parts, others to be read but not curiously, and a few to be read wholly, and with diligence and attention”. Essentially, there are various types of reading recommended: skim and scan reading and more in-depth reading when you will be more actively and critically engaged.

13.4.2 What Involved with Skim and Scan Reading? Skimming allows you to become familiar with the organisation and general content of the material and, more importantly, provides an indication of what the paper might yield for you in relation to your research. With scanning, you are quickly searching for specific information within the text, but still without reading the entire text. This involves looking for key headings, phrases or words, i.e., methods, limitations, conclusions, instruments, future directions, research questions or hypotheses. When skimming or preview reading you are undeniably approaching the material from the perspective of “what is in this for me?” That is, what is in this that could be of relevance or use to my study? The trick is to identify where in your research this material does or does not fit. Is it related to or can it inform the rationale for your study, the underpinning theory, the methodology, analyses, interpretations and conclusions, and/or the application of research findings in practice? Skimming is just to gain a very quick overview. You will be rapidly eyeballing the paper and looking at key headings, although this will not always be useful given that most academic papers are structured in a similar way. It will, however, give you a feel for the paper. When skimming, focus on titles, headlines, beginnings of sentences, and the first sentence in each paragraph, the abstract, introduction and the conclusion. By reading these items, you should gain a basic understanding of the topic and the main ideas, what the paper is about and whether it will be useful to you. If the paper looks to be a dud, move on. If the paper looks as if it might have some value to you, go through it again this time scanning. Knowing a little bit about the paper, you will be alert to its potential usefulness to your own research, so you will now be looking out for key words, phrases and terms. Also keep an eye out for figures or diagrams that display conceptual frameworks as these can give you a quick snapshot of what the authors are looking focusing on. Work your way through the paper, looking for mention of relevant theory, methods, participants and for additional references that could be of use to you. If the paper looks to be of value, you can decide whether you are going

526

13

How Should I Select, Read and Review the Literature?

to read it in more detail now or later (remember that you may wish to vary your tasks every two hours in a work session). For skim and scan reading, we recommend a five-step process: • Skim the paper. When skimming, let your eyes move quickly over the sentences and get a feel for what is being said and whether the paper could be useful for your study. • Scan through the references. Are there any references you would like to follow up? Make a note of them for follow-up and source later (keep focused on your reading). • Scan to see whether there is anything of further use to you. Look for key words, terms etc. • Once you have an indication of what the paper is about, enter the reference into your database. For hard copy, make a notation on the top right-hand corner. For example, this notation or label may be relevant to a theory, in which case, you would put “literature–theory”. You could be even more specific as to what the theory is. Or possibly the paper refers to your methodology or an alternative methodology, so you could label it “methodology”. • You may want to have a file (either electronic or hard copy) which is your ‘to read’ file for papers that you want to read in more detail. These should be read at an allocated reading time or for those moments when you are waiting for, say, a dentist appointment. Now, move on to the next paper. Once again, Step 1, skim it; Step 2, pull out any references you wish to follow up; Step 3, scan a little further for areas of interest; Step 4, label it and, Step 5, file it. This five-step process means you will be able to process several papers but does not require you to read each one from cover to cover, word by word, which would be very time-consuming. You will also find you are more motivated to read the material in detail when it is timely or appropriate, such as when you are trying to get to grips with a theory or methodology. At the skim and scan stage, your main aim is to determine the level of importance of the paper and to ensure you have distilled the key areas to which the paper relates. You are also using the paper as a means of hitching onto the paper trail of other papers that may be in the area. As we have mentioned, you may notice that, after a period, you start to exhaust the supply of papers that need to be searched. You will realise you have covered most or all the literature in that domain when the references recur, and you realise you have already sourced and reviewed them.

13.4.3 What Is Active Reading? When you go back to the papers that you have previously skim read and found to be of potential value, you will now be reading them in more detail. If you want to increase your comprehension and retention, become what is called an ‘active

13.4

Reading

527

reader’. A slow reader tends to read passively, just letting the information jump out at them, whereas an active reader actively engages with material by asking questions or having a goal as to why they are reading. Questions such as: How extensive is the Literature Review? What are the critical theories? What methods did they use? What was the response rate? What were their key findings? What were the limitations? What were the strategic and practical implications of the findings? What were the suggestions for future research? Are there any obvious gaps in conceptualisation, logic or design that I could work with/build on? An active reader recognises that they don’t have to read everything in the paper but enough to gather the information they need to answer their questions. Essentially, you are not letting the author direct you; you are in charge and are taking control of your reading. If you feel that your concentration is slipping away, stop and rephrase the question. The use of questioning is a valuable tool, not only with determining whether the material that is being read is of primary or secondary importance, but also in keeping you focused. The technique is to approach the paper with specific questions in mind (see Appendix 1: Questions You Could Ask During Active Reading). In this way, you are searching the paper, not in a distracted manner hoping for meaningful points to reveal themselves, instead you are actively searching. When you have found the material that answers your question, ask another question. Initially, you may use the question list as a guide but, after a while, the questions will come quite naturally. The important point is always to approach the reading with a question in mind to heighten your engagement. When reading a paper, you will not be reading all the material within the paper at the same speed. Some parts will require slow reading, some you will speed through, others you may skip over entirely. If you still find you aren’t getting anything out of it, this is a clear signal that it is time to abandon that item of reading. This approach can be used not just for journal articles. For example, if you are reviewing a prior thesis, the question that you may approach the reading with is: What was their original contribution? At another point in the thesis, you may be looking at the question or which methodology they used, or what the limitations accompanied their methodological choices.

13.4.4 How Do I Engage in Critical Reading? Critical reading requires you to not just read the paper but to also make some form of evaluative judgement as to the convincingness, worth or contribution of the paper, its potential linkages to other papers and, possibly, what limitations accompanied the research reported in the paper. Critical reading is more commonly reserved for the second or third pass on the paper and is usually only for papers you feel may have a valuable connection to your research, i.e., they are the papers that will help you to build the platform for your own research. With critical reading you are not only processing but also significantly reflecting on the material and making evaluative judgements, such as, what is the quality of the paper, that is, how

528

13

How Should I Select, Read and Review the Literature?

convincing is it in terms of the strength of their argument and the care and thoroughness of their research approach? What has been omitted and why? The trick to reading critically is to read with confidence and don’t hesitate to form an opinion. For example, you may feel the conclusions are a bit too general or you couldn’t see the connection or relationship between the evidence provided and the claims made. What was it they didn’t do? How would you have done the study differently? What suggestions would you have made? What guiding assumptions have they made, did they make sense and was their research approach consistent with them? What were the contributions of the research and how useful are they? Have they appropriately interpreted their results? Have they supported their argument? What evidence do they present to support their position? How does the paper advance the theory (if it intended to do so)? Does the paper generate further questions? What is new and significant? What is the contribution? Have they answered the ‘So what?’ question? How would this work in practice? Petre and Rugg (2010) have suggested that, “If you find a paper impossible to understand, it is because the paper is far too brilliant for you to understand or because it is a pile of pretentious, obfuscatory garbage” (Petre & Rugg, 2010, p. 56). Be ready to say which of these two perspectives it is. More difficult, but something you should be developing, is to reflect not only on the paper in front of you, but also on how it might relate to other papers you have read. So, when reading, be alert for opportunities to do the following: • Make comparisons with other papers, i.e., comparing results and conclusions by different authors? • Look for consensus and contrast, considering carefully any results that appear to lead to different conclusions across papers. • Reassess results considering new information that might not have been available to the original authors. (Lindsay, 1995). With critical reading, you will likely be staring into space more than looking at the page as you reflect more deeply on the paper. If you feel your reflections are poignant, record them, as they may be useful for the writing up of your literature review. To summarise, when reading we recommend: • Be focused: Read with a purpose. Remember what are you trying to achieve before you start reading? Start with the over-riding question of how could this help my research? • Be selective: There is a lot of material so being selective for detailed reading is important. While you may go broad in the initial stages, being selective in what you read at the detailed level is important. • Be active: As you are reading, continue to pose questions to keep yourself engaged. What are the key questions they are asking? How did they categorise the literature? How do they define the terms? What are their guiding assumptions (these could be explicitly stated or left for you to infer)? What theoretical foundations are they using? What constructs/variables/phenomena are they

13.4

Reading

529

examining? Where relevant, how have they operationalised the variables they are using? What methods of data collection and analysis have they used? What are the main conclusions and implications? Appendix 1: Questions You Could Ask During Active Reading provides a list of the sorts of questions you could ask yourself as you read actively and critically. • Be critical: Go one step further, try and reflect on what it means. What might the likely impact be on my study? How could this theory be revised or extended? Are their methods consistent with their guiding assumptions? Are their conclusions justified, given their approaches to gathering and analysing the data? Was the paper convincing overall? Make an assessment about the quality of the material; don’t be afraid to exercise your judgment. Form your opinions and substantiate them, being clear as to why you formed those opinions and compare the paper with others you have read. Record your thoughts. Some further considerations in relation to reading material for your literature review are: • Be careful of distractions when you are reading, that is, when you have other things on your mind or tangential thoughts that are seemingly unrelated to what you are studying. To deal with these distractions, if you have other things on your mind, use your journal to record them, get them out of your head and onto paper. It may be that what seems like a random, distracting or tangential thought early on may later emerge to be quite important. If you have recorded it, you can revisit and re-evaluate it later. We know of a PhD student studying the role of stakeholder values in reference to native vegetation landscapes, who felt the tug of what appeared to be distracting and irrelevant thoughts about the complexity and messiness of formulating and implementing government policies with respect to conservation of native vegetation in the first few weeks of her PhD journey. She noted these thoughts but didn’t dwell on them. About a year later, after discussing the issues with her supervisors and conducting a small pilot study, it became apparent that these early thoughts pointed to the core of what she really needed to address in her research. From that point, her project took on entirely new dimensions. If the thoughts are completely tangential and have nothing to do with something you need to remember to be a bit more disciplined about keeping the thoughts at bay or stop and quickly record it, but don’t get side tracked. • You don’t have to read a paper in any order. Most papers are structured with title page, abstract, key words, introduction, literature review, methods, results, discussion, conclusion and references. This is a traditional format and it makes reviewing particularly easy, as you can jump around the paper. Having read the abstract, you might move straight to the results section to see what the findings were, or, if you are particularly interested in the methods they used, and possibly the response rates they may have received, you would go straight to the methods section.

530

13

How Should I Select, Read and Review the Literature?

• You also do not have to read word for word. Thankfully, most academic articles are amazingly repetitive. Try and push on as often material is repeated in some form at least three times—in the introduction, in the body of the paper, and again in the conclusion. • Using a highlighter (either hard copy or digital) to mark the paper can channel your engagement with the text and shorten your re-engagement time when you return to the paper at a later time to extract key elements. But use highlighting sparingly by only highlighting the main points. • When reading a paper, avoid back-tracking on something you have just read. It will slow you up considerably. • One often hears about speed reading. It is possible to increase your reading speed markedly, and there are courses you can take. In the absence of a course, push yourself faster than you would normally read, try to take in chunks of information, and have few points of eye contact on the page. These are called eye fixations and the greater the number of times you concentrate on points within the paper, the greater the time it takes to read. Chunking has also been suggested, this is taking in chunks of information rather than reading word for word. It will feel uncomfortable at first, but you will get used to it. Comprehension will suffer initially but picks up remarkably quickly. Speed has the added advantage of keeping you engaged and more focused. The faster you read, the less time you will engage in tangential thoughts. It is also giving you more information. If you wish to look at some reading exercises to establish your speed and comprehension, try: – http://www.readingsoft.com/index.html – http://www.turboread.com/read_checks.htm – http://www.freereadingtest.com/. Should you wish to accelerate your reading speed, you could look at Scheele (2000). • Do not let the “to read” pile become so big that it discourages you. Always have one or two papers with you so that when you have a spare moment you can do some reading. • Keep up with your filing, preferably in your bibliographic management program. For hard copy users, put the papers which you have annotated in the top right-hand corner with a key word into boxes labelled with the same key word. If the paper is relevant to two or more areas, copy the front page and place the one-page copy in a relevant box. Store the full copy in the box which contains the material you will likely be working with next. If the contents of the box become too large, you may wish to subdivide the box by refining the key words and creating a separate box, or merely marking a second box with the same key word and labelling it but with No. 2. A similar approach can be taken for electronic copies of papers.

13.5

13.5

Taking Notes

531

Taking Notes

13.5.1 What Should I Be Doing When Making Notes on the Literature? While actively and critically reading papers, you should also be writing, that is, not only recording the reference but also your reflections and observations. In this way, you are going further than being a passive recipient of information and are attempting to digest the information. It is, therefore, essential to capture what you are thinking now, not later, as it is unlikely you will get to this insightful point again. A further benefit of these notes is that they can greatly assist in the writing stage of your literature review as you already have some preliminary observations and opinions recorded. Having indicated that note-taking is important, a common mistake for novice researchers is that they take too many notes about papers that end up, ultimately, being irrelevant or only of tangential interest to their study. Taking too many notes can be time-consuming, and you may not use all the material. As a rule, avoid extensive note-taking in the early stages of your review of the literature as you will not be sure what is immediately relevant or not relevant. If you’ve copied it or electronically scanned or stored the paper, the material will always be there for you to go back to and look at in more detail. Note-taking will be more useful to you once you have started to get an understanding of the literature and can better discern what is important to you. Note-taking is likely to be more useful following active and critical reading rather than at the skimming and scanning stage. Note-taking is the actual recording of the reference as well as distilling the key elements that relate to your research by capturing of the essence of the findings and your critical observations of the material, usually into your reference database or research journal.

13.5.2 Recording Your References There are several bibliographic management software programs which you are encouraged to choose from, such as: • • • • • • •

EndNote Desktop (www.endnote.com) EndNoteWeb (www.endnote.com) Pro-cite (https://www.dataone.org/software-tools/procite) BibTeX (http://www.bibtex.org/) Bookends (https://www.sonnysoftware.com/index.html) RefWorks (www.refworks.com) BiblioExpress (https://biblioexpress.en.softonic.com/)

532

• • • • • • • • • •

13

How Should I Select, Read and Review the Literature?

Bibliographix (home.mybibliographix.com) Biblioscape (https://www.biblioscape.com/) Easy Bibliography (www.easybib.com) Library Master (http://www.balboa-software.com/) Mendeley (www.mendeley.com/) MyBib (https://www.mybib.com/) Notabene Scholar’s Workstation (http://www.notabene.com/) NoodleBib (www.noodletools.com) Papyrus (http://www.researchsoftwaredesign.com/) Zotero (https://www.zotero.org/ (Hernandez, El-Masri, & Hernandez, 2008).

EndNote is probably the bibliographic management system most commonly used by postgraduate students. EndNote allows you to create, store and manage your own reference database as well as create bibliographies and change or create reference styles. EndNote interfaces well with Word and enables instant citation while you are writing so that you don’t have to go back to find and tidy up citations and references. EndNote allows you to import references from some databases directly into your EndNote library, which is a considerable saving of time. Your records could include PDF files of the articles and your own labels and notes. EndNote can be run on Windows and Apple OS platforms. Check to see that your university has a site licence for EndNote and enrol in any training that is offered. The term ‘library’ in EndNote is used to refer to the database of references which is being compiled by you and kept on your computer. You will need to save your library with a file name. EndNote will automatically append the extension “.ENL” to the name you give it. You can change the ‘style’ you want your references to be formatted in at any time in both EndNote and Microsoft Word. This makes preparation for publication easier, as different publishers or schools have different style preferences. Check with your department and any regulations regarding the formatting of postgraduate research outcomes to see which bibliographic format is required, as it is best to start with the correct style. There are a gob-smacking 2,800 styles you can use, however, there are four or five basic styles favoured by most universities. Your university may have already set a default to a specific EndNote or created its own style. When entering a reference, make sure it is done accurately. There is nothing worse than having to spend extra time, when you are writing up, chasing and tidying incorrect references. One of the beauties of EndNote is that you can generate a list of references on a subject, or with characteristics, by using EndNote’s search function or importing citations from databases. For this reason, when you are creating a reference, don’t be too skimpy in the key words field, as it may become quite useful later, particularly when you are pulling up references for writing papers. You can make your search more accurate by adding extra terms. The interface between your word processing document and EndNote is brilliant and you can insert citations quickly into your document from EndNote. A further feature is that you can input references into EndNote from online databases. Some databases export directly to Endnote while others you must save or download your

13.5

Taking Notes

533

marked lists in a file and/or in the correct ‘tagged’ format so EndNote can read them and then import them into your library. For further discussions on using EndNote, see: • http://research.library.gsu.edu/endnote • https://www.thoughtco.com/endnote-research-paper-1690650 • https://www.researchgate.net/post/What_are_the_main_benefits_of_using_ Endnote.

13.5.3 What Is the Best Way to Record My Summarising and Evaluating? The not-mutually-exclusive options for recording one’s summary and critical reflection and evaluation notes are: • in your bibliographic record database; • directly onto a hard copy of paper or, if you have a PDF version, using Adobe Acrobat Reader’s commenting/highlighting capabilities to make and save notes on the paper electronically; • into your research journal; and/or • on a separate template (e.g., a database or a mind map; a computer support system like Microsoft Access (database), Excel (spreadsheet) or even Inspiration [mind map] could help here). The choice of how you take notes is largely personal, although consider the time it takes and the transferability, that is, being able to use the notes later in your writing. When note taking directly on the paper, consider using codes. You may have already developed this skill but, essentially, you provide a one-word code in the margin which signifies what that material and your notes are about, for example: • • • • • • • •

Def = definitions Assm = guiding assumptions or other assumptions evident RQ’s = key research questions Meth = the methodology used Rel Lt = relevant literature that may be worth going back and looking at RR = response rate Key Fd = the key findings or conclusions of the paper Imp = the implications and recommendations, i.e., the more practical suggestions or outcomes of the document • Lim = limitations the authors have noted, or you have noticed • Theory = the theories that have been used as a basis of that literature.

Feel free to make up your own codes. Having a code system like this means you can quickly scan through the document and find what you are looking for by merely

534

13

How Should I Select, Read and Review the Literature?

reviewing the margin codes. All the above could be included in each reference record under Notes in EndNote. When note-taking, you may be answering the questions that you posed as part of your active reading, although, be careful about restating what is already in the paper as this type of note-taking could slow you down. The more valuable notes are your personal observations, reflections and critical evaluations, e.g., answering the question, “How is this paper like, or different from, the paper you just read before?” With writing directly on the paper, it may seem a bit obvious but make sure you can read your notes. Don’t make the notes too cryptic. As you may wish to use the notes later, they are probably best typed into your record system or just in Word. For the latter, this can either be in a separate document or in a template (for examples, see Appendix 2: Concise Critical Notes: Articles and Papers Template and Concise Critical Notes: Analysing Arguments in a Theoretical Paper Template). Although potentially time-consuming, you could record your notes in your research journal using bullet points or full prose, or by creating a detailed mind map (a mind map can also be a useful device for relating/comparing several papers in the same general area or for contrasting papers). Choose whichever method works best for you. When making notes for a literature review, there will be the occasional direct quote or close paraphrase that you might want to record and, if you do, make sure you also record the page numbers in the reference from which you have taken the quote. However, for the most part, the notes should, in fact, be key words or phrases. From experience, a good piece of advice is, when taking notes, come up with a method to ensure that you do not confuse your thoughts and observations with those that require appropriate attribution. You should ensure that those notes containing explicit and word-for-word comments of others or very close paraphrases are embedded in quotation marks, while those without quotation marks are your thoughts and observations. You may like to use a different colour, or even a star, to indicate your thoughts and words. Alternatively, it has been suggested that you might like to split your note page in two (as we showed in Chap. 3 in Fig. 3.1 a). On the left, put the information directly relating to what you are reading, and on the right your thoughts, reactions and comments. Thus, you can differentiate between your views and the work of others, thus reducing the risk of plagiarism. One useful way of evaluating the quality of a specific research outcome, such as a journal article, thesis, government report or other research outcome is to use the meta-criteria, introduced in Chap. 9. Using the meta-criteria can help inform your overall judgment of convincingness with respect to the research outcome you are evaluating. This is a deeper and more comprehensive evaluation and you would likely reserve it for the most critical works you want to use to inform your own research contextualisation, frame and configuration. Appendix 3: Meta-criteria Research Outcome Evaluation Framework provides an integrative tool to help you implement the meta-criteria to evaluate a research outcome. All 12 meta-criteria are represented, organised into their three higher-order domains, Contextualisation, Realisation and Explication. Each meta-criterion has a trigger question associated with it to prompt your thinking. The three meta-criteria domains also have a central

13.5

Taking Notes

535

trigger question as does the final convincingness judgment. The framework can be used qualitatively, through textual comments, or quantitatively through ratings that can then be averaged together in ways indicated by the various brackets, or both, as you would prefer. [Ray has used a version of this tool to evaluate PhD theses and portfolios that he has examined.]

13.6

Writing

13.6.1 What Is Involved with Writing up a Literature Review? Having been busy reading and taking notes for many months now you have the task of committing it all to paper and writing up your literature review. Once again, there is a series of essential and sequential stages that can help you: Stage 1. Develop a structure This is deciding, in outline, the structure or format that the groupings of the literature will take. It is quite a creative process as three academics dealing with the same literature could come up with three different structures. There is no right or wrong way; you just need to select the structure that you feel comfortable with. Imagine that you tossed all the papers into the air and they fell into their natural groupings, what might those groupings be? Wellington, Bathmaker, Hunt, McCulloch, and Sikes (2005, p. 82) suggested a way of approaching the literature review with the use of: • zooming in on a topic, which essentially means starting with a wide-angle view and eventually focussing in on a key area; • picturing the literature as being in three or four areas which intersect with some areas of overlap, and your central focus is on the intersection; • funnelling, which is like zooming, but the topic area is slowly narrowed down as you prune away branches that you cannot or don’t wish to follow; and • patchworking, where there is a wide range of areas of reading, without any necessary overlaps, but that collectively makes up the representation of your field of study. Structuring your review is an important aspect before you start writing and there are a variety of different ways that you can present the material: • The critical topic domains relevant to your study—the key words that you have been using for your database searches will give you a good start on what the primary domain might be, i.e., non-profit governance, non-profit sport. • The principal theories—highlighting the main theoretical groupings and the related research.

536

13

How Should I Select, Read and Review the Literature?

• The key findings—somewhat similar to the principal theories but this approach is narrower, i.e., all those in this camp are …, and those in that camp are … Regrettably, this does not enable you to compare and contrast and is probably going to be too narrow for a doctoral study. • The historical development—this is by chronological order, i.e., presenting the material in the order it was developed, starting with earlier research and finishing with the most recent. • The methodologies—grouping of the literature around relevant or differing methodologies used. • By the type of research—for example, grouping by literature that provides theory, literature that provides evidence, and literature that provides application. It is important to realise that your literature review is, essentially, a large-scale argument that sets the stage for your research. You want to write that argument in such a way as to lead the reader directly to your research questions. The literature review is where a good deal of contextualisation occurs. You want readers to see where your emerging research fits with the research that has already been done. The most common expectation for a literature review chapter is that it will conclude with clear statements of your research questions or hypotheses. At that point, you do not want the reader to be surprised by anything you incorporate in your research questions. The way to ensure this is to take the time to develop and shape your review so that it covers the landscape in the necessary depth, and then progressively and logically ‘walk’ the reader through that landscape to arrive at your research questions. Stage 2. Construct a pictorial depiction of the categorisation you are using This could be using a simple diagram, a flow chart, or a mind map in order to demonstrate a visual categorisation of the literature. Mind maps can be particularly useful for sorting out themes in the literature, first as a pictorial and visual depiction of the groupings of the literature, which can then help identify when the literature fragments into different groupings. If you want more detail on mind maps, try Buzan (2018). A map or diagram should be relatively quick to construct and change. The visual element also helps with depicting a lot of information on one page which may be beneficial for recall and for discussion with your supervisor(s). Stage 3. Populate your diagram/map with actual references Make your pictorial diagram big enough so that you can write in some, if not all, of the key references relative to the group. You just want the names and dates of the authors, e.g., “Cooksey and McDonald (2019)”. Alternatively, you may prefer a standard table rather than a diagram but at least you will know which references sit in which grouping. It is OK to have overlaps and some repetition but try to keep them to a minimum. As you now know which material sits in specific groupings within your structure, you will be in a better position to start writing up the material on these groupings.

13.6

Writing

537

Stage 4. Start writing Don’t write the introduction section of the literature review first, leave it to last and start by working through the groupings of the literature. For each grouping, look at your references and their related material; consider how they can best be presented. What are the sub themes within that group of references? What research is similar and what research is different? Write these up drawing out the key issues and presenting the arguments. It has been suggested your references could begin with old seminal references, continue with more recent key references and then assorted examples of less important references, and end with the most recent foundational references (Petre & Rugg, 2010, pp. 56–57). What you are looking for is representation and integration. This requires that each source cited should be collated into categories with other related literature. Summarise and capture the essence of the findings of the materials you have selected (Thody, 2006, p. 93). Avoid taking large quotes from papers, as brevity is the key, and start to craft your own perceptions of the material. Try to give balance to the attention that you give to each grouping. The liberal use of sub-headings throughout your literature review will enable you to walk the reader through the main components of your structure. Clearly flag when you are transitioning from one literature grouping to another. The more you write, the more you will enhance your understanding of the literature. Don’t wait until you know what to say; use writing to aid your thinking. Stage 5. Critique the material The subtle distinction is often made between a literature survey, that is, seeking out and referencing sources, and the literature review which is the need to report and synthesise the literature you have found. One can go further and make a distinction between a summary and a critical evaluation, indicating that a literature review is “an orderly discussion supported by evidence, not a summary with occasional comment” (Finn, 2005, p. 91). The aim of a literature review is to show that the writer has studied existing work in the field with insight. “It is not the reader’s responsibility to make sense of a pile of references indiscriminately grabbed from the internet and then tacked together with semi-coherent prose” (Petre & Rugg 2010, p. 56), and it is not enough to merely show what others have discovered. The work needs to be critically reviewed. You are indicating not just your reading of the literature but also your comprehension and evaluation. This is “demonstrated by summarising, differentiating, interpreting, contrasting and understanding the significance of what is being reported” (Levy & Ellis, 2006, p. 193). A literature review should not just attempt to illustrate main areas of understanding but should also point out current areas and gaps that are less well understood or addressed. Where there are disparities between the studies, articulate a logical explanation. Go back to the notes you took when you first critically read the paper as well as re-reading the paper. To achieve critical comparative evaluation, you could:

538

13

How Should I Select, Read and Review the Literature?

• • • • • • • • •

compare different authors’ views on an issue; group authors together who draw similar conclusions; critique aspects of methodology; note areas in which authors disagree; highlight exemplary studies, perhaps contrasting with poorer quality studies; highlight gaps in the literature where aspects have not been investigated; review and raise questions; identify areas to be explored; show how your study will address these gaps and how it relates to previous studies; and • show how your study relates to the literature in general. Reviews should, therefore, not only synthesise the material but also critique, debate and critically evaluate the findings. Critical evaluation, essentially, compares and contrasts material, rather than just describes it. To develop critical evaluation skills, it is recommending that you inspect relevant journals in the forum, discussion or comment sections, as they are devoted to posing questions, discussing and sometimes rebutting new concepts and theories (Finn, 2005). Don’t be fearful or intimidated by the literature. All too often students are too deferential in their approach to the literature and are reluctant to make a critical comment given their immaturity in the field and with research processes. Criticism merely entails having a sceptical attitude, asking what lies beneath the appearances, giving credence to other alternative arguments or explanations, being tactful not destructive, as well as being positive and appreciative and/or negative and disapproving. When you critique research in your literature, you are showcasing your judgement. Anyone can summarise what someone else has done; it is an altogether higher level of skill to critically assess what they have done. The latter is what is expected of postgraduate as well as academic researchers. Good criticism is impersonal and reflects balance in focus, clarity in argument, soundness in reasoning and defence of judgement. There are things you can do to ensure that you get the tone of your critical evaluations right. Good criticism should especially reflect: • • • • • • • •

heavy scepticism, but not cynicism; confidence, but not cockiness or arrogance; judgment which is critical, but not dismissive; opinions, without being opinionated; having a voice, without “sounding off”; being respectful, without being too humble; careful evaluation of published work, not serial shooting at random targets; being fair and assessing fairly the strengths and weaknesses of other people’s ideas and writing, that is, without prejudice; • having your own standpoint and values with respect to an argument, research project or publications, without getting up on a soap box;

13.6

Writing

539

• making judgments based on considerable thought and all the available evidence, as opposed to assertions without reasons, and • putting forward recommendations and conclusions, whilst recognising limitations without being too apologetic (Wellington et al., 2005, p. 84). As you can see, the approach to criticality involves as much an attitudinal position as an intellectual discussion. Stage 6. Pull it together Having critiqued the main groups or domains of the literature, go back and write the introduction to the literature review which will briefly outline those domains and what their contributions to the field are. Turning to the conclusion, pull the material together so that it focuses on your study, highlighting what the deficits are with the current research, where the gaps are or where unanswered questions remain that your study will address. Not all research will receive equal attention. Some references may be listed in parentheses as examples, but others that are more relevant will receive more critical attention. Also keep in mind who your audience(s) are. It is first your supervisor(s), then your examiners, then other key stakeholders. Write as if you are speaking to those people, which might make the process appear less abstract. To improve the readability, a literature review (as well as your entire research outcome) it is less about articulating results and prior findings than telling a story of the journey. “At the height of your literature review is a good plot. The story should start with a problem of some sort… follow the steps taken by previous work to solve the problem. The literature review end at the point where you, the hero or heroine, enter the scene armed with your enhanced approach” (Petre & Rugg, 2010, p. 57). As the literature review is one of the first significant attempts at writing for your thesis/dissertation/ portfolio, you may wish to obtain some additional writing support. Suggested online guides for grammar and writing are: Charles Darling’s Guide to Grammar (http://grammar.ccc.commnet.edu/grammar/index2.htm) and the Online Writing Lab (http://owl.english.purdue.edu/). You may also find Kamler and Thomson (2014), especially Chap. 3, very useful to review in regard to dealing with literature reviews. They present a range of particularly vivid metaphors, grounded in actual PhD student experiences, for the process of grappling with and writing a literature review. One of the most delightful metaphors describes the process as one of ‘persuading an octopus into a glass’ (Kamler & Thomson, 2014, p. 34). Stage 7. Referencing There are a few recurring sins when citing published material. The first is not keeping complete and accurate bibliographic information, the second is attempting to fill in missing information by guessing, and the third is overloading the literature review because of the common feeling that many students are reluctant to leave out any material they have sourced. Hence, the first suggestion is to try not to overload

540

13

How Should I Select, Read and Review the Literature?

your literature review; include only relevant and meaningful material, and ensure it is correctly referenced. Although the terms ‘citation’ and ‘referencing’ are used interchangeably, for our purposes we use the term citation to refer to citation reports discussed earlier and that are used in measures of the impact of papers and journals. Referencing usually refers to in-text referencing and bibliographic referencing, for which the aims are: • to demonstrate that you know the rules of academia; • to assure examiners that you have obtained and read the relevant works; and • to assist others who are also on the paper trail and seeking references to their area(s) of study, that is, facilitating someone else back-referencing through your work. In-text citations have been described as ‘wedding confetti’—“scattered liberally and indiscriminately, in the hope that they will bring joy” (Thody, 2006, p. 192). They are, however, an inevitable feature of a literature review and your first concern will be to maintain consistency in formatting. Life would be a lot easier if there was one formatting system for references but, unfortunately, there are numerous formatting systems. Have a look at the variations available in EndNote to get an indication of the principal ones (e.g., APA, Chicago, Harvard). Having the right format, that is consistent with the presentation requirements of your department or university, is important, and will be of even more relevance when you publish, as each journal has its own preference for which formatting system it uses. The usual practice is that for multiple authors (three or fewer), the first time you reference them in the text, you cite all the authors, but, thereafter (where there are three or more) they would be presented as “Cooksey et al. (2019)” (et al. meaning ‘and others’). If you have several acronyms or specific terminology, you may wish to consider the use of a glossary in your front materials or an appendix (see further discussion in Chap. 22). With the extensive use made of the internet, it would not be uncommon now to see reference made to hyperlinks within both the body of the text and in the bibliography at the end of a research outcome. Caution should be used regarding the number of hyperlinks you are referencing. Given the volume of information that is available on the internet, it is very tempting to sprinkle hyperlinks liberally throughout your research outcome but try to be judicious in your choices. Regarding references, there are some mandatory and desired outcomes. Mandatory: • Are your references laid out correctly with all required information, down to the last full-stop/period and comma? • Have you cited all the seminal and core references? • Have you cited a good spread of sources, ranging from the seminal texts to something within the last year?

13.6

Writing

541

• In pluralist or transdisciplinary research, have you cover an adequate spread of literature across disciplines? • Are your references mostly from respectable journals rather than textbooks or the internet? Desirable extras: • Have you cited work which is little known except to people doing advanced work in the area? • Have you cited anything which is “in press”? This implies you are sufficiently part of the research community to be given pre-prints by researchers? • Have you cited a discrete number of your own papers in respectable journals, preferably co-authored with an authority in the field? (Petre & Rugg, 2010, p. 64).

13.7

Revision

13.7.1 Why Do I Need to Revise? The final step in our recommended six-step process for preparing a literature review is the step involving on-going revision. This is necessary as new information becomes available during your study and to more accurately reflect the nature and intention of your study if it has changed from its original conceptualisation. As you progress through your research, you will be continually collecting relevant material as it comes to light and as you actively search out papers as part of the paper trail process. Also, quite possibly, your literature review will be revised in the light of your findings, as you will want to emphasise the literature that is more relevant to the key elements of your research. In the end, you will be writing not one but three literature reviews. The first is for your proposal, the second is the first draft at the beginning of your research, and the third is the revised version that you write to complete your examinable research outcome. The last and final literature review should be appropriately revised to not only incorporate more recent literature that has been presented or published since you started your project, but also to better align the literature with the focus of your study. It is not uncommon, in the analysis and write-up stage, to emphasise significant findings and, therefore, it will be necessary to go back and realign your literature with this emphasis. Or, having redefined your research questions, you may find some of the literature is no longer relevant, and you will need to go back and cull it. To minimise the additional time taken on the final version of the literature review that will be contained in your research outcome, you are advised to be continually

542

13

How Should I Select, Read and Review the Literature?

updating your literature review as new material becomes available to you. The revision process will therefore be occurring at various intervals throughout your writing process and, more acutely, towards the end. You only want material in your literature review that is of critical importance to your research and, ultimately, you want your review to be of examinable standard.

13.7.2 How Is a Literature Review Assessed? By now you will be coming to grips with what constitutes a good or bad literature review. As you will realise, there are multiple dimensions which determine the properties of a good literature review. Interestingly, these have been developed into a Literature Review Scoring Rubric. According to the rubric, a literature review can be assessed on five dimensions: • Coverage—provides criteria for deciding on the inclusion and exclusion of material from the review. • Synthesis—which critically examines what has and has not been done in the field, placing the topic within the broader scholarly literature in the context of the field, and in the historical context, demonstrates a good understanding of the key vocabulary and contexts/theories/constructs/variables relevant to the topic and provides a new perspective on the literature. • Methodology—identifies and critiques the appropriateness of the main methodologies and research techniques and theories used. • Significance—critiques the practical significance of the research. • Rhetoric—evaluates how well the review is conceptualised and whether it has a coherent structure and is clearly written (Boote & Beile, 2005, p. 8). For the full version, see Appendix 4: Literature Review Scoring Rubric.

13.8

Conclusion

As a general guideline, a literature search for a proposal should take at least two weeks full-time, and a literature search for the literature review chapter in your research outcome should take at least six months, plus additional top-ups throughout your research journey. For example, in a 100,000-word PhD thesis, a literature review will probably take up at least 10,000 words. By now you will realise that there are significant benefits to undertaking a literature search that will enhance your research. To recap, these are:

13.8

Conclusion

543

• Helping you explore, shape and confirm your topic, identify past research questions that prompt new research questions; and ensuring that you are not undertaking research that has previously been conducted. • Gaining an indication of the range of literature that surrounds your research problem as well as an indication of the outcomes of prior studies. • Identifying what theoretical and conceptual frameworks are being used by other researchers working in a similar area. • Gaining an insight into what key patterns of guiding assumptions, issues, perspectives, concepts, dimensions, variables, constructs and contexts have been examined in the past and how and why other researchers saw them as important to explore. • Learning about the range of methodological strategies researchers have used to navigate the ‘data triangle’, e.g., sampling strategies, measurement and other data shaping strategies, interview, observational or artefact-based strategies, experience structuring strategies, analytical approaches and research configurations and seeing the outcomes that emerged from those choices, such as relational patterns, common response rates, stories about data quality and consistency, emergent themes and insights, grounded theory, and so on. Clearly, you need to ensure that your research will be original, that the ground has not been covered before and that you will be able to demonstrate to examiners that you can synthesise and critique related research and relevant material. From the analysis of the literature, you will become acquainted with the existing theoretical propositions and approaches and/or conceptual frameworks. Not only will this reveal what questions have been posed before, and the concepts utilised but, more importantly, this will help you identify what hasn’t been asked and where gaps or unanswered questions are that you could address. The strategy we advise is, essentially, to start a paper trail. For example, when you read a relevant journal paper, turn to the references and with highlight the papers that you believe are relevant to your study. Obtain those papers and undertake a preliminary overview of them. Once again, turn to the reference section, highlight the papers you do not have then obtain them. This back-referencing process will happen several times until you notice that you are, in fact, coming to the end of the paper trail in that you have acquired most of the papers. Speak to your supervisor(s) who will be aware of many of the seminal or early papers that may be important (and are, possibly, contained in the supervisor’s own research). Keep an eye on current key journals, do regular literature searches, speak to colleagues, obtain conference proceedings (even if you’re not able to attend, ask someone whether you can have a look at their conference proceedings which are invariably contained online).

544

13

How Should I Select, Read and Review the Literature?

To recap, when you are looking at a paper: • Decide whether it is relevant or not. If it is not, skip it; don’t bother to read it all. • If it is relevant, skim it, do an overview, look at the Abstract and the Conclusion. Flick through the paper looking at the paragraph headings, gauge what’s involved—is it a dense paper or is it a relatively easy read; does it employ quantitative, qualitative or pluralist methodologies? • Scan for specific key words, theory, terminology, methods and useful references. • Decide whether you want to read it further now or later. Record and file using key words. • When reading in detail, possibly later, pose questions (active reading) that you would like to focus on. By asking questions, you become more engaged with the material and it provides you with a focus and a purpose for reading. • When you hit an area that is of interest, dive into it and read it in more detail, underline or make comments on what it is that you observed in the paper, recalling your questions, preferably saying it in your own words. The good news is that articles in most journals have a common structure, so they are easy to read rapidly. • Critically read and seriously reflect on what you see as the strengths and weaknesses of the paper, and how it might relate to other papers. Naturally, this may not be the only time you look at the paper. The opposite of the concern for leaving things out, is thinking you have read everything. As one student commented, “The literature search remains a never-ending headache”. “The extendibility of the academic enterprise means that it is very difficult to know when to stop” (Delamont et al., 2000, p. 53). Don’t worry, there is always the likelihood that you may miss something but, if you have done the groundwork, it is very unlikely to matter much given the coverage you will be able to demonstrate. It may make you feel very virtuous to have read every article published on your topic, as well as related ones, but it won’t help you finish your research project. Making appropriate choices about when enough is enough is the hallmark of a maturing researcher. It is one thing to demonstrate that you have access to and have described a body of work relevant to your topic, however, that is merely descriptive writing. It is important to tell a story and to demonstrate that you can dismantle theories, concepts and findings, to critique them and, possibly, reassemble them, adding a new perspective or insight by forming your own opinions. Some students in the early stage of their studies find it difficult to be critical of, or critique, the literature they are reviewing. They feel inexperienced and somewhat daunted by the task or like they have no place critiquing the works of others. They also acknowledge that most of the material they are reading has already been through an extensive blind review process before being published, and that the work is, therefore, of sufficient value to warrant publication in an appropriate journal. However, putting this aside, it is part of the intellectual process that students should not accept material without question

13.8

Conclusion

545

and that they are able to see not only what was being studied but what should have been studied or, potentially, what weaknesses might be in the material they are reviewing. Remember, there is no perfect study—even the most rigorously reviewed article will have some flaws and the author(s) will have made some trade-offs to achieve their goals within the constraints they have faced (just as you will have to make such trade-offs). By way of a heads-up, most convincing papers will have in their conclusion section an identification of what the writer perceives as the limitations of their study. This can most certainly be of assistance in the initial stages of your critiquing process. To conclude, a good literature search will ensure that: • the key researchers in the area have been identified; • the most relevant theoretical and conceptual domains, both in the discipline and in related disciplines, have been identified; • where, appropriate, potential variables or concepts relevant to the research topic have been identified, prior experiences and key findings associated with these variables or concepts have been reviewed and the salient variables or concepts relevant to your research have been isolated; • where appropriate, alternative patterns of guiding assumptions and perspectives and the research they have informed have been explored (especially important if you are adopting a pluralist approach, but generally important for demonstrating that you have read diversely and have been willing to consider alternative points of view and attacks on the problem); • you have appropriately critiqued the strengths and weaknesses of the existing body of material and identified where there are gaps in the literature; and • because of these deliberations, your research topic and research questions are robust enough to be deemed original and are informative enough for you to pass.

13.9

Key Recommendations

• You will not do one literature review but a series: the literature review for the preparation of your proposal, the first draft of your literature review chapter and significant revisions following continuous reading as papers become available to you. • The more you read the firmer and more relevant, or irrelevant, the material will appear. So, read, read, read! Usually, about 50–100 papers form the core of the relevant literature. • The more you read, the more extensive and comprehensive your key word list for searches will become. • Become acquainted with the search rules for the databases that you are using.

546

13

How Should I Select, Read and Review the Literature?

• Electronic bibliographic databases will contain a number of journals, but not all and, possibly, not the seminal works. So, additional digging in the library will be required on your part. If you are not able to find materials in your own library, then don’t forget you can use inter-library loan services. • With reading there is licking (just skim reading and scanning) versus chewing (more active and critical in-depth reading). Never feel obliged to read all the way through a paper. • Avoid the temptation to read every paper in-depth but learn to scan to determine each paper’s worth, value or contribution to your study at that time. • Skim read to gain an overview of the text, i.e., to find out what the paper is about and whether it is relevant to your study. Hover and dip into the text. Record what you have read in a bibliographic file. • When actively and critically reading papers ask questions, imagine yourself in a dialogue with the author. Think about what’s good about the paper, what’s wrong with it, what its contribution is and, possibly, how the paper could have been improved. • When writing-up, avoid taking large chunks of material from the literature you are reading. Take only what is immediately relevant. You will be judged on the quality and character of your critiquing, your ability to identify the gap(s) your research will be addressing and your arguments that build between the literature and your research questions/hypotheses. Draw from the literature any relevance to your own area of research. • Find a structure and put it in diagrammatic or table form before you commence writing. The literature review should be a story with references, not references with a commentary. • Do not feel limited to one chapter. For example, it is not uncommon to see a separation into one chapter covering the main theoretical domains and one covering the empirical research relating to the problem. If you are doing a professional doctorate portfolio, your literature review will likely be broken into several pieces, each aligned with different aspects of your portfolio. If you are doing a PhD by publication, then you will have a literature review section in each component publication as well as literature reviewed in the bridging and linking sections between the various publications that work to produce a coherent research outcome document. • Check with your supervisor for confirmation that you have covered the seminal papers in the field. • Do not attempt to write the ‘perfect’ literature review chapter straight off. The first one is a rough draft and will be added to and reshaped throughout the duration of your study as the revision process continues. In fact, the literature review chapter(s) will typically be the last section you finalise in your thesis/ dissertation/portfolio.

13.9

Key Recommendations

547

Appendix 1: Questions You Could Ask During Active Reading • Who are the authors of this piece? • What do you know about them? • What is the perspective of the writer(s)—goes to the issue of the author(s) guiding assumptions? • What is the journal quality? • How old is the material, and on what date was the research done (it may have been many years before it was published)? • What were the authors trying to discover—how did they position their study? • Is it an original study, or a report of other people’s work? • Is it empirical (has data in it), theoretical, or polemical (argumentative)? • How was the literature review structured? • What theory is cited? • What was measured or gathered? • What methods were used (computer modelling, experiments, field measurements, interviews, participant observation, etc.)? • What information is available on their sample and the sampling process (is there a breakdown of the sample by age, race, gender etc.; was the sampling process random, purposive or some other scheme)? • What were the response rates? • How were the data collected? • What analyses were used? • What were the results? • Are the arguments logical? • What support or evidence has been provided for the key message? • What do the authors conclude, and to what do they attribute their findings? • Can you accept the findings as convincing? • Why is this piece of research important? • Could this be applied in practice? • Is the material correctly and fully referenced? • How can I apply these findings to my own work? (adapted from Wellington et al., 2005, p. 75 and Delamont et al., 2000, p. 55) Note that Quinton and Smallbone (2006, pp. 87–88) offer two similar frameworks for ‘deconstructing journal articles using primary data’ and for ‘deconstructing journal articles using theory’.

548

13

How Should I Select, Read and Review the Literature?

Appendix 2: Concise Critical Notes: Articles and Papers Template Names of author(s) Full title of article Full title of journal Year published

Month:

Volume number

Issue number:

Research questions: What is the paper setting out to demonstrate? What is the theoretical position underlying the research? Type of theory(ies)? What is the key literature used as background to the article or paper? What guiding assumptions and research methods are used? What kind of sample is used and in what context? What are the key results? Key conclusions or recommendations Strengths of the research: What is good about the paper? How does it advance our understanding of the subject or how to research it? Are the methods used appropriate to the research context and consistent with guiding assumptions adopted? Consideration of ethics? Weaknesses of the research: In what ways is it limited? When and where would it not apply? What are the flaws in the research, in conceptualisation, design and methods, approach to analyses, conclusions drawn on the basis of the results? Relationship to other papers read: How does this paper relate to others I have read? What are the similarities and differences?

Adapted from Cottrell (2011, p. 157).

Appendix 2: Concise Critical Notes: Articles and Papers Template

549

Concise Critical Notes: Analysing Arguments in a Theoretical Paper Template Names of author(s)/source Title of book/programme Web-site address

Date downloaded:

Date and/or time

Edition:

Publisher/channel

Place published:

Volume of journal

Issue:

Author’s position/ theoretical position Essential background information Overall argument or hypothesis Conclusion Supporting reasons

1

5

2

6

3

7

4

8

Strengths of the line of reasoning and supporting evidence

Flaws in the argument and gaps or other weaknesses in the argument and supporting evidence

Adapted from Cottrell (2011, p. 155).

2

Central Trigger Question

How well does the researcher set out and leverage their own choices (e.g., patterns of guiding assumptions, research questions) and fit within the research context?

How well does the researcher clarify and leverage the fit of the various data sources within the research context?

How well does the researcher clarify and leverage knowledge about context to add richness and/or qualification to their research process and emergent findings?

Researcher Positioning

Positioning of Participants & Other Data Sources

Contextual Sensitivity

How well do the processes and outcomes of data analyes lead to or support defensible and clear conclusions, given the quality of data to hand?

Where intended, how effectively does the research produce or yield meanings or implications for other contexts?

Analytical Integrity

Extensional Reasoning

How well has the researcher hit the mark with respect to ensuring their research story is suitable for intended audience(s)?

Acknowledgement of Limitations

Presentational Character

Poor

3

Very Poor

2

Just adequate 4

Rating of the Reader's Impression of the Quality of 'Answer' to the Trigger Question as Reflected in the Research Outcome being Evaluated

6

Very Good

Adapted from Cooksey (2008, Table 1, pp. 7–9)

5

Good

Is the story in the research outcome being considered, in its entirety, convincing with respect to research processes undertaken and arguments being made?

How well has the researcher dealt with any surprises and unanticipated outcomes that emerged in their research?

How effectively and openly has the researcher signalled what constrains the learning value and applicability of their research?

How effectively has the researcher shown what others can run with or how they could build upon what has been learned from their research?

Fertilisation of Ideas

Handling of Unexpected Outcomes

How well has the researcher argued for what others can take away as important messages/meanings from their research?

Value for Learning

How well has the researcher's story been configured, argued and displayed for consumption/use by relevant target audiences within the research outcome being considered?

How well does the research, as a whole, hang together as a coherent process to permit the inferences and conclusions the researcher seeks or claims?

Internal Coherence

How well has the research been planned and executed, relative to the guiding assumptions that have been adopted and how the study was planned, executed and ultimately reported within the research outcome being considered?

How well does the researcher build on, and display its fit with respect to a body of the relevant works of others?

Juxtapositioning with Other Research

To aggregate raƟngs, simpling average the individual meta-criterion raƟngs covered by each bracket.

Extremely Poor 1

Overall Judgment of Convincingness

1

Meta-Criterion

How well does the research reflect and utilise effective considerations of contextual situatedness, scope, shape and boundaries when establishing and reflecting on the research problem and research questions/hypotheses within the research outcome being considered?

Extremely Good 7

Observations and Thoughts

Individual MetaCriterion Rating1 Domain Aggregate2

13

Explication

Realisation

Contextualisation

Meta-Criterion Domain

Appendix 3: Meta-criteria Research Outcome Evaluation Framework

550 How Should I Select, Read and Review the Literature?

Appendix 4: Literature Review Scoring Rubric

551

Appendix 4: Literature Review Scoring Rubric Category

Criterion

1

2

3

1. Coverage

A. Justified criteria for inclusion or exclusion from review B. Distinguished what has been done in the field from what needs to be done.

Did not discuss the criteria inclusion or exclusion Did not distinguish what has been and has not been done

Discussed the literature included and excluded Discussed what has been and has not been done

Justified inclusion and exclusion of literature Critically examined the state of the field

C. Placed the topic or problem in the broader scholarly literature

Topic not placed in broader scholarly literature History of topic not discussed

Some discussion of broader scholarly literature Some mention of history of topic

Topic clearly situated in broader scholarly literature

Key vocabulary not discussed

Key vocabulary defined

Discussed and resolved ambiguities in definitions

Key variables and phenomena not discussed

Reviewed relationships among key variables and phenomena

Noted ambiguities in literature and proposed new relationships

Accepted literature at face value

Some critique of literature

Offered new perspectives

H. Identified the main methodologies and research techniques that have been used in the field, and their advantages and disadvantages

Research methods not discussed

Some discussion of research methods used to produce claims

Critiqued research methods

I. Related ideas and theories in the field to research methodologies

Research methods not discussed

Critiques appropriateness of research methods to warrant claims

J. Rationalised the practical significance of the research problem

Practical significance of research not discussed

Some discussion of appropriateness of research methods to warrant claims Practical significance discussed

2. Synthesis

D. Placed the research in the historical context of the field E. Acquired and enhanced the subject vocabulary F. Articulated important variables and phenomena relevant to the topic G. Synthesised and gained a new perspective on the literature 3. Methodology

4. Significance

4

Critically examined history of topic

Introduced new methods to address problems with predominant methods

Critiqued practical significance of research

(continued)

552

13

How Should I Select, Read and Review the Literature?

(continued) Category

Criterion

1

2

3

4

Critiqued K. Rationalised the Scholarly Scholarly scholarly scholarly significance of significance significance of significance of the research not discussed research research problem discussed 5. Rhetoric L. Was written Poorly Some coherent Well-developed, with a coherent, conceptualised, structure coherent clear structure that haphazard supported the review Note The column-heading numbers represent scores for rating dissertation literature reviews on 3-point and 4-point scales (adapted from Boote & Beile, 2005, p. 8, who cited the original source as Hart, 1999)

References Benzies, K. M., Premji, S., Hayden, K. A., & Serrett, K. (2006). State-of-the-evidence reviews: Advantages and challenges of including grey literature. Worldviews on Evidence-Based Nursing, 3(2), 55–61. Berkeley Library. University of California. (2018). Retrieved November 4, 2018, from www.lib. berkeley.edu/TeachingLib/Guides/Internet/FindInfo.html. Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation review in research preparation. Educational Researcher, 34(6), 3–15. Booth, A., & Sutton, A. (2016). Systematic approaches to a successful literature review (2nd ed.). London: Sage Publications. Borgman, C. L. (2007). Scholarship in the digital age: Information, infrastructure, and the internet. Cambridge, MA: MIT Press. British Library. (2018). Retrieved December 29, 2018, from http://catalogue.bl.uk. Buzan, T. (2018). Mind map mastery. London: Watkins. Can You Find Reliable Information on the Internet? (2018). Retrieved December 29, 2018, from https://peopledevelopmentmagazine.com/2016/07/10/information-internet/. Clarivate Analytics: Master Journal List. (2018). Retrieved December 29, 2018, from http://mjl. clarivate.com/index.html. Clarivate Analytics: Web of Science. (2018). Retrieved December 29, 2018, from https://clarivate. com/products/web-of-science. Cooksey, R. W. (2008). Paradigm-independent meta-criteria for social & behavioural research. In Proceedings of the 2nd Annual Postgraduate Research Conference, University of New England, Armidale, NSW (pp. 4–17). Cooper, H. M. (1984). The integrative research review: A systematic approach. Beverly Hills, CA: Sage Publications. Cooper, H. M. (1988). Organizing knowledge synthesis: A taxonomy of literature reviews. Knowledge in Society, 1(1), 104–126. Cooper, H. M., & Hedges, L. V. (1994). The handbook of research synthesis. New York: Sage Publucations. Cottrell, S. (2011). Critical thinking skills: Developing effective analysis and arguments (2nd ed.). Houndmills, UK: Palgrave Macmillan. Craswell, G. (2005). Writing for academic success: A postgraduate guide. London: Sage Publications. Delamont, S., Atkinson, P., & Parry, O. (2000). The doctoral experience success and failure in graduate school. London: Falmer Press.

References

553

Dogpile. (2018). Retrieved November 5, 2018, from http://www.dogpile.com/. Evaluating Internet Resources. (2018). Retrieved December 29, 2018, from https://www.library. georgetown.edu/tutorials/research-guides/evaluating-internet-content. Evaluating Internet Sources: A Library Resource Guide. (2018). Retrieved December 29, 2018, from https://library.nmu.edu/guides/userguides/webeval.htm. Farace, D., & Schöpfel, J. (2010). Grey literature in library and information studies. New York: De Gruyter Saur. Finn, J. A. (2005). Getting a PhD: An action plan to help manage your research, your supervisor and your project. London: Routledge. Free Online Courses in Research Skills. (2018). Retrieved December 29, 2018, from https://www. class-central.com/tag/research-skills. Free Reading Speed Test. (2018). Retrieved December 29, 2018, from http://www.freereadingtest. com/. Gash, S. (2000). Effective literature searching for research (2nd ed.). Aldershot, UK: Gower Publishing. Georgia State University Library: Research Guides. (2018). Retrieved December 29, 2018, from http://research.library.gsu.edu/endnote. Google. (2018). Retrieved November 5, 2018, from https://www.google.com/. Google Scholar. (2018). Retrieved February 6, 2018, from https://scholar.google.com. Guide to Grammar. (2018). Retrieved December 29, 2018, from http://guidetogrammar.org/ grammar/index2.htm. Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015). The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLoS ONE, 10(9), e0138237. Hart, C. (2001). Doing a literature search: a comprehensive guide for the social sciences. London: Sage Publications. Harzing: Research in International Management. (2018). Retrieved December 29, 2018, from http://www.harzing.com. Hernandez, D. A., El-Masri, M. M., & Hernandez, C. A. (2008). Choosing and using citation and bibliographic database software (BDS). The Diabetes Educator, 34(3), 457–474. Hopewell, S., Clarke, M., & Mallett, S. (2005). Grey literature and systematic reviews. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 48–72). How Reliable is Wikipedia as a Source of Information, and Why? (2018). Retrieved December 29, 2018, from https://www.quora.com/How-reliable-is-Wikipedia-as-a-source-of-informationand-why. Hudson, H. T. (2018). The 6 online research skills your students need. Retrieved December 29, 2018, from https://www.scholastic.com/teachers/articles/teaching-content/6-online-researchskills-your-students-need/. International Directory of Search Engines. Search Engine Colossus. (2018). Retrieved November 5, 2018, from http://searchenginecolossus.com/. Kamler, B., & Thomson, P. (2014). Helping doctoral students to write: Pedagogies for supervision (2nd ed.). London: Routledge. Kilbourn, B. (2006). The qualitative doctoral dissertation proposal. Teachers College Record, 108 (4), 529–576. Learning Online: Searching and Researching. (2018). Retrieved December 29, 2018, from https:// www.futurelearn.com/courses/searching-and-researching. Levy, Y., & Ellis, T. J. (2006). A systems approach to conduct an effective literature review in support of information systems research. Information Science Journal, 9, 181–212. LexisNexis. (2018). Retrieved November 4, 2018, from https://www.lexisnexis.com/en-us/ gateway.page. Library of Congress. (2018). Retrieved December 29, 2018, from http://www.loc.gov. Lindsay, D. (1995). A guide to scientific writing (2nd ed.). Melbourne: Longman Cheshire.

554

13

How Should I Select, Read and Review the Literature?

Mahood, Q., Van Eerd, D., & Irvin, E. (2014). Searching for grey literature for systematic reviews: Challenges and benefits. Research Synthesis Methods, 5(3), 221–234. Networked Digital Library of Theses and Dissertations. (2018). Retrieved December 29, 2018, from http://www.ndltd.org. Nimbekar, T. P., Wanjari, B. E., Bais, Y. G., & Nema, M. V. (2012). The scenario of pharmacy journals with special emphasis on impact factor. International Journal of Pharmacy, 2(3), 507– 512. Nordquist, R. (2018). What are endnotes and how to use them. Retrieved December 29, 2018, from https://www.thoughtco.com/endnote-research-paper-1690650. Pappas. C. (2016). 7 Tips to enhance online research skills through eLearning. Retrieved December 29, 2018, from https://elearningindustry.com/7-tips-enhance-online-research-skillselearning. Pautasso, M. (2013). Ten simple rules for writing a literature review. PLoS Computational Biology, 9(7), e1003149. Petre, M., & Rugg, G. (2010). The unwritten rules of PhD research (2nd ed.). Maidenhead, UK: Open University Press. Phelps, R., Fisher, K., & Ellis, A. (2007). Organizing and managing your research: A practical guide for postgraduates. London: Sage Publication. Polonsky, J. M., & Waller, S. D. (2015). Designing and managing a research project: A business student’s guide (3rd ed.). Los Angeles: Sage Publications. ProQuest Dissertations & Theses Global. (2018). Retrieved December 29, 2018, from https:// www.proquest.com/products-services/pqdtglobal.html. Quinton, S., & Smallbone, T. (2006). Postgraduate research in business: A critical guide. London: Sage Publications. Randolph, J. (2009). A guide to writing the dissertation literature review. Practical Assessment Research and Evaluation, 14(13), 1–13. Reading Checks: Speed Reading Technology Since 1988. (2013). Retrieved December 29, 2018, from http://www.turboread.com/read_checks.htm. Ridley, D. (2012). The literature review: A step-by-step guide for students (2nd ed.). London: Sage Publications. Scheele, P. R. (2000). PhotoReading (3rd ed.). Minnetonka, MN: Learning Strategies Corporation. Scimago Journal & Country Rank. (2018). Retrieved December 29, 2018, from https://www. scimagojr.com/journalrank.php?category=3321. Search. (2018). Retrieved November 5, 2018, from https://www.search.com/. Search Engine Showdown. (2018). Retrieved December 29, 2018, from http:// searchengineshowdown.com/. Speed Reading Test Online. (2017). Retrieved December 29, 2018, from http://www.readingsoft. com/index.html. Thesaurus. (2018). Retrieved November 4, 2018, from https://www.thesaurus.com/. Thody, A. (2006). Writing and presenting research. London: Sage Publications. The University of Auckland. Doctoral policies and guidelines. (2018) The University of Auckland. Retrieved November 4, 2018, from https://www.auckland.ac.nz/en/students/academicinformation/postgraduate-students/postgraduate-policies-guidelines-and-forms/doctoralpolicies-and-guidelines.html. Wellington, J., Bathmaker, A. M., Hunt, C., McCulloch, G., & Sikes, P. (2005). Succeeding with your doctorate. London: Sage Publications. What are the Main Benefits of Using Endnote? (2013). Retrieved December 29, 2018, from https:// www.researchgate.net/post/What_are_the_main_benefits_of_using_Endnote. Windweaver. (2006). Windweaver’s Search Links. Retrieved December 29, 2018, from http:// www.windweaver.com/searchlinks.htm. Your Dictionary. (2018). Retrieved November 4, 2018, from http://www.yourdictionary.com/.

Chapter 14

What Data Gathering Strategies Should I Use?

There are many data gathering strategies that can be used by postgraduates in social and behavioural research. In this chapter, we explore each strategy in more detail, but within a novel organisational framework, unlike those used in most research methods texts. In light of our pluralist perspective, we consider each data gathering strategy, not only as a distinct and self-contained strategy (which may encompass a range of more specific data gathering approaches), but also as part of a larger more interconnected and dynamic toolkit for social and behavioural researchers. Figure 14.1 presents a two-level mindmap of data gathering strategies and shows that data gathering strategies can be organised into six primary categories: • Interaction-based strategies—you (or other trained interviewers) interact directly with people; • Observation-based strategies—you (or one or more trained observers) directly or indirectly observe the behaviours and interactions of people; • Participant-centred strategies—you invite participants to produce their own data in some form; • Artefact-based strategies—you sample, extract data from or analyse artefacts that serve as sources of data (a very wide definition of ‘artefact’ is applied here, following the arguments set forward by Plowright, 2011, pp. 92–97); • Data-shaping strategies—you construct specific instruments, questionnaires, devices, coding systems or other data-generating or data-transformation processes to produce quantitative and/or qualitative data; and • Experience-focused strategies—you gather data in the context of specific structured experiences that participants undergo, or you gather data in the context of experiences structured and provided by others or in the context of natural events, which participants live through and deal with. These six primary data gathering strategy categories can also be usefully organised into one of three higher-order domains (superimposed ovals):

© Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_14

555

556

14

What Data Gathering Strategies Should I Use?

Connecting with People

Structuring People’s Experiences

Exploring People’s Handiworks

Fig. 14.1 Two-level mindmap of the toolkit of data gathering strategies showing the first-order and higher-order categories and some common dynamic combinations in the context of research configurations and processes [typical pattern(s) of guiding assumptions alignment(s) for each strategy are shown in parentheses: P = Positivist (and perhaps critical realist); I/C = Interpretivist/ Constructivist); signals that the data gathering strategy can be useful under more diverse patterns of guiding assumptions]

• Connecting with People—strategies that facilitate connecting with people through direct or indirect interaction and observation; • Exploring People’s Handiworks—strategies that facilitate exploration and understanding of the handiworks people produce or perform in specific contexts; and • Structuring People’s Experiences—strategies that facilitate construction, provision and/or evaluation of designed, structured or naturally-occurring experiences of people. Some data gathering strategies are congruent with a specific pattern of guiding assumptions whereas others can be aligned with more diverse patterns. Some strategies form natural obligatory pairings such as experience-focused strategies and data-shaping strategies under the positivist pattern of guiding assumptions, and where

14

What Data Gathering Strategies Should I Use?

557

this happens, we would not consider that pairing to constitute a pluralist approach. However, many other pairings of strategies can provide synergistic gains in the context of different MUs, notably in pluralist investigations where different data gathering strategies, different types of data (quantitative and qualitative) and/or different types of data sources are to be included. Figure 14.1 signals some of the potential synergistic combinations between the primary strategy categories and, for each specific strategy, indicates its dominant or preferred alignment with one or more patterns of guiding assumptions. [Note that the limitations of two-dimensional space prevent the visualisation of synergistic combinations between more than two data gathering strategies in the mindmap. It is important to realise that three or more data gathering strategies may be used synergistically in different types of MU configurations within different research frames and under different patterns of guiding assumptions.] Our intention in this chapter is not to provide a comprehensive how-to coverage of each strategy. There are many fine research methods reference books and textbooks that can do that job. Our goal, instead, is to highlight some key considerations and issues associated with each strategy that might be relevant to your decision making about which might be appropriate for you to use as part of your research journey, given your research frame, pattern(s) of guiding assumptions, contextualisations, positionings, research questions/hypotheses, scoping and shaping considerations and MU configuration.

14.1

Strategies for Connecting with People

One important way you can gather data for research purposes is to connect directly or indirectly with the people whose behaviours, social interactions, relationships, perspectives, cultures, beliefs, values, attitudes and/or thoughts are of focal interest. With interaction-based strategies, you or some other interviewer creates and maintains a direct social connection with participants as a fundamental part of the data gathering process. Just like any other social interaction, particularly between strangers, things can go well, things can go badly, or things can start well but then head south. The key is developing and maintaining trust. This means that effective social interactions need to be planned for and practiced including how to get things going (e.g., ‘breaking the ice’) and how to wind things up. With observation-based strategies, you employ perception rather than social interaction for data gathering. Here, the tricky part is ensuring that those perceptions are not biased or misconstrued and, for you or some other observer, this means learning how and where to look and what to look for or what to take note of and record. Figure 14.2 expands, to three-levels, that portion of the two-level mindmap shown in Fig. 14.1 displaying all the data gathering strategies that share the goal of connecting with people in some way. The three-level expansion reveals branches that identify two or more key features and/or considerations associated with each data gathering strategy. The interconnections between the connecting with people data gathering strategies and other data gathering strategies in the toolkit remain visible as a constant reminder that synergies are always possible for you to consider.

14

Fig. 14.2 Expanded 3-level mindmap branches focusing on strategies for connecting with people (encompassing interaction-based and observation-based data gathering strategies) as well as some key considerations associated with each strategy

558 What Data Gathering Strategies Should I Use?

14.1

Strategies for Connecting with People

559

14.1.1 Interaction-Based Strategies Interaction-based data gathering strategies involve some mode of social connection and interaction, either face-to-face or virtually through technological mediation and support (e.g. telephone or mobile phone, video-conference, Skype call, FaceTime or chat room). Interviews are essentially conversations with the explicit purpose of providing you with research data. They are often conducted one-on-one with a participant, but can be done at a group-level, as a group interview or focus group. The data gathered in an interview may be quantitative, qualitative or both, depending upon the type of interview conducted. Typically, the less structured the interview, the more in-depth you can go with respect to exploring participants’ perspectives and the less you will be interested in condensing or transforming the data into quantitative indicators. You may conduct the interviews yourself or may enlist and train others to conduct them. Where possible, it is preferable that you conduct at least a portion of the interviews to make sure you stay connected to and familiar with the data being gathered. Interviews are necessarily a more personal form of data collection, especially as their structure decreases and expectations of participant involvement increase. This can enhance response rates, especially for face-to-face interviews, because once the interview has commenced, it is more difficult, both psychologically and socially, for a participant to pull the plug unless they become truly uncomfortable. Equally, however, some people find interviews, especially face-to-face, difficult to participate in precisely because they involve social interaction, typically with a stranger, and this may not suit their personality or social preferences. This may then turn them off participating in a face-to-face interview. Having another modality for interview participation available as a Plan B to suit such instances could be a viable data gathering tactic for you to adopt. Face-to-face interviews allow you access to nonverbal as well as verbal cues, which can add richness to the data gathered. Interviews mediated by some sort of technological interface will typically sacrifice access to at least some types of nonverbal and other contextual information with non-visual modalities affected more so than visual modalities. Thus, in terms of richness of access to verbal and nonverbal information as well as contextual information, the ranking of interview modalities, in decreasing order of data richness, is: face-to-face (visual, auditory, olfactory, tactile and 360-degree contextual information all available) ! Skype, FaceTime or other video-conferencing technology (narrowly-focused visual and auditory information plus limited and narrowly-focused contextual information available) ! phone or mobile phone (auditory and very limited and narrowly focused contextual information available) ! web-based (computer-interface only, without contextual information). Thus, a critical consideration for choosing the most useful modality for an interview is how important it is for you to be able to access nonverbal and contextual information along with interview content. Interviews, particularly those involving less researcher-determined structure, combine well with other data gathering strategies, making them a popular choice for

560

14

What Data Gathering Strategies Should I Use?

pluralist research, especially under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions. In general, interviews of one kind or another can be useful in virtually every research frame you could adopt and in every type of MU configuration you can envisage, because they yield data directly from participants. This direct sourcing of data is the strength of the interaction-based strategies. However, unless you are very careful in attending to how the interactions are managed from start to finish, that strength can turn into a weakness, resulting in very poor data as well as potential adverse cascading effects for your project as well as for other researchers (e.g., one bad interview can put a person off participating in any future interviews). Interviews are a resource-intensive mode of data gathering, especially if conducted one-on-one. The less structured interviews are, the more effort you must invest to carry them off successfully and many researchers can find this exhausting. The risk of errors created by negative interviewer attitudes and inadvertent/ inappropriate behaviours is relatively high, especially as the degree of structure increases. For interviews, even highly structured ones, to work well, you need to devote some initial attention to building at least a minimal level of rapport with the participant, so their responses have some validity or authenticity. The less structured the interview, thereby expecting proportionally greater depth of participant contribution, the more attention you or other interviewers need to devote to building rapport. Establishing rapport helps to build trust and the presence of trust paves the way to authenticity or validity, depending upon the pattern of guiding assumptions adopted. You should recognise that participant responses to questions or the information/insights they offer or are willing to offer during an interview may be influenced away from validity or authenticity by attitudes, beliefs, fears, worries about what others might think of them, suspicions about the interviewer, you or the research itself, feeling threatened or unsafe, feeling resentment or frustration that they can’t voice their own views (a risk especially for structured interviews) and political or other agendas (real or imagined). Success in an interview depends on how well you plan for and consistently work at building and then maintaining participant rapport and trust. The four interviewing strategies reviewed below each unfold in a somewhat different manner, under different patterns of guiding assumptions, and with differing degrees of structure. They should not be seen as distinct non-overlapping strategies but as strategies whose characteristics may be blended to some extent to help you achieve your purposes. Thus, for example, a focused interview may have attributes of a semi-structured interview and focus group interviews may be highly structured, semi-structured or unstructured. Important Considerations for Interviewing in Indigenous Contexts It must be noted that each of the interviewing strategies reviewed below reflects a westernised relational stance between you and those who are participating in your research. The westernised relational stance emphasises (1) the importance of the individual rather than the collective or community when gathering data, (2) the expectation that you should have access to the worldviews of the participants as a social

14.1

Strategies for Connecting with People

561

science investigative right; (3) that the knowledge possessed by the participants is not privileged nor has primacy (including historical primacy) over your knowledge as researcher; and (4) the only relationship that needs to be safeguarded is that between yourself, as researcher, and the researched (the participants) in their individual interactions, ignoring the fact that, historically, the westernised stance has dominated Indigenous stances and lives through the processes of colonisation (see, for example, the various discussions in Chilisa, 2012; Chilisa & Tsheko, 2014; Kovach, 2009, 2018). From an Indigenous perspective and in the context of the Indigenous research frame, these emphases are seen to be inappropriate, oppressive and culturallyinsensitive, which is why certain aspects of the pattern of Indigenous research guiding assumptions mirror aspects of the pattern of critical social science guiding assumptions. [As a positioning note, both Gael and I, as authors of this book, are non-Indigenous causasian people from western cultural backgrounds, Gael, originally, from New Zealand and Ray, originally, from the United States, so we cannot speak as insiders to any Indigenous culture. What we can do is point to pathways for addressing Indigenous concerns associated with westernised research approaches.] Unless you are a member of the Indigenous culture involved in your research, adopting a postcolonial stance works to counteract the view that you are the ‘coloniser’ and participants are the ‘colonised’ by ‘decolonising’ these roles through how interactions for gathering data, and the knowledge they grant access to, unfold. Accordingly, when conducting interviews within the Indigenous research frame, you need to adopt a completely different ‘decolonised’ mindset. This gives rise to the process of engaging in ‘postcolonial Indigenous interviews’ (Chilisa, 2012, Chap. 7), where the westernised stance is rejected through adoption of the inverse of every emphasis listed in the previous paragraph. This repositions participants as ‘cultural insiders/knowers’ and you as a ‘cultural outsider/learner’, especially when and where you are non-Indigenous. Virtually every interviewing strategy can be recast in a postcolonial mode. For example, focused, semi-structured or unstructured interviews can be recast as relational interviews or conversations that emphasise relational ways of knowing. This gives primacy to the worldviews, knowledge and relationships that are part of the lifespaces, histories and environments/ecologies relevant to the participants (consistent with the ‘culturally-privileged’ assumption choice for pivotal question 7 for deciding on guiding assumptions; recall Table 9.1). Such interviews may not work well as one-on-one interactions, needing instead to incorporate voices and value the knowledge shared by others and with each other at the same time, becoming a form of group-based conversation. This comes about because, in general, no one Indigenous person holds or has access to all relevant knowledge or relationships. As part of the interview process, cultural objects and symbols may serve as important focal stimuli for the interactions. As another example, focus group interviews can be recast as talking circles where every member of the group enjoys equal voice, time and space to contribute. An important process for gaining access to conduct interviews with Indigenous people (whether on Indigenous land/in Indigenous country or not) may be to interview tribal elders first to gain insights and build upon their wisdom and long

562

14

What Data Gathering Strategies Should I Use?

life history and experience (what Chilisa, 2012, refers to as ‘philosophical sagacity’). This may also serve as a springboard to bringing others into the interview process where they have relevant knowledge. Kovach (2009) indicates that there may be cultural protocols to be followed (such as the offering of a gift) before access may be given for interviews (an ethical consideration) and reinforces the importance of negotiating all aspects of the data gathering process with relevant tribal representatives (e.g., whether conversations can or should be recorded or transcripts produced, whether pictures or videos can be taken, what can and cannot be shared beyond the boundaries of the research). This consideration of cultural protocols is an important sign of respect for an Indigenous community and is not something that can or should be taken lightly. Structured Interaction-Based Strategy This strategy employs structured interviews to gather data, which typically involves the oral delivery of and responses to a structured questionnaire. The structured interview is most closely aligned with the positivist pattern of guiding assumptions, typically yielding quantitative or quantifiable qualitative data. A structured interview creates two interacting roles, interviewer and participant, where the power dynamic strongly favours the interviewer. This means that it is not an adaptable type of interview. Instead the interview follows a strict script and any deviations from that script can create internal validity and reliability problems with your data. To work effectively, you need to develop a minimal level of rapport to establish a baseline level of trust. However, in general, this type of interview involves minimal social interaction, focusing mostly on what is needed to yield valid and reliable data. Throughout a structured interview, the participant’s role is passive—only responding to questions. Any information participants volunteer or questions they raise (with respect to their own context or to research issues they are being interviewed about) on their own initiative are typically ignored. Figure 14.3 graphically illustrates the dominance of interviewer talk over time in a structured interview and shows the typical segmentation of such an interview. Structured interviews are especially useful in a Survey, Explanatory, Evaluation, Cross-Cultural or Descriptive research frame, where research purposes include obtaining comparative responses to a standard set of questions, where literacy or other constraints may preclude obtaining written responses to questions from participants and/or where you feel that face-to-face communication may yield better data. Structured interviews may be used in single, simultaneous or sequential MU configurations. Structured interviews are always combined with the Measurements data-shaping strategy which scripts the questions to be asked as well as the probes to be used in following up unclear or conditional responses. Structured interviews may also be used in conjunction with experience-focused strategies where questions focus on participants’ experiences. Interviewers must be highly trained, ideally to an identical level of proficiency, in: • standardised verbal delivery and ordering of questions and managing the interview so it is as comfortable as possible, given its artificiality;

14.1

Strategies for Connecting with People

563

Participant Interviewer Establishing rapport

Setting context & instructions

Easier Demographic Questions

More Complex & Specific Rating Questions

Question-Answer interchanges following Interview Schedule

Easier More Generalist Questions

Winding down

Interview Timeline Notes: The interviewer drives the session using a strict Q & A style of interaction. Talking over each other or interruptions are to be minimised as they can influence data quality. Deviations from interview schedule are not permitted. The interviewer probes more deeply if more detail needed from participant and such probes are generally scripted.

Fig. 14.3 Graphic illustration of the typical trajectory of interchanges between interviewer and interviewee over time in a structured interview.

• for face-to-face interviews, managing the setting for interviews so that quiet and comfort are maximised, and risks of interruptions and distractions are minimised; • for technologically-mediated interviews, ensuring that the internet or phone connection is clear, static-free and reliable with minimal delays (a good broadband connection helps where the internet is used) and the environment, at least at the interviewer’s end, is quiet and free from distractions; • maintaining control over nonverbal behaviours (e.g., managing eye contact, non-judgemental reactions and behaviours, personal space), mode of dress and the context in which interview is conducted; • implementing accurate data recording processes; • managing further probing, where required, to follow-up a response to a question within the boundaries set for such probing (even probing is scripted in structured interviews); and • how to effectively end a failing interview without creating bad feelings with the participant as well as how to handle problematic interviewee behaviours. Structured interviews may be conducted face-to-face or via phone, internet, FaceTime or Skype. Face-to-face interviews are the most personal and tend to yield higher response rates/lower withdrawal rates relative to technologically-mediated interviews. Importantly though, the potential impact of nonverbal behaviours tends to be much higher in face-to-face or visual technologically-mediated interviews. Interviews mediated by non-visual technology run a higher risk of participant withdrawal in that it is far easier for a participant to hang up or disconnect a phone call or switch away from a webpage, because they don’t risk seeing disappointment or disapproval on the interviewer’s face. Non-visual technologically-mediated interviews may be more efficient from a data gathering and recording perspective and the impact of nonverbal behaviours (except those evident in speech) will tend to be lower, but generally at the cost of the interview seeming less personal and engaging to the participant.

564

14

What Data Gathering Strategies Should I Use?

The design, ordering, complexity and flow of interview questions (i.e., the interview schedule) needs to be carefully thought through. The schedule should commence with easy (and less sensitive) questions, then move to more complex (perhaps more sensitive) questions that require more cognitive effort, and finally end with a couple of easier (less sensitive) questions. Any questionnaire that the interviewer uses as a script should be evaluated for construct validity and reliability and for content validity if an objective measurement approach is used (see the discussions of the Measurement data-shaping strategy as well as Chap. 18). Structured interviews, especially if long or complex, can be boring/tiring for participants, so careful attention to length and complexity is required. One strategy for reducing this risk is to include a few open-ended questions in the interview schedule so that participants can voice their own views (even if the information they provide is ultimately not of interest to you, as the researcher) and get a break from the rigid question and answer rhythm. If the structured interview is intended to rely on cross-cultural interactions and content, interviewer background and behaviour (e.g., eye contact, mode of dress, gender, cultural, ethnic or religious matching of interviewer and participant) must be attended to and the conceptual content and meaning of questions must be shown to be equivalent for the different cultural groups being sampled. The language for cross-cultural interviews needs to be carefully considered and planned for (e.g., avoid use of negatively worded and/or overly-complicated questions if interviewing Non-English-Speaking Background (NESB) participants in English). Participants should know, before they agree to participate, roughly how long the interview should take and what recompense they can expect, if any, for their participation, how their responses will be recorded and whether their responses will be anonymous or confidential. Time at the start of each interview should be set aside for building up a minimal level of rapport/trust, sufficient to sustain participation. No data should be gathered during this rapport building period and talk during this phase may or may not be scripted. A short period of time should be allocated at the end of the interview for winding down and thanking the participant. The interview should always close on a simple, clear and positive note. Data recording procedures need to be very clear and foolproof. If the interview process is technologically mediated, data recording processes must be tested for accuracy and have appropriate error-trapping procedures in place. Pilot testing of the questionnaire/interview schedule, interview process and data recording procedures is essential but is resource-intensive and adds to the timeline for your research; more so if multiple interviewers are used at different research sites. The training of all interviewers should be evaluated during the pilot testing process, which will also add to the timetable for your research. There are several systems you can use to effectively support the structured interviewing strategy, including: • using your research journal for recording notes and reflections on the interview process itself, after the fact;

14.1

Strategies for Connecting with People

565

• designing and utilising clear data recording/coding protocols and forms to improve data accuracy and reliability, a process that can be enhanced using computerised data capture; and • using computer-assisted interviewing software to support the interview process in real-time (e.g., Computer-Assisted Telephone Interview (CATI; e.g., http:// www.surveysystem.com/interviewing-cati.htm; http://www.surveysystem.com/ online-interviewing-cati.htm) or Computer-Assisted Personal Interview (CAPI; e.g., http://www.surveysystem.com/android-survey-software.htm). Babbie (2011, pp. 291–296), Bryman and Bell (2015, Chap. 9) and Gillham (2005, Chap. 11) all provide excellent general and well-illustrated discussions about using the structured interview as a data gathering strategy. Focused Interaction-Based Strategy The focused interaction-based strategy employs focused interviews that target one or more issues that have been identified, either by you, as the researcher, or by the participant, as important or useful to explore. Focused interviews may be used in single, simultaneous, sequential or case-based MU configurations. They may also be useful in a hierarchical MU configuration as either the dominant or subordinate approach. Focused interviews were originally conceptualised as a strategy under the positivist pattern of guiding assumptions but are now commonly used under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions as well. Focused interviews may be useful in the Case Study, Exploratory, Transdisciplinary, Evaluation, Developmental Evaluation, Action, Feminist or Indigenous (if ‘decolonised’) research frames, especially where you are targeting a specific focal event, innovation or experience or the focus is nominated by the participant. Under the positivist pattern of guiding assumptions, focused interviews tend to be conducted in a structured manner, often in combination with data-shaping strategies and/or Textual artefact-based strategy (where documents associated with the focal events, incidents or experiences are content analysed). Under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, focused interviews tend to be conducted in a semi-structured or unstructured manner (thus combining it with the Depth interaction-based strategy) and may also be combined with other approaches that yield qualitative data such as the Visualisations participant-centred strategy, the Participant observation-based strategy or the Textual artefact-based strategy. Focused interviews can be particularly useful in conjunction with the Non-manipulative experience-focused strategy. The critical incident technique (Chell 2004; Flanagan, in 1954, originated the technique under the positivist pattern of guiding assumptions) is an important type of focused interview. A critical incident or event is one that may have occurred in the past (giving a retrospective focus), may be currently unfolding (giving a present-day focus) or may be planned or anticipated in the future (giving a prospective focus) and should have clear boundaries, purposes and outcomes (Flanagan, 1954). The interview questions and discussion are then used to explore this incident or event with the participant in some depth including issues, feelings,

566

14

What Data Gathering Strategies Should I Use?

Participant Interviewer Establishing rapport & trust

Setting/Negotiating focal context/events

Conversation interchanges w/ interviewer keeping dialog focused

Winding down

Interview Timeline Notes: Proportion of talk favours the participant although talking over each other and interruptions may also occur (not depicted). Interviewer and participant may negotiate the focus of the interview (as with a critical incident approach) or the researcher/interviewer may nominate the focus (linked to research questions).

Fig. 14.4 Graphic illustration of the typical trajectory of phases and interchanges between interviewer and interviewee over time in a focused interview.

attitudes, causes, consequences and reactions associated with each incident. If participants are asked to identify or describe critical incidents, then it is important that you concentrate at least part of the interview on why they identified those incidents as critical. If you, as researcher, identify the critical incidents, then you must tell a clear story about how and why you selected specific incidents as being critical. If you conduct focused interviews under the positivist pattern of guiding assumptions, you will be more likely to prescribe the focal events or incident(s) for data gathering to focus on. Alternatively, you may ask the participant to nominate one or more focal or critical incidents/events in some context to describe in detail (what they nominate may form one purpose for the interview). Your role in such an interview will tend to be more directive and dominant. If you conduct focused interviews under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, you will be much more likely to invite the participant to nominate one or more critical incidents/events in some context to focus on in the interview and what counts as a critical or relevant focal incident may itself be up for negotiation early in the interview as such ‘criticality’ may be subjectively experienced. Your role in such an interview will tend to be more active listener than directive, using a semi-structured depth interview format with more generic who-, what-, where-, why- and how-type questions (see Chell, 2004) and focusing on unpacking the participant’s understandings, feelings, views and constructed meanings surrounding the nominated event(s) or incident(s). Figure 14.4 depicts the typical interaction pattern between interviewer and participant over time as well as important phases of a focused interview. Focused interviews need to have time allocated early on for the building of rapport and trust followed by a period for setting up or negotiating the focal events/incidents with the participant. The focal incidents/events must be relevant to or meaningful to the participants, something that can be ensured if the participant helps to identify them. Focused interviews may be much less useful if participants have not directly experienced the incidents/events being discussed. The focused interview may be

14.1

Strategies for Connecting with People

567

constrained by its focal incidents and events, which may mean you do not gain access to the larger contextual picture. This can be countered by tapping into other data sources, by devoting a portion of the interview to discussing the larger context surrounding the critical incidents/events or by pairing focused interviews with a different data gathering strategy in a pluralist MU configuration. Contextualisation of each focused interview is essential, including reflections on the interview process itself and how it unfolded, so that you can consider the context of the interview as part of your interpretation processes (such contextualisation observations can easily be recorded in your research journal). Equally, where possible, you should look for and note any contextualising commentary offered by the participant in association with a specific issue or event they are discussing. In some cases, the participant might be invited to further clarify their discussion by providing some context for it. Under an interpretivist/constructivist pattern of guiding assumptions, all such contextualisation will assist in maintaining authenticity with respect to the participant’s perspectives and helps to ensure contextual sensitivity. Under the positivist pattern of guiding assumptions, all such contextualisation will assist in ensuring the comparability of learning across different interviews, enhancing both internal and external validity in the process. Trust is important to build in focused interviews as, in many cases, there may be sensitivities attached to the incidents/events being discussed (e.g., feelings of anger, frustration, guilt, shame or blame). In an Indigenous research frame, the focused interview must be postcolonial in conception and execution (e.g., see Chilisa, 2012) and follow cultural protocols for trust to emerge. Here, trust becomes the base platform for willingness to share viewpoints and information with the researcher and part of such trust involves the participant believing that (1) you, as the researcher, will properly receive, respect, respond to and safeguard what has been offered; and (2) that you have followed the cultural pathways and protocols necessary to be given the opportunity to discuss the focal issues with them. There are several systems you can use to effectively support the focused interview strategy, including: • using your research journal for recording notes during the interview process as well as for recording your impressions and observations, after the fact (this can help to enhance the transparency of your reflections); • employing a digital recorder or mobile phone/tablet with a voice recording app (such as Dragon Professional, see https://www.voicerecognition.com.au/ dragon-software.htm (speech recognition software for Mac or PC); Voice Record Pro (for Apple devices), see https://itunes.apple.com/au/app/voicerecord-pro/id546983235?mt=8 or Hi-Q MP3 Voice Recorder (for Android phones); see https://play.google.com/store/apps/details?id=com.hiqrecorder. full&hl=en) for recording interviews; and • accessing any documentary evidence associated with the focal events, incidents or experiences.

568

14

What Data Gathering Strategies Should I Use?

Cohen, Manion, and Morrison (2011, Chap. 21, pp. 437–439), Collis and Hussey (2009, pp. 147–148) and Flick (2014, Chap. 16, pp. 211–217) all provide sound discussion and illustration of the focused interview strategy. Flanagan (1954) and Chell (2004) discuss the critical incident approach in detail in the context of differing sets of guiding assumptions. Chilisa (2012, Chap. 7) provides an excellent discussion of Indigenous approaches to focused interviews. Depth Interaction-Based Strategy The Depth interaction-based strategy can be enacted using either semi-structured or unstructured interviews. Semi-structured interviews (sometimes called ‘in-depth’ interviews) are largely open-ended but guided by an overarching framework of conversation topics, called a ‘topical landscape’. Unstructured interviews (sometimes called ‘non-directive’ interviews) are the most open-ended qualitative data gathering strategy, closest to what might be considered a natural conversation between two people. Semi-structured Interviews Semi-structured interviews are nearly always associated with interpretivist/ constructivist or another non-positivist pattern of guiding assumptions. In particular, with appropriate reconceptualisation as a postcolonial interview, semi-structured interviews can work well under the Indigenous pattern of guiding assumptions. Semi-structured interviews may be useful in the Case Study, Exploratory, Explanatory, Transdisciplinary, Evaluation, Developmental Evaluation, Cross-Cultural, Action, Feminist or Indigenous (if ‘decolonised’) research frames. Research that employs a grounded theory approach in the Explanatory research frame will frequently use semi-structured interviews. Semi-structured interviews can be employed in virtually any MU configuration, but often play a key role in the lead-off MU in an exploratory sequential MU configuration and as a primary MU strategy in a hybrid case-based MU configuration (within the Case Study research frame). Semi-structured interviews can be effectively combined with participant observation or with almost any participant-centred or experience-focused data gathering strategy (particularly the Non-manipulative experience-focused strategy), depending upon the pattern of guiding assumptions, research frame and MU configuration you adopt. In the Indigenous research frame, semi-structured interviews may be effectively combined with the Stories participant-centred strategy; such interviews then become largely story-sharing sessions. Semi-structured interviews are generally conducted face-to-face, one-on-one and yield qualitative data. Unlike structured interviews where the conversation is intentionally highly scripted, semi-structured interviews involve a more natural conversation, but with an interest in ensuring that certain specific topics or issues are touched upon or discussed. The flow of the interview is guided more by the participant than by you, as researcher (as depicted in the interaction flow over time shown in Fig. 14.5). You should map the general range of issues to be covered in the interview by creating a ‘topical landscape’ (a mindmap is a useful nonlinear form for

14.1

Strategies for Connecting with People

569

Participant

Interviewer Establishing rapport & trust

Setting context

Conversation interchanges touching base with topical landscape

Winding down

Interview Timeline

Sample topical landscape for semi-structured interviews of HR directors Notes: The proportion of talk favours the participant. Talking over each other and interruptions may also occur (not depicted). The interviewer attempts to ensure conversation touches all aspects of intended topical landscape through periodic topic shifting segues and perhaps following cues from the participant themselves. A strict Q & A-style of conversation to be avoided, i.e., the interview should not be scripted.

Fig. 14.5 Graphic illustration of the typical trajectory of interchanges between interviewer and interviewee over time in a semi-structured interview, with an illustrative topical landscape mindmap.

the landscape, since the interview will tend to unfold nonlinearly; see the example in Fig. 14.5). The issues informing the topical landscape should emerge from your research questions and should resonate with the participant’s context. The topical landscape is typically not traversed in any fixed order and you should look for early discussions to trigger natural shifts to other branches in the map. As far as possible, the interview should be guided by the participant. Your role is largely one of redirecting the conversation where necessary or possibly introducing a topic for discussion. If you need to redirect the focus of discussion or get the interview back on track, you should look for emerging conversational cues from which to subtly move the conversation in a desirable direction. A semi-structured interview is highly flexible and adaptable in that the interview can cover issues in any order while remaining open to following up any emergent issues and associated leads. This is what makes it a valuable strategy for grounded theory research because it facilitates theoretical sampling for testing emerging interpretations and ideas by allowing the topical landscape to evolve through the course of the interviews.

570

14

What Data Gathering Strategies Should I Use?

Questioning during the interview should be open and neutral so as not to suggest or presuppose specific meanings or interpretations; e.g., questions such as “What do you think about X?” or “How did X affect you?” are more open/neutral than questions such as “do you think that X was a poor outcome?” or “did the fact that X happened make you feel angry?”. Semi-structured interviews tend toward being longer interviews (60- to 90-min interviews would not be that uncommon) and are typically physically as well as mentally tiring because you must be ‘on your toes’ for the entirety of the interview. You should schedule some preparation time prior to each interview to review your topical landscape and some self-debriefing time after the interview to record and reflect on the interview immediately after its completion (a strategy for enhancing transparency as well as authenticity as well as providing you with an opportunity to re-centre and re-energise yourself). This means that you should not plan to conduct more than 3 or 4 semi-structured interviews in any one day (e.g., assuming 60-min interviews, 15 min for preparation and 30 min for reflective debriefing, 4 interviews in a day would mean a total of 420 min or 7 h time on task—very exhausting for most people). The location and environment for the interview should be quiet and comfortable, on the participant’s own turf, where possible. Interviews should be digitally recorded but only with the consent of the participant and only after they are made aware of how you will handle and safeguard their recorded talk. Semi-structured interviews take effort and practice to become proficient at conducting. This means you should allocate project time, prior to commencement of data gathering, to conduct and reflect upon some trial interviews. The flexible and adaptable nature of the semi-structured interview imposes a greater demand on you to keep clear notes on how and why the topical structure of the interviews changed over time as well as from one interview to the next and to consider this evolution in all subsequent analyses. In this regard, your research journal becomes an indispensable resource for ensuring transparency and helping to maintain authenticity. Contextualisation of every interview is essential (including your reflections on the interview process itself and how it unfolded) so that you can consider the context of the interview as part of the interpretive process. Equally, where possible, you should look for and note any contextualising commentary offered by the participant. In some cases, you might invite the participant to further clarify their discussion by providing some context for it. All such contextualisation will assist in maintaining authenticity with respect to the participant’s perspectives and helps to ensure contextual sensitivity. The biggest danger associated with semi-structured interviews is the potential for your own preconceptions to inadvertently influence or dominate the course of the conversation. If this occurs, trust will be eroded, and authenticity may be irreparably damaged. Accordingly, you must be constantly vigilant in letting the participant steer the flow of conversation as far as it is possible to do so. Shifts between topics should be as natural as possible, with only minimal ‘nudges’ from you. If semi-structured interviews transcend cultural boundaries in the Cross-Cultural research frame (where you and the participant come from different cultural backgrounds), you must take care to form appropriate interviewer/

14.1

Strategies for Connecting with People

571

interviewee pairings and maintain culturally appropriate language and behaviour (verbal and nonverbal) throughout the interview. Such interviews will typically take longer to complete because it will take longer to ensure meanings and intentions are clear. If you intend to use semi-structured interviews in an Indigenous research frame, your interview should be recast and conducted under a postcolonial mindset with appropriate respect for and attention to relevant cultural protocols and expectations. Failure to do so will immediately and dramatically erode authenticity and your contextual access might be revoked. Unstructured Interviews Of all the interviewing strategies, unstructured interviews are the most difficult to carry out successfully, being dependent upon developing a high level of rapport and trust. They require the most delicate balancing act between you, as the researcher, wanting to learn things in the context of your research and needing to listen to the unprompted perspectives of the participants. Unstructured interviews are thus very well-suited for constructing oral or life/biographical histories. Clinical psychologists and counsellors often use unstructured or non-directive interviews as part of their therapeutic process, acting more as a mirror or active listener than an interviewer. Unstructured interviews are always associated with an interpretivist/ constructivist pattern of guiding assumptions. In addition, with reconceptualisation as a postcolonial interview, unstructured interviews can work well under the Indigenous pattern of guiding assumptions. Unstructured interviews often take multiple sessions to complete which means they are well-suited for use in a sequential MU configuration. If such interviews are aligned to specific time periods within a participant’s life, then a sequential MU configuration morphs into a longitudinal MU configuration. However, unstructured interviews may also be useful within a case-based MU configuration. If you are assembling life histories or oral histories, these can unfold within a single MU configuration. Unstructured interviews are frequently combined with the Stories or Visualisations participant-centred strategy or the Multi-media artefact-based strategy. In the Indigenous and Feminist research frames, unstructured interviews may evolve into vehicles for participants to share life stories, oral histories or political histories. The unstructured interview is not scripted, rather it unfolds as a natural conversation. Accordingly, the building of strong rapport and deep trust is critical and, in some cases, may not be fully developed until several interviews with the same person have been conducted. You must be non-directive and non-judgmental throughout the interview process in order to strengthen and maintain the bond of trust. The unstructured interview is entirely driven by the participant in that you go where the participant leads. Your role is as an active listener or mirror for the participant’s views, following the participant down the paths he/she is exploring. Essentially, the participant is in the foreground of the interview, you are in the background (note the interaction pattern across time in Fig. 14.6). An unstructured interview is completely open in that the participant can discuss any issues that come to mind. Your contributions to the conversation will consist mostly of requests for

572

14

What Data Gathering Strategies Should I Use?

Participant

Interviewer Establishing rapport & deeper trust

Participant tells their story, with only irregular input by researcher

Winding down

Interview Timeline Notes: Unstructured interviews are marked by the strong dominance of participant in the conversation. Talking over each other and interruptions may also occur (not depicted), however the interviewer’s role is solely as an active listener.

Fig. 14.6 Graphic illustration of the typical trajectory of interchanges between interviewer and interviewee over time in an unstructured interview.

clarification (e.g., ‘what do you mean by X?’ or ‘can you clarify what you mean by that?”) or for more detail (e.g., ‘can you tell more about this?’ or ‘why do you think that happened?”) on some issue the participant has raised. Throughout the interview process, you need to be sensitive to cues from the participant that signal just how far they are willing to go in discussing any issue; the more sensitive the issues discussed, the more trust needs to be built up and maintained. If social cues suggest it is time to move on or steer away from an issue, then do so immediately. In the Indigenous research frame, the sharing of stories and experiences, perhaps throughout the participant’s life and even their ancestors’ lives, may be the goal of unstructured interviews. However, this goal will only be achievable in a postcolonial context where the participant’s history, experiences and knowledge are privileged over yours, as researcher/interviewer, and where cultural protocols are acknowledged, followed and respected. In the Feminist research frame, unstructured interviews can be used to explore the gendered life or work experiences/ histories of women (and, in some cases, men) or the sexualised/marginalised work/ life experiences/histories of LGBTIQ+ or differently-abled individuals. Unstructured interviews tend to be very long interviews (2+ hour interviews would not be that uncommon) and, in many cases, may not be completed in a single session, so this needs to be considered in your scheduling. You should also schedule (1) a substantive block of time prior to each interview to psychologically prepare yourself for the interview and (2) a substantial block of reflective debriefing time after the interview to record any retrospective observations, reflect on the interview immediately after its completion (a strategy for enhancing transparency as well as authenticity) and refresh your own energies and focus. Unstructured interviews can be extremely tiring interviews to conduct because the you have to be ‘on your toes’ for the entirety of the interview. This means that you should not plan to conduct more than 1 or 2 unstructured interviews in any one day (e.g., assuming 120-min interviews, 30 min of preparation time and 60 min of reflective debriefing time, 2 interviews in a day would mean a total of 420 min or 7 h of extremely exhausting time on task).

14.1

Strategies for Connecting with People

573

The location and environment for an unstructured interview should be quiet and comfortable, on the participant’s own turf, where possible. Unstructured interviews should be digitally recorded to enhance authenticity but only with the consent of the participant and only after they are made aware of how their recorded talk will be handled and safeguarded by the researcher. Digital recording of unstructured interviews in Indigenous contexts needs to be carefully negotiated, if allowed at all. Where digital recording is not allowed, you will need to devise other strategies for recording data as transparently and authentically as possible. Since unstructured interviews take a great deal of effort and practice to become proficient at conducting; you should allocate time, prior to commencement of data gathering, to conduct and reflect upon some trial interviews. The participant-driven nature of the unstructured interview imposes a much greater demand on you to maintain clear and detailed notes, observations and reflections on the entire interview process. In this regard, the split page research journal format becomes an indispensable resource for ensuring transparency as well as authenticity. Contextualisation of every interview is essential (including reflections on the interview process itself and how it unfolded) so that you can consider the context of the interview as part of the interpretive process. Equally, where possible, you should look for and note any contextualising commentary offered by the participant. In some cases, the participant might be invited to further clarify their discussion by providing some context for it. All such contextualisation will assist in maintaining authenticity with respect to the participant’s perspectives and helps to ensure contextual sensitivity. Like semi-structured interviews, the biggest drawback associated with unstructured interviews is the potential for your own preconceptions to inadvertently influence or bias the course of the conversation. Any hesitancy or judgmental signals on your part with respect to whatever issues the participant is discussing runs the risk of eroding trust and ruining authenticity. In the Cross-Cultural (where you and the participant come from different cultural backgrounds), Feminist or Indigenous research frames, this would include insensitive/negative reactions to what the participant is trying to communicate or insensitive or inappropriate use of language or nonverbal behaviours during the interview. The authenticity of unstructured interviews conducted within the Indigenous research frame is a delicate thing to achieve and maintain. You must always be alive to the cultural implications of what you are hearing, seeing and saying and if trust is threatened in any way, authenticity will evaporate, and your access might be revoked. In unstructured interviews, especially within the Indigenous research frame and Transdisciplinary research frame, you and the participant(s) co-create new knowledge, relationships and experiences through your interactions. There are several systems you can use to effectively support interview processes in the Depth interaction-based strategy, including: • using your research journal for recording notes throughout the interview process and reflections/observations after the fact (a tactic for enhancing transparency and authenticity);

574

14

What Data Gathering Strategies Should I Use?

• employing a digital recorder or mobile phone/tablet with a voice recording app (such as Dragon Professional, see https://www.voicerecognition.com.au/ dragon-software.htm (speech recognition software for Mac or PC), Voice Record Pro (for Apple devices), see https://itunes.apple.com/au/app/voicerecord-pro/id546983235?mt=8 or Hi-Q MP3 Voice Recorder (for Android phones), see https://play.google.com/store/apps/details?id=com.hiqrecorder. full&hl=en) for recording interviews (digital recorders help enhance authenticity, but, if you are working within the Indigenous research frame, you will need to take great care to gain permission to record and be ready for the likelihood that your request might be denied, in which case, you must rely on taking field notes in your research journal); and • harnessing mindmapping software (such as Inspiration, see http://www. inspiration.com/ or Freemind, see http://freemind.sourceforge.net/wiki/index. php/Main_Page) or Microsoft PowerPoint for setting out/displaying the topical landscape for a semi-structured interview. Bryman and Bell (2015, Chap. 20), Galletta (2013, especially Chaps. 2 and 3), Gillham (2005, Chap. 10), Flick (2014, Chap. 16), Kvale (2007), Legard, Keegan and Ward (2003), Minichiello, Aroni and Hays (2008, Chaps. 3 and 4) all provide extended and well-illustrated discussions of the semi-structured interviewing strategy. Chilisa (2012, Chap. 7) reviews semi-structured interviews in the context of Indigenous research methodology and Kovach (2009, Chap. 4) provides an excellent discussion of decolonisation with respect to Indigenous methodology. Collis and Hussey (2009, pp. 144–147), Gillham (2005, Chap. 7) and Kvale (2007) all provide useful general discussions regarding the conduct of unstructured interviews. Minichiello, Aroni and Hays (2008, Chap. 5) discuss the possibilities of assembling life histories through unstructured interviews. Kovach (2009, Chap. 5) and Chilisa (2012, Chap. 7) provide detailed discussions of Indigenous approaches to interviews and life stories and oral histories. Onwuegbuzie, Leech and Collins (2010) provide an insightful discussion that focuses on nonverbal communication data in the content of interviews and the usefulness of having a neutral third party debrief the interviewer to enhance reflection on interview process and content. Group Interaction-Based Strategy The Group interaction-based strategy is implemented when an interviewer or facilitator interviews a group of individuals simultaneously. One of the more popular forms of group interview is the focus group (see, for example, Barbour, 2007). A focus group is essentially a planned group-based interview process where one or more researchers/interviewers/facilitators engage in a group-level conversation on specific topics with a number of people collected together according to some common interest, role or shared characteristic or experience. The nature of the group interview can range from fully structured (using scripted questions) to unstructured (open group discussion) as your needs and guiding assumptions dictate (the former guided by the positivist pattern of assumptions; the latter by interpretivist/constructivist or other non-positivist pattern of guiding assumptions).

14.1

Strategies for Connecting with People

575

The twist is that it is conducted in a group setting, often to take advantage of group dynamics and synergies to enhance and enrich the conversation. Focus groups or group interviews may be useful in the Case Study, Survey, Exploratory, Descriptive, Transdisciplinary, Evaluation, Developmental Evaluation, Action or Feminist research frames. Focus groups are often used as the initial MU in a sequential exploratory MU configuration, often within an Explanatory research frame. This is a popular approach in marketing, health services, community and product/innovation development research. Group interviews (if ‘decolonised’) may be especially useful in the Indigenous research frame, where they may reflect a story-telling/story-sharing dynamic (combining with the Stories participant-centred strategy). Under the positivist pattern of guiding assumptions, focus groups tend to be combined with the Systematic observation-based strategy, Transformative data-shaping strategy and/or the Textual artefact-based strategy. Under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, focus groups or group interviews tend to be combined with other strategies that yield qualitative data such as the Visualisations or Stories participant-centred strategies, the Participant observation-based strategy or Textual artefact-based strategy. Focus groups or group interviews may involve an intact pre-existing group or a group of strangers gathered together for research purposes. If the group is pre-existing, then trust will likely be present, but viewpoints might also be more homogeneous, which may be problematic if you are seeking a diversity of views. If the group comprises strangers, the interview will need a much longer lead-in time for the group to get used to discussing things with and revealing things to each other (interpersonal rapport and trust and at least a minimal sense of being a ‘group’ needs to be built up). In all cases, you must allow time for the group to get used to your presence and role in the group (especially in a pre-existing group, where you might be seen as an interloper or viewed with some suspicion as to your agenda). In a focus group interview, under the positivist pattern of guiding assumptions, you will determine the focal topic for discussion and that topic must be relevant to or meaningful to the participants. The use of tangible objects or experiences (such as a product, innovation, multi-media presentation, policy or political event such as an election), where appropriate, can assist in achieving this goal. Figure 14.7 illustrates one possible interaction flow pattern for a focus group across time. Using a highly-structured interview protocol in a focus group under positivist assumptions will mean that every group member is asked the same questions, which means that group interaction may not be the main point of the exercise. A focus group requires a facilitator (which may be also your role, as researcher) and the facilitator needs to be carefully trained/practiced in how to communicate the guidelines for the session, how the group will be run and what it will focus on, how to manage the group, how to manage social interactions without dominating them, how to multi-task (manage the conversations, observe, record and ensure the group remains on task) and how to move the conversation on between topics. Under positivist guiding assumptions, since a more structured interview schedule and/or interview content coding protocol is more likely to be used, the facilitator’s role will be more directive.

576

14

What Data Gathering Strategies Should I Use?

Participant C Participant B Participant A Interviewer Establishing rapport

Setting context & instructions

Focus group discussion, facilitated/redirected by interviewer; can Be more or less structured with respect to Q & A

Winding group down

Interview Timeline Notes: A focus group interview is marked by turn-taking but talking over or interrupting each other (not depicted) can also occur. Group members can interact with each other as well as with the interviewer. The diagram shows that Participant C is more dominant in the exchanges than Participants A or B.

Fig. 14.7 Graphic illustration of the typical trajectory of interchanges between interviewer and interviewee over time in a focus group interview, involving multiple participants.

A group interview will tend to be of a semi-structured or unstructured nature and, in that context, group interaction will always be an important aspect of the process. Under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, since a semi-structured or unstructured interview format is used, your role is more active listener than directive. In an unstructured group interview, the topics for a group interview will tend to be free-ranging, following the natural flow of group interaction. In a semi-structured group interview, you may follow a topical landscape, exerting only subtle influences to redirect group discussion as the conversation must still be the group’s own, if authenticity is to be maintained. You will also need to be take on a participant observer role, at least in terms of observing how the group dynamics play out during the session. Group interviews in an Indigenous research frame may include cultural symbols or objects as focal discussion objects. Trust is very important to build and maintain in more general group interviews, where members don’t know each other, as, in some cases, there may be sensitivities attached to the topics being discussed. In a pre-existing group, your observations should be alive to the possibility that some group members might be marginalised or silenced, either because of the topic under discussion or because of their relationships with other group members. The location and environment for the focus group or group interview should be quiet and comfortable with comfortable chairs, ideally placed in a circular arrangement. A whiteboard or flipchart should be made available if participants are permitted to make drawings or notes. For a group interview in the Indigenous research frame, the interview setting may be a natural landscape familiar to or special to the Indigenous people you are interviewing (i.e., you will likely be interviewing them on their ‘turf’). Focus groups and group interviews should be digitally recorded (to enhance authenticity) but only with the consent of all participants and only after they are made aware of how you will handle and safeguard their recorded talk. If you plan to digitally record a focus group or group interview,

14.1

Strategies for Connecting with People

577

you will need to plan for how you will manage the transcription process so that each speaker can be clearly identified (pairing with the Transformative data-shaping strategy). Contextualisation of a group interview is essential (including your reflections on the interview process itself, the people involved in the group and how the interview unfolded) so that you can consider the context of the interview as part of the interpretive process. Equally, where possible, you should look for and note any contextualising commentary offered by participants; this will help to ensure contextual sensitivity. A focus group should be no smaller than 3–4 members and no more than 8 to 10 members (see Barbour, 2007, pp. 59–60). Group size should be decided based on the nature of the issue(s) to be discussed and the types of members to be included, but the larger the group, the more risk there will be of some members feeling talked over, marginalised or ignored. A focus group is generally more productive if run as an interactive group discussion, rather than as a question and answer exercise between you and individual group members. Full group interaction works better for building trust and for generating ideas and sharing viewpoints. If a pre-existing group is serving as a focus group, you need to understand the history and nature of the relationships between all members of the group and whether power dynamics (e.g., manager/subordinate, acquaintances/friends versus non-acquaintances, different departments) are likely to influence group interaction and willingness to discuss topics. This becomes even more important if you are conducting a more general group interview in a pre-existing group. In an Indigenous research frame, a group interview will work better if conducted from a postcolonial mindset that harnesses natural group interaction processes with appropriate group members within the Indigenous culture of interest (e.g., see Chilisa’s, 2012, discussion of a talking circle). There are several systems you can use to effectively support the group interview or focus group interviewing process, including: • using your research journal for recording notes during the interview process as well as recording impressions and observations, after the fact (a tactic for enhancing transparency and authenticity); • employing one or more digital/video recorders for recording group discussions; and • using a digital camera for taking snapshots of any writing on a whiteboard or flipchart. Barbour (2007), Edmunds (1999), Kamberelis, Dimitriadis and Welker (2018) and Stewart, Shamdasani and Rook (2007) all provide excellent discussions of the focus group approach. Bryman and Bell (2015, Chap. 21), Finch and Lewis (2003), Flick (2014, Chap. 17); Gillham (2005, Chap. 9), Minichiello, Aroni and Hays (2008, Chap. 6) all provide useful discussions and illustrations of the group interview approach. Chilisa (2012, pp. 213–217) provides discussion of Indigenous focus group interviews.

578

14

What Data Gathering Strategies Should I Use?

14.1.2 Observation-Based Strategies Observation-based strategies for data gathering involve purposeful watching and recording of activities, behaviours and interactions in a specific context. Observations may be partially or completely recorded using a specifically-designed coding protocol (pairing with the Transformative data-shaping strategy) or they may be entirely open-ended. Each strategy unfolds differently depending upon the pattern of guiding assumptions adopted. In many cases, observations occur in the participant’s natural environment (i.e., in situ), but it is also possible to gather observations in contrived settings (such as a laboratory or simulated environment, which means pairing with the Manipulative or Immersive experience-focused strategies). Observation-based strategies can be differentiated along several dimensions: • Depth of participation: Depth of participation varies along a continuum ranging from complete observer to complete participant according to the extent of your immersion and contact with participants in the observation context. Four distinct stances along this continuum can be identified (Cohen et al., 2011, p. 457): – Complete observer: You adopt an objective stance to observe and record or assess participants’ activities, behaviours and/or interactions. Your contextual immersion is minimal, and you use a highly structured observational protocol, designed either by you or another researcher. The complete observer stance is consistent with the positivist pattern of guiding assumptions. – Observer as participant: You are partially immersed in the research context as an ‘outsider’ participant but with a role defined as ‘being there to observe’. One example is ‘shadowing’, where you follow a participant around as they carry out activities. Another example is the ‘fly on the wall’ where you are in the room with the group or individuals being observed but keep your interactions with those participants to a minimum. This stance may be implemented under the guidance of the positivist pattern of assumptions or an interpretivist/constructivist or other non-positivist pattern of assumptions. Observational protocols tend to be semi-structured at best, where you look for certain things but remain open to new and unexpected occurrences. – Participant as observer: You are more deeply embedded in the research context with a role of an ‘insider’ participant member of the group who is also observing. For example, this could characterise some types of action research in a group that you join temporarily within your own organisation. It could also characterise ethnographic research or your stance in a group interview. This stance is predominantly implemented under the guidance of an interpretivist/constructivist or critical social science pattern of assumptions. Within this stance as well, observational protocols tend to be semi-structured at best, where you look for certain things but remain open to new and unexpected occurrences.

14.1

Strategies for Connecting with People

579

– Complete participant: You are completely immersed in the research context as a full group or community member where observation is simply a by-product of your participation for your ‘researcher role’ to reflect upon; it is the ultimate form of insider research. This characterises some forms of action and ethnographic research. In some cases, the group may not be aware of your other role as researcher/observer (see covert observation discussed below). This stance is only implemented under the guidance of an interpretivist/constructivist pattern of assumptions. Accordingly, your observations are very unstructured, where what you look for/at and its meaning/significance emerges rather than being pre-determined. • Observer mode: Observer mode refers to whether, in the context under observation, your role as observer/researcher is overt, covert or absent from the context. – Overt mode: In this mode, participants know that you are observing and recording their activities, behaviours and interactions. This mode of observation is ethically more defensible because participants must give their informed consent to be observed. However, the fact that people know they are being observed may alter the very phenomena you are observing, creating what are called reactive effects (something that can adversely impact on both internal and external validity). The more sensitive the activities, behaviours or interactions you are observing, the greater the risk that those phenomena may change or even disappear under observation. This can influence both the surface/manifest appearance as well as the deep or latent meaning of observations (see discussion below). Overt observation may be conducted in a way where your presence is not visible to those observed (e.g., you observe from behind a one-way mirror) as long as those observed have given their prior consent to being observed. This can help to reduce reactive effects, as long as the technology or structures used to hide your presence do not themselves create reactive effects because they serve as a reminder that you are there watching them. – Covert mode: In this mode, participants do not know that you are observing and recording their behaviours, i.e. you are operating ‘undercover’. The people observed cannot be considered as ‘participants’ per se. Thus, this mode of observation is much less ethically defensible because participants are not asked for their informed consent to be observed and recorded. For this reason, university ethics committees, for example, may not approve research proposing to employ covert observation. Covert observation does mean that you can observe genuine behaviours unaffected by your researcher role, but great risks confront you if you are discovered, i.e. if your ‘cover is blown’ and your role as researcher is revealed. In addition, depending upon what you observe and in what context (e.g., criminal, ethically or morally suspect behaviours), covert observation may create a situation that could constitute entrapment of the participant, which may have legal ramifications,

580

14

What Data Gathering Strategies Should I Use?

or you could become aware of behaviours that should be reported to authorities, creating an ethical dilemma for you. – Absent mode: Unobtrusive observation is a third observer mode which occurs when the behaviours of and interactions between people are not directly observed at all; in fact, the cooperation of participants is not required (Webb, Campbell, Schwartz, & Sechrest, 2000). Instead, you observe the contextual/ environmental impacts that result from people’s activities, behaviours and interactions. With this mode, the concept of ‘participant’ effectively disappears; what is of interest is the evidence/traces of their past presence. This mode is non-reactive because you cannot influence the patterns of behaviour and interaction you are observing. In many instances, it can be an ethically safe mode (i.e., informed consent not required) since no individuals or groups are directly observed. However, informed consent may be required in cases where the traces left by people’s activities that you analyse could potentially be linked back to individual identities (e.g., receipts, shopper dockets, security camera recordings in stores or community environments). • Focus of learning: Focus of learning considers the extent to which you are interested in or able to make sense of the activities, behaviours and interactions being observed. – Surface/manifest appearance: For observations to have meaning in terms of their surface or manifest appearance, you must be able to perceive them using your six main human senses. Aural and visual observational data are typically the most important sensory sources of data; however, tactile, proprioceptive, taste or olfactory data may also be relevant in certain types of research (e.g., in ergonomic research, product development and testing research). This is a much more descriptive level of observation, finding meaning through counting, recording, summarising and/or displaying what you observed in what context. – Deep/latent meaning: Here, you are looking to understand the deeper perhaps hidden or latent meanings of activities, behaviours and interactions, going beyond what you observed in what context to make interpretations about why they were observed and what they might mean in that context. Finding/learning about deep/latent meaning requires you to make inferences based on the data and how those inferences are arrived at depends upon the pattern of assumptions guiding your research. Under a positivist pattern of guiding assumptions, deep/latent meaning may represent constructs that are to be measured in specific (usually quantifiable) ways. Under an interpretivist/constructivist pattern of assumptions, deep/latent meanings are best understood by trying to look at the observations through the eyes and minds of the participants. Table 14.1 depicts these three dimensions in relation to each other. Each cell in the table represents a positioning on each of the three dimensions and all possible combinations are shown. In this chapter, we focus on three specific strategies for

14.1

Strategies for Connecting with People

581

observation: systematic observation, participant observation and unobtrusive observation. In Table 14.1, the dimensional combinations implicated by these strategies are coded symbolically in each cell of the table. Darker-shaded cells symbolise increased viability for a strategy to access or make sense of observations within that specific combination; lighter-shaded cells symbolise reduced viability. If a cell has no shading, this symbolises a dimensional combination where none of these three observation-based strategies make sense. Using the table, we can see that systematic observation can be conducted in either overt or covert mode, but only from a complete observer depth of participation stance; surface/manifest appearance learning is much easier and more viable to access than deep/latent meanings. Unobtrusive observation can only be implemented from a complete observer stance where the observer is absent from the context and surface/manifest appearance learning is much easier and more viable to access than deep/latent meanings. Participant observation is the most versatile of the three observation-based strategies. It can be conducted in either overt or covert mode in all stances except complete observer and access to genuine (i.e., nil reactive effects) observations and meanings is generally greater in covert mode (but with important ethical dilemmas to consider, associated with lack of informed consent to participate). In terms of pursuing deep/latent meaning in observations, the greater the depth of participation, the more viable that level of learning becomes. Table 14.1 Dimensions along which observation-based strategies can be differentiated

Focus of Learning

Depth of Participation

Complete Observer

Observer as Participant

Observer Mode

Surface/Manifest Appearance

Deep/Latent Meaning

Overt

Systematic

Systematic

Covert

Systematic

Systematic

Absent

Unobtrusive

Unobtrusive

Overt

Participant

Participant

Covert

Participant

Participant

Absent Participant as Observer

Overt

Participant

Participant

Covert

Participant

Participant

Overt

Participant

Participant

Covert

Participant

Participant

Absent Complete Participant

Absent

582

14

What Data Gathering Strategies Should I Use?

Systematic Observation-Based Strategy The systematic observation-based strategy typically involves the use of objective observers (either you or others trained in the observation task) to record activities, behavioural observations and interactions, using a paper-and-pencil or electronic recording form/instrument or technological support such as a video camera and/or hand-held data entry device. It is most closely aligned with the positivist pattern of guiding assumptions and typically yields quantitative measurements, which must be shown to be both valid and reliable. Inter-observer reliability is especially important to demonstrate when recording observations in order to reduce problems associated with observational bias. Systematic observation involves you or other observers adopting the ‘complete observer’ stance and observations can be obtained overtly, in which case participants are aware of your presence and intentions to observe their behaviours. They may be obtained overtly but hidden, where you remain hidden, perhaps behind a one-way mirror or by using a video recording camera in some context of interest but are known to be observing by participants. Finally, they may be obtained covertly, where the fact that you are making observations of their behaviours is not known to participants (e.g., using GPS tracking or a hidden observation post) or if you record behavioural observations of people from surveillance recordings obtained for other purposes, such as maintaining store, community or airport security. It is far easier to access the surface/manifest levels of behaviour (what is happening) using systematic observation than to access the deep/ latent meanings behind those behaviours (why is it happening). The systematic observation strategy is useful in the Action, Case Study, Exploratory, Descriptive, Explanatory, Evaluation or Cross-Cultural research frames. Intervention time-aligned, sequential or hierarchical MU configurations are often used for such observation research. The systematic observation strategy is always paired with either the Measurement or Transformative data-shaping strategy to focus the observational process and facilitate recording observational ratings appropriately and consistently. This creates the most challenging task in the systematic observational strategy: developing a reliable and valid scheme for recording the observations made by observers. It is important that you construct (or adopt) a recording scheme that minimises the intrusion of irrelevant and erroneous judgments on the part of the observers. This becomes even more critical if you intend to gather inferential data where the internal state of an observed person (i.e. deep/ latent meaning) will be inferred from their overt (i.e. surface/manifest) actions. For example, you might observe that person A pokes person B with a finger but wish to infer something about person A’s emotional state (e.g., anger, frustration, taking offence, joking around, making a point) rather than simply noting factual data (counting the number of times A pokes B with a finger). For this reason, systematic observation studies frequently use multiple observers observing the same people at the same time, at least in the early stages, to facilitate the assessment of inter-rater agreement or reliability of observations. You need to take additional care if your systematic observations are gathered within the Cross-Cultural research frame, as behaviours and interactions may take

14.1

Strategies for Connecting with People

Fig. 14.8 Illustrative systematic overt observational scenario in a museum gallery paired with a quasi-experimental insertion of a new exhibit (adapted from Cooksey & Loomis, 1979, Fig. 1) [showing typical exploration pathways; 0 = observer’s location; darkest thickest were most prevalent; time spent stopped at each exhibit was recorded]

583 BEFORE NEW EXHIBIT

7

6

5

4

3

O

8 AFTER NEW EXHIBIT

7

9 6

1 5

4

2

3

O

8

9

1

2

on very different meanings in different cultures. For example, you may observe one person bowing to another; in a cross-cultural context. Does this behaviour mean respect, subservience, acknowledgement of a service, an inside joke or part of a story being told? Such ambiguity in meanings can create serious construct validity problems for the cross-cultural researcher and usually implies the need to more closely attend to the wider context in which observations occur to enhance the validity of recording inferences about deep/latent meaning of behaviours. The systematic observation strategy is frequently employed in conjunction with the Manipulative, Non-manipulative or the Immersive experience-focused strategy. Sometimes, in the spirit of implementing a pluralist approach, you might employ both the systematic observation and self-report Measurement strategies and look for convergence and/or divergence between the conclusions each kind of data supports. This tends to strengthen your research, however, it does so at the cost of greater research complexity. For example, systematic observation methods are frequently employed, in conjunction with a quasi-experimental time series design, for ergonomic research and for time-and-motion or in-basket simulation studies of work processes (it is thus a favourite technique in quality management and control systems and in assessment centres), as well as in the evaluation of certain types of training programs. Figure 14.8 provides an illustration of systematic overt observation combined with a quasi-experimental manipulation in an intervention time-aligned configuration. The observational context is a museum gallery containing wildlife dioramas. The quasi-experimental manipulation involved inserting a new wildlife exhibit into the middle of the gallery (Cooksey & Loomis, 1979). Behaviours recorded by an observer, positioned on the bench to the left of the gallery, included tracing the path taken by a visitor through the gallery and the time they spent physically stopping and attending to each diorama or exhibit. Systematic observation with observer present versus observer hidden creates different issues that you must deal with. Your physical presence as an observer can

584

14

What Data Gathering Strategies Should I Use?

create potential reactive effects where behaviours change because of your presence (especially if you are seen to be writing) and you need to allow time for such effects to dissipate. You also need to take care not to interfere with behaviours (even if inadvertent). If you are a hidden observer, but participants aware they are being observed, this may reduce some reactive effects but create others, especially if a camera or one-way mirror (devices that signal the hidden presence of the observer) is visible. In such cases, you need to allow time for such effects to dissipate or hide/ screening the technology. If your role as observer role is covert where people don’t know you are observing them, this creates ethical concerns about lack of informed consent from the people being observed. If you are observing by shadowing (as when you follow a shopper around a store as they shop, or follow a manager around their workplace), then recording your observations, using pre-constructed and validated recording forms or hand-held recording device, make observing on-the-run in real-time much easier. If activities, behaviours and interactions are video recorded, coding of observations can occur while watching the recording rather than watching the phenomena live. There are several systems you can use to effectively support the systematic observation strategy, including: • using your research journal for recording notes and anomalous occurrences that emerge during the observation process; • using, where appropriate, a video or smartphone camera of some description, hand-held or locatable in the environmental context in which observations are to occur (especially important if observations are to be made later from the recorded file; • using a formal observation scoring protocol, designed or adopted and pre-tested so that its construct/content validity and inter-rater reliability in your observational context are known; and/or • using a handheld data recorder (or smartphone/tablet software) for recording observations while walking around (e.g., Noldus Pocket Observer, see http:// www.noldus.com/the-observer-xt/pocket-observer) and associated event-logging and analysis software packages (e.g., The Observer XT, see http:// www.noldus.com/human-behavior-research/products/the-observer-xt). Bryman and Bell (2015, Chap. 12), Cohen et al. (2011, pp. 459–464) and Gray (2014, Chap. 16) all discuss and illustrate the systematic observation strategy. Participant Observation-Based Strategy Participant observation is a general data gathering strategy that describes observational research occurring at various depths of researcher involvement in context. It typically involves greater depth of participation (participant as observer or complete participant) and is usually associated with an interpretivist/constructivist pattern of guiding assumptions. On occasion, it may be undertaken consistent with the positivist pattern of guiding assumptions, but only where depth or participation is minimal (observer as participant). As a data gathering strategy, participant observation is clearly distinguishable from systematic observation which distances

14.1

Strategies for Connecting with People

585

you, as far as possible, from potential involvement in the research context. Participant observation can be useful within the Case Study, Action (e.g., as participatory action research), Evaluation, Developmental Evaluation, Transdisciplinary, Feminist, Cross-Cultural, Exploratory and Explanatory research frames. Chilisa (2012) argued that participatory action research could be effectively employed in the Indigenous research frame as an approach for learning, change and transformation within Indigenous communities, where a decolonised form of participatory action research would be used involving participants as co-researchers. Participant observation is often combined with other data gathering strategies, such as the Textual artefact-based and/or Depth interaction-based strategy. It can also be effectively combined with the Non-manipulative experience-focused strategy, in cases where a group experiences change/upheaval due to an outside event or with the Manipulative experience-focused strategy in cases where a specific program is being experienced by a group (as might be the case for research conducted within the Evaluation research frame). The central feature of participant observation is your partial or complete immersion in the research context for a period of time—a characteristic of ethnographic research (see, for example, Angrosino, 2007; Jorgensen, 1989; Waddington, 2004). The deeper your level of participation, the greater your involvement in co-creating any data gathered and the greater the chance that you could inadvertently influence contextual changes. Also, the deeper your level of participation, the longer your research process will generally take, especially if you have adopted a grounded theory approach. Taking a grounded theory approach (see, e.g., Bryant & Charmaz, 2007; Charmaz, 2014) involves a process whereby repeated immersions (partial or complete) in the research context are punctuated by phases of data analytic/interpretive activity and what is learned during earlier contextual immersions at least partly shapes what you look for in later immersions. Theory emerges through this iterative process which essentially cycles its way around the ‘Data Triangle’ (recall Fig. P.1) multiple times. Maintaining extensive field notes is an indispensable tool for managing the data recording and preliminary analysis processes in participant observation. Figure 14.9 provides a visual depiction of the move between observational context and researcher field notes in a research journal. Where your participation is toward the ‘shallow’ end of the spectrum (observer as participant or being a ‘fly on the wall’), you remain somewhat at a distance from central involvement in the day-to-day activities of the groups/people being observed (which limits the risk of inadvertently influencing the context). Your participation may be short-term and limited to a single event (such as shadowing a consumer while they shop or observing a small sample of days or events like meetings), which restricts your depth of learning. Even though your participation is minimised in the observer as participant role, the group must still get used to your presence. This suggests that very early observations sessions should be treated as adaptation/habituation sessions, rather than as full data gathering sessions. Where your participation is toward the ‘deeper’ end of the spectrum (participant as observer or complete participant), you deliberately seek a closer connection to

586

14

What Data Gathering Strategies Should I Use?

R

Researcher records field notes for a meeting held in this context

Fig. 14.9 Illustration of context-to-observations recording sequence in participant observation

14.1

Strategies for Connecting with People

587

the research context and its people. Here, your participation would require a greater length of time in the research context in order to achieve a shared understanding of participants’ perspectives through observation of interactions, rites and rituals, symbols and so on (in the case of complete participation, the research moves closer to an ethnographic approach). Where you engage in a greater depth of participation in cross-cultural or Indigenous contexts, you may be exposed to or may be prevented from being exposed to behaviours, relationships, objects, rituals and/or knowledge that is special and any observations focusing on these may be considered locally or culturally privileged information, not for release by you or not for release without specific safeguards in place. The resulting gaps in your learning could influence sufficiency. Reducing this risk requires negotiation at the front end of the research project, before any data are gathered, so that all are clear on what is or is not fair game for you to gain access to and report on. If your participation is both deep and covert, you must take care to avoid role discovery (‘getting your cover blown’) or your access may be destroyed, and you may be put at grave risk, depending upon the context. Covert participant observation can be very dangerous and is very difficult to ethically justify. Furthermore, you need to plan very carefully how you will record your observations. Many participant observers record their day’s observations in their field notes at night, after hours, before going to sleep, because recording data during the day as you observe is not feasible without risking your cover. The nightly recording strategy may help to reduce memory problems but be aware that it is likely that you will remember very early and very late observations more clearly and more often (because of primacy and recency effects in human memory) as well as any surprising events. What you may not remember well are the more mundane daily occurrence events and behaviours and this can create important gaps in your observations. Some memory effects can be offset if you have access to other data sources in the context. The risk of going native is very real in participant observation research. It involves the loss of your ‘researcher attitude’ and, in some instances, distortion of your values. This risk is especially potent with complete participant contextual immersion or with the covert observer mode. Mitigating this risk requires you to take active steps such as planning for periods of time out of context and re-connecting with colleagues and peers. There is also a risk of inadvertently influencing context and changing behaviours/relationships as a consequence, which comes about if you are not effectively managing your researcher role. Managing this risk requires constant monitoring and transparent recording of your reflections on your own behaviours in context. There are several systems you can use to effectively support the participant observation strategy, including: • using your research journal for recording notes throughout the observation process as well as recording impressions and observations, after the fact—we cannot over-stress the importance of being both efficient and effective at note taking/keeping in your research journal/field notes (for the participant

588

14

What Data Gathering Strategies Should I Use?

observation strategy, the split-page format (illustrated in Fig. 14.8) for your research journal can be particularly effective); • using a digital recorder, smart phone or camera for times when it is possible, appropriate and permissible to record conversations/meetings/gatherings or for taking pictures (not really viable for covert participant observation, unless used surreptitiously); • using a smartphone can also assist with voice recording of field notes and observations. Angrosino (2007), Bryman and Bell (2015, Chap. 19), Cohen et al. (2011, pp. 459–464, 464–475), Gray (2014, Chap. 17), Jorgensen (1989), Waddington (2004) and Yin (2011, pp. 121–125) all provide excellent discussions and examples of the participant observation data gathering strategy. Unobtrusive Observation-Based Strategy Unobtrusive observation (sometimes referred to as non-reactive research; see Kellehear, 1993; Webb et al., 2000) emerged as a data gathering strategy intended to circumvent the ethical problems associated with covert observation. Data are gathered in forms not directly traceable to or directly observed as being produced by specific people. It is thus a very indirect way of connecting with people. The focus of observations is on the physical traces that people leave behind as a consequence of their behaviours and in ignorance of their identity (e.g. litter, walking pathways, products or services purchased, wastage or recycling of materials). Figure 14.10 shows two illustrative contexts for unobtrusive observation; one (a) in an art gallery where you can learn which and how long people study paintings and in what order; the other (b) a campus development context where observations may help to locate the best areas to place rubbish bins and walking pathways. The side benefit of unobtrusive observation is that you cannot influence the patterns of behaviour being observed. As a data gathering strategy, unobtrusive observation can be conducted in a manner consistent with either the positivist or interpretivist/ constructivist patterns of guiding assumptions; however, it has predominantly been associated with positivist, especially quasi-experimental, research. The unobtrusive observation strategy can work well in combination with the Participant observation-based strategy, Visualisations or Diaries/Journals participant-centred strategy as part of a focus on triangulation (see Mathison, 1988). The indirect nature of unobtrusive observations means that it would not be the first choice of strategy that you, operating under an interpretivist/constructivist pattern of guiding assumptions, would make. Rather the unobtrusive observation strategy would work better in an adjunct or supplemental role, to support other strategies such as participant observation. Unobtrusive observation may be useful in the Cross-Cultural or Indigenous research frame for gathering indirect evidence of rituals and other symbolic behaviours. However, access to observational contexts, including Indigenous land or country, must be carefully negotiated with assurances given that nothing will be disturbed or removed from those contexts (taking photos may also have to be negotiated). Unobtrusive observation is generally useful in the Explanatory,

14.1

(a)

Strategies for Connecting with People

589

Art gallery floor laid with pressure-sensitive grid (a hodometer) under carpet; [path traced and stopping times (> 5s) recorded as well as total time spent exploring the gallery] 10s

13s

13s

34s

19s 29s

36s

72s

37s

15s 22s 18s

23s

55s

36s 37s 12s

Gallery Visitor A

19s

Gallery Visitor B

16s

61s

62s

49s Total time in gallery = 647s

35s

Total time in gallery = 366s

Tracing exploration pathways and attention times in an art gallery for two visitors; artworks along the walls

(b)

Tracing walking pathways and rubbish concentrations on a new university campus

R R

R

High frequency travel route Moderate frequency travel route Low frequency travel route R = Rubbish concentration

R R

Parking areas R R

R

R

R

R

R

Fig. 14.10 Two illustrative contexts and problems for unobtrusive observation: a art gallery; b university campus

590

14

What Data Gathering Strategies Should I Use?

Descriptive, Case Study, Cross-Cultural and Exploratory research frames, especially if combined with the Manipulative or Non-manipulative experience-focused strategy or Participant observation strategy. It is best used in conjunction with other data gathering strategies in simultaneous, sequential or hierarchical MU configurations, which can make it a useful component of a pluralist approach. Some forms of unobtrusive observation can be seen as truly anonymous (such as analysing patterns of littering at various city sites, patterns of wear and tear on surfaces, patterns of waste materials around machines, patterns of stationery use in an office) in that the people leaving the traces are never observed nor can their identity ever be recovered. It is possible to conduct social quasi-experiments using unobtrusive observation as when social psychologists dropped wallets or ‘lost’ letters, containing specifically manipulated owner information (e.g., race, age, sex, social class), varying amounts of money and information about where to return the wallet if found, randomly around a city and waited to see which wallets/letters were returned and how much money remained in them (see, e.g., Farrington & Knight, 1979). In other forms of unobtrusive observation, the activity traces that you access (e.g., Facebook postings, internet browser histories, graffiti (where tags could be recognised), scanner data/shopping dockets; security video recordings in stores or in the community) may inadvertently reveal clues to or evidence of a person’s identity, whereupon ethical concerns may be raised regarding lack of informed consent from those people to use their traces as data sources. When gathering such data, the best ethical practice would be for you to seek individual informed consent to access and use trace data such as scanner purchase patterns and always deal with the resulting data in aggregate form rather than at the level of the individual. Alternatively, you could give a guarantee to people in a specific context, such as a shop or meeting room, about exactly which data are and are not being recorded when they enter that context. Inferences about human behaviour and relationships are always indirect with unobtrusive observations. Patterns may be detected, but the reasons why patterns have developed may be very hard to discern or convincingly argue for, unless other data gathering strategies are also used. Thus, surface/manifest content is easy to access; deep/latent meaning may not be. For example, if you observed a large pile of waste materials and off-cuts around certain manufacturing machines during certain work shifts compared to other shifts, should you infer that this reflects poor attitudes toward quality control, poor machine design or maintenance, poor worker attitudes, or poor working conditions more generally (thus, what was done is easily observed; why it was done is not). There are several systems you can use to effectively support an unobtrusive observation strategy, including: • using your research journal for recording notes throughout the unobtrusive observation process as well as recording decisions made about sites to record and traces to observe; • using a smartphone or camera for recording traces in environments; and

14.1

Strategies for Connecting with People

591

• using technology such as a hodometer or pressure-sensitive grid flooring (see, e.g., Bechtel, 1967) or other technology (e.g., time-lapse cameras) for recording traces of human passage through an environment (however, such technological support for unobtrusive observation may be expensive, unreliable or unavailable, which means you should have backup systems or approaches in place). Sources like Webb et al. (2000), Gray (2014, Chap. 19) and Kellehear (1993) provide useful explorations of the unobtrusive observation strategy.

14.2

Strategies for Exploring People’s Handiworks

During their lives, as individuals, as members of groups, as workers in organisations and institutions, as members of communities and societies, as members of cultures or nations, as inhabitants of environments, people produce an amazing array of handiworks. Such handiworks can have personal and social meaning, may be socially, culturally, environmentally and/or spiritually relevant, may carry implications for the future, may offer insights into and records of the past and so on. We are using the more general term ‘handiworks’ so that we can distinguish between tangible works produced by research participants and artefacts in the more general sense intended by Plowright (2011, p. 92) who defined them as “those objects or events that are produced by people”, which we would modify slightly to ‘produced and used or enjoyed by people’. Exploring handiworks produced by people anchors the second important branch in our system of data gathering strategies. We broadly classify strategies available for exploring people’s handiworks into participant-centred strategies and artefact-based strategies as shown in the expanded 3-level mindmap in Fig. 14.11. In participant-centred strategies, research participants actively create handiworks during the research process, at your invitation as researcher, or you may have created them yourself as a participant in your own research process (e.g., your research journal or field notes), based upon your prior interactions with people. In artefact-based strategies, artefacts may be completely disconnected from their creators/users or may have come into existence or use well before you become interested in them. For all but a couple of strategies, you have little or no opportunity to influence the original shape, meaning, content or use of artefacts.

14.2.1 Participant-Centred Strategies Participant-centred strategies rely on research participants to generate representations, visualisations, stories/narratives, texts and/or data records of experiences and events that provide the information needed for you to achieve an understanding of their experiences and perspectives or to undertake an evaluative test for a specific

592

14

What Data Gathering Strategies Should I Use?

Exploring People’s Handiworks

Fig. 14.11 Expanded 3-level mindmap branches focusing on strategies for exploring handiworks produced by people (encompassing participant-centred and artefact-based data gathering strategies) as well as some key considerations associated with each strategy

research question or hypothesis. Under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, you look for experiences and meanings communicated/embedded in the handiworks produced by and in the participant’s own voice/by their own hand explicitly for purposes of your research. Those handiworks may also reflect participants’ modes and patterns of thinking. Under the positivist pattern of guiding assumptions, you seek quantitative records of events and experiences to identify relevant, perhaps hypothesised, patterns and relationships. A key characteristic of almost all participant-centred strategies is that you, as researcher, give control over the data recording/generation/creation process to your participants. This represents an important shift in the researcher-researched power dynamic where data quality and quantity are placed squarely in the hands of the participants. You may provide a loose framework or some guidance on or technological support for the data generation/creation process, but the final say over form, content, consistency, quality and quantity rests with the participants. For participant-centred strategies to work, you need to invest time and effort in building up a level of trust with participants sufficient to interest and motivate them to take responsibility for their data generation/creation process and to maintain that motivation until the data recording/generation/creation process is completed. You must

14.2

Strategies for Exploring People’s Handiworks

593

be prepared to accept that not all participants will do equally well in managing their data recording/generation/creation process. For example, some people are not visual thinkers, which could make the Visualisations strategy difficult for them to implement. You should have a fall-back data gathering strategy to implement for such participants, such as an interaction-based strategy, the Stories strategy or the Structuring experience-focused strategy. The strengths of participant-centred strategies are two-fold: (1) an enhanced feeling of respect and involvement for participants engendered by the trust you place in them to manage their own data recording/generation/creation, and (2) minimal opportunity for your preconceptions to influence that process (especially important under an interpretivist/constructivist pattern of guiding assumptions). This in turn enhances participant commitment to stay the course until data gathering is completed. The weakness of participant-centred strategies is that they rely on participant commitment and diligence to manage the data recording/ generation/creation well enough that sufficient data of appropriate quality are captured. Commitment means little if poor quality data are produced. This can be especially problematic for the Diaries/Journals strategy, which may require days or weeks of consistent data recording. The Visualisations and Stories strategies are much more episodic in nature where the commitment is more related to depth and clarity of thinking than length of time. You need to acknowledge that using a participant-centred data gathering strategy, under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, may result in sacrificing elements of transparency (because participants have control over what they produce and at least partial control over how they produce it) to gain heightened authenticity (i.e., deeper and more open revelations of the participants’ perspectives in the data they produce). If employed under the positivist pattern of guiding assumptions, giving control over data production to participants may have adverse impacts on construct validity, data reliability and internal validity, because you have given away your typically tight control over such quality criteria and participants may have very different takes on what you expect them to do. Visualisations Participant-Centred Strategy The Visualisations participant-centred strategy encompasses any approach where you ask participants to produce a specific type of visual representation which can provide insights into their thinking and understanding as well as into their perspectives on an issue. Such visual representations could include sketches, drawings, cartoons, paintings, rich pictures, photographs, systems thinking diagrams, mindmaps, geographical maps, cause maps, concept maps, cognitive maps, flow charts, graphs and Venn diagrams. In some instances, participants could be invited to create some type of 3-dimensional representation, such as a clay or Lego® block model. These representations may be produced for research that is guided by virtually any pattern of assumptions. Under the positivist pattern, aspects of the visualisation may have components coded and quantified for statistical analysis (creating a pairing with the Transformative data-shaping strategy). Under interpretivist/constructivist or other non-positivist pattern, the visualisation is

594

14

What Data Gathering Strategies Should I Use?

viewed more holistically and qualitatively and the participant’s perspective that guided its production is the understanding sought. Visualisations may be produced according to a set of rules that you provide (creating a pairing with the Structuring experience-focused strategy), or free-form as determined by the participant, or some hybrid combination thereof. They may be hand-drawn, drawn using some type of computerised support system (such as a tablet and stylus) or created using materials such as Post-It Notes®, a whiteboard or other physical materials/tools. The goal is for the participant to produce a visualisation that provides a meaningful representation of how they think/feel about some object/problem/process/event of interest. The Visualisations strategy can be useful in virtually any research frame as an alternative or a supplement to interviews. For example; it can be especially useful in the Exploratory, Transdisciplinary, Cross-Cultural or Indigenous research frame as a way of gaining insights into concepts that may not be familiar or meaningful to you. They may also provide a vehicle for supporting the Stories participant-centred strategy. The Visualisations strategy may be used in a single MU configuration or as part of a multiple simultaneous, sequential or hierarchical MU configuration, in a pluralist investigation. Visualisations may be requested with some specific target object/issue/process/event in mind, usually at your invitation as researcher. They can provide insights into a participant’s mental model of or perspective on some issue (see Eppler, 2006, for a comparison of visualisation techniques). Under an interpretivist/constructivist pattern of guiding assumptions, the role of a visualisation is to provide the participant with a platform for representing/sharing meanings. The Visualisation strategy can be combined with a Structuring experience-focused strategy such as process tracing via think-aloud protocol (verbalising what is being done throughout the process of producing the visualisation), the focused interaction-based strategy (discussing the visualisation) or the Story participant-centred strategy (storytelling to accompany/explain the visualisation, which could be especially relevant in the Indigenous research frame). It could also be combined with the Group interaction-based strategy to produce group-level visualisations. There is a wide range of rule-guided visualisations that can be produced by participants, often benefitting from pairing with the Structuring experience-focused strategy and/or an interaction-based strategy. • A rich picture is a form of visual representation most commonly associated with soft systems methodology, intended to display a participant’s thinking about how things (e.g. people, events) are interconnected in the context of a specific problem. They can often serve an action learning function and are typically drawn free-hand (see, for example, Checkland & Poulter, 2006). • A systems thinking diagram is a rule-driven approach for depicting the perceived dynamic interrelationships between different aspects or components of organisational systems, showing how stocks, flows and feedback loops appear to work from the participant’s perspective (see, for example, Anderson, 1997; Pidd, 2009; Walsh & Clegg, 2004). Note that, technically speaking, a rich picture is one form of systems thinking diagram.

14.2

Strategies for Exploring People’s Handiworks

595

Interpretivist assumptions Cause Map constructed by participant using colour-coded Post-It Notes and cloth links on a large sheet of paper Fig. 14.12 A cause map visualisation produced by a participant (Sandall, 2006, p. 160)

• A flow chart is a simple linear visual representation of system processes, event or problem-solving sequences, which, when generated by participants, can yield insights into how various problem-solving processes unfold through time. • A cause map provides a representation of a person’s thinking about concepts, events and issues, and how he or she perceives them to be causally interlinked; non-causal links (e.g. associations) can also be displayed in cause maps. Technically, a cause map is a specific type of cognitive map, but it has received recent increasing attention as a data gathering approach (see, for example, Eden & Ackermann, 2002; Hine, Montiel, Cooksey, & Lewko, 2005; Laukkanen, 1998). Figure 14.12 provides an example of a physical cause map produced, using colour-coded Post-It Notes®, felt cloth links and a large piece of paper, by a participant during a focused interview about implementing a new state policy on conservation of native vegetation (Sandall, 2006, p. 160). • A graph can be used as a data gathering device for certain types of investigations (almost always positivist in assumptions) where a participant’s understanding of or preference for functional relationships between variables or measurements is important to display (e.g. investigations of management perceptions of system dynamics and concept relationships, see Behaviour over Time graphs in Anderson, 1997, or specifying judgment policies in decision research, see Cooksey, 1996, pp. 265–270). Quantitative graphs or sketches are

596

14

What Data Gathering Strategies Should I Use?

often paired with the Immersive experience-focused strategy as a way of gaining insights into participant’s perceptions of how specific variables in a system simulation might behave over time. • A Venn diagram is another specific type of graph that participants could use to represent probabilistic relationships or conceptual overlaps between sets of objects or ideas. Visualisations may also be free-form in nature or partly rule-guided and partly free-form and would include mindmaps, concept maps, sketches, drawings, cartoons, paintings and 3-D objects that the participant creates. • A sketch, drawing, cartoon, painting, geographic map or 3-D object is generally free-form in nature (i.e. where you do not constrain the participant with imposed structural or compositional rules). They are most closely associated with phenomenological research, where the participant’s perspective is reflected in the overall meaningfulness, composition, structure and level of detail in the representation itself (see, e.g., Guillemin, 2004; Stiles, 2004). • A cognitive or concept map represents a person’s thinking or understanding about concepts, events and issues, and how he or she perceives them to be interrelated and can provide insights into a person’s ‘mental model’ of some aspect of knowledge, or personal, organisational or social life. One important feature of many cognitive or concept maps is the use of relational links between concepts, which can be logical, implicative, qualifying, constraining, temporal or causal in nature, and, in the map, are frequently anchored by a short verbal phrase that conveys the meaning of the relational link (see, e.g., McDonald, Daniels, & Harris, 2004; Kane & Trochim, 2007). • A mind map is a simple diagrammatic method for producing a nonlinear shorthand display of a person’s knowledge, understanding, or perceptions around an issue or idea (see, e.g., Buzan, 2003, 2018; Meier, 2007). This can be a useful visualisation for capturing complex ideas and thinking. Patterns of guiding assumptions influence whether and how you might ask a participant to produce a visualisation and what features you would like them to include. A participant in research guided by the positivist pattern of assumptions may be asked to produce a cause map visualisation that not only qualitatively labels all causes or concepts they include, but also quantifies aspects of their map, such as the relative strengths of various causal or conceptual links (see, for example, Laukkanen, 1998). A participant in research guided by an interpretivist/ constructivist pattern of assumptions might be given little or no guidance as to what to include in their visualisation or they may be asked to include certain features to help display their perspective. Individual or group-based visualisations can be produced by participants, depending upon your research purposes. Group-based visualisations, for example, may be created on whiteboards or flipcharts within a large meeting room. For many research participants, creating a visualisation as part of data gathering might make their participation more meaningful as well as more interesting and

14.2

Strategies for Exploring People’s Handiworks

597

involving. For effective data gathering and visualisation production, you should provide or secure a relaxed, comfortable and private location, with good lighting and free from risk of interruptions or distractions, with ample space and with all materials/support systems (e.g. pens, paper, large sheets of paper, graph paper, flip charts, whiteboard and markers, ruler, Post-It Notes® of various sizes and colours, Blu-Tack®, materials such as felt ribbon to serve as moveable links and arrows, scissors for cutting shapes and link lengths, laptop/tablet with relevant software/ apps pre-loaded) they might need for working on their visualisation. If group-based visualisations are to be created, comfortable seating and tables should be provided. You should define any specific visualisation rules, characteristics and materials for participants to use beforehand, as well as level of detail or quantified features expected in any visualisation labels, objects and links. If you use a software support system (e.g., Inspiration for diagramming or mindmapping, or a sketching app) or other technology, you may need to provide training and/or ensure prior experience on the part of participants and this would need to be factored into the timeline of your research. Some people will likely struggle with or prefer not to work with technology, so a back-up visualisation production medium should be put in place for these people. When asking participants to produce any type of visualisation, you need to think through your expectations and necessary instructions and rules carefully, ensuring they are clear and unambiguous, otherwise the task may risk becoming onerous, frustrating and self-defeating for some participants. In some cases, providing a hypothetical illustration or example of the kind of visualisation you are after could be useful, but note that this could inadvertently constrain the participant’s thinking. You should not expect everyone to cope well with producing a visualisation because some people are not visual thinkers or skilled in creating a what amounts to a piece of art. Many people are verbal thinkers and may therefore struggle with your request to produce a visualisation; people who lack visualisation/drawing/mapping skills or are self-conscious about their skills may actively refuse to produce a visualisation for you. Accordingly, you should formulate a fall-back data gathering strategy (such as a think-aloud process-tracing protocol or a focused interview) to accommodate such participants. For all participants, a good tactic is to encourage the participant to think aloud and verbalise as they generate the visualisation and record what they say digitally or in your research journal so that later ambiguities in their visualisation may be cleared up. Handwriting, smudging, scratch-outs and other error-corrections and lack of drawing neatness or accuracy are potential problems with participant-generated visualisations, so you need to be ready to ask for clarification where needed before your session with the participant concludes. Pilot testing or trialling your intended visualisation data gathering process is very useful for checking that everything required to produce the required visualisations is available and can be worked with. You also need to plan an image capturing method for the visualisations that are produced (unless they are automatically captured in electronic form by a software package) and ensure that the capture method works and produces a readable/ interpretable permanent image that can be worked with. This will ensure that you

598

14

What Data Gathering Strategies Should I Use?

will be able to import the visualisation images into a qualitative data analysis support software system, such as MAXQDA, NVivo or dedoose. For example, if a participant draws a cause map on a whiteboard, you either must use a printing whiteboard (which may sacrifice any colour-based aspects of the representation) or a camera/smartphone to capture the image. Visualisations produced using physical materials on a large piece of paper can be kept in original form or, for analysis purposes, you can redraw or photograph them (refer to the Post-It Notes® example shown in Fig. 14.12) for electronic storage of the image. There are several systems you can use to effectively support the Visualisations strategy, including: • using your research journal for recording notes and observations you make throughout the visualisation production process; • using mindmapping/diagramming software (e.g., Inspiration, see; http://www. inspiration.com/; Freemind, see http://freemind.sourceforge.net/wiki/index.php/ Main_Page); • using tablet drawing apps, especially where a stylus can be used (such as S Pen or SketchBook for Android; Procreate or Paper by FiftyThree for iPad); • using cause mapping software (e.g., Decision Explorer, see http://banxia.com/ dexplore/); and/or • using a camera/smartphone/scanner for taking photos/scanning images of visualisations produced by participants, if not automatically stored in electronic form. Buzan (2003, 2018) and Meier (2007) provide excellent discussions and illustrations of mindmappping. Eden and Ackermann (2002), Hine et al. (2005) and Laukkanen (1998) provide discussions and examples of cause mapping and inclusion of quantitative aspects. Guillemin (2004), Margolis and Pauwels (2011, Chap. 12) and Stiles (2004) all discuss the use of sketches and drawings as data gathering approaches. Anderson (1997), Checkland and Poulter (2006), Pidd (2009), Walsh and Clegg (2004) discuss systems thinking and soft systems and rich pictures approaches. McDonald, Daniels and Harris (2004) and Kane and Trochim (2007) discuss concept/cognitive mapping approaches and Eppler (2006) provides a useful comparison of mapping and drawing techniques. Stories Participant-Centred Strategy Storytelling and the creation of narratives are natural modes of communication between people and form frequently informal ways of sharing culture, history, beliefs and values, information, emotions, fears, hopes and so on. Furthermore, storytelling and narratives may follow different genres, such as autoethnography, metaphors, poetry, fables and may accompany performances, such as dance, theatre and film. In many cultures, including Indigenous cultures, oral histories are handed down from generation to generation, often in the form of stories. Such stories embody important cultural/spiritual/historical knowledge and can serve as an “instruments of socialization” between generations (Chilisa, 2012, p. 138). We can see the same things occur in organisational and institutional stories. Some stories can

14.2

Strategies for Exploring People’s Handiworks

599

take the form of metaphors, which provide a device for people to reveal their thinking in a more meaningful and perhaps less personally threatening way. They are typically conveyed in a rich, extended, coherent and densely interconnected form. Stories and narratives can provide vital data for achieving a deeper understanding of specific contexts and people, hence their intrinsic value as data for social and behavioural research. The Stories strategy is used predominantly under the guidance of an interpretivist/constructivist, critical social science or Indigenous/ feminist pattern of assumptions. There can be a political dimension associated with storytelling and narratives under these specific patterns of assumptions when used to gather important qualitative life stories/histories (see Chase, 2005; Elliott, 2005). However, the Stories strategy can occasionally be used under the positivist pattern of guiding assumptions, in which case the narrative focus is on retrospective event histories that have a quantitative orientation (i.e., focusing on the timing, content and causality of events, see Elliott, 2005, Chaps. 4 and 5). The Stories strategy may be useful in the Case Study, Action, Exploratory, Explanatory, Developmental Evaluation, Descriptive, Transdisciplinary, Cross-Cultural, Indigenous or Feminist research frame as a pathway for giving participants a safe space to convey their perspectives. It can often work best if used in conjunction with another data gathering strategy in a pluralist approach, say in a multiple simultaneous MU or sequential MU configuration. It can also be used quite effectively in a case-based MU configuration. Stories and narratives may be generated explicitly for purposes of the research (e.g., see Czarniawska, 2004; Elliott, 2005; Gabriel & Griffiths, 2004) or specifically sought in data that participants have already provided for the researcher or in contexts where story sharing can be observed (thus combining the Stories strategy with the Participant observation-based strategy, for example). When dealt with in a culturally appropriate and sensitive manner, under Indigenous guiding assumptions in an Indigenous research frame, you can observe, solicit and analyse the stories, narratives, metaphors and performances that people generate and share about their culture, history, lives, organisation, group, beliefs, important objects and symbols, other people or events. In a cross-cultural or Indigenous context, you may need to follow up a storytelling session with participants with a focused interview about what they have created. This may be required because, if you are not a member of their culture, they may make references to people, objects and events in their stories that you do not understand. Narratives typically have a sequential flow, connecting events, thoughts, feelings, memories and the like into a coherent discourse that explains or accounts for those experiences (i.e., facilitating sense-making in a specific context). Storytelling is a more informal type of narrative that can harness images, employ metaphors, involve characters and plot trajectories and use other linguistic devices to help add texture and meaning to what is being conveyed. Stories may have specific target audiences in mind as they are being conveyed and may be intended to create a specific learning or emotional outcome for the listener such as a moral, a social norm/expectation or empathy/enmity for a situation or character (see, e.g., Boje, 1991; Gabriel & Griffiths, 2004). Storytelling in Indigenous and other contexts

600

14

What Data Gathering Strategies Should I Use?

where people have been traditionally marginalised frequently adopts a critical stance seeking to counter the majority-dominated stories that typically emerge. This has led to the process of counter-storytelling, where the stories of marginalised groups, not typically represented in majority society and associated with issues of race and culture, are brought into the foreground with high relief to counter the majority stories (e.g., Chilisa, 2012, pp. 138–157; Kovach, 2009, Chap. 5; Solórzano & Yosso, 2002). Such stories may not only be shared with you, as researcher, they may also serve socialising and social commentary functions within their own communities and outward into other communities. Soliciting stories and narratives is not a question-answer conversational type of data gathering approach. It is a much more of a one-sided affair, where the participant is given the power to tell their story in whatever fashion they wish. Typically, your role is to get things kick-started by inviting the participant to share a story about their life, workplace, a person or persons (including themselves) or a specific event. If a specific event, series of events or person(s) is to be the focus of the story, you may suggest this focus in your invitation. As researcher, you must limit your interference with or prompting of the unfolding of a narrative or telling of a story once it has commenced. Asking questions during storytelling or narrative creation can disrupt the participant’s flow of thinking, which can lead to distortions of originally-intended meanings. It is important to realise that stories may represent or contain privileged or culturally sensitive knowledge, especially in Indigenous cultures, and that this will need to be explicitly and sensitively negotiated and handled. Counter-storytelling may be particularly confronting for you, if you are not a member of the marginalised group from which the counter-story is emerging (i.e., when you are a member of the ‘majority’) and you must be able to manage the listening process without judgmental or defensive reactions. Gathering retrospective life histories or event histories may suffer from recall bias and other memory distortions, with the risk increasing the further back in time the participant reaches. Having access to other corroborating sources of data can assist in managing this problem and, where appropriate and available, a physical reminder of a specific event or point in time (e.g., a photograph, report card, newspaper article) may also stimulate recall. Narratives or stories may be invited or sought in written or oral form or as a performance and may or may not be accompanied by tangible objects and symbols. Many stories/narratives capture elements of life in the form of metaphors that create an anchoring image for the person’s story, such as a machine, a river, a sports team, an island, a brain, unity, an organism or animal. Inviting participants to create and elaborate on an appropriate metaphor for an aspect of their life or work (perhaps illustrating their metaphor with a sketch or drawing) may provide a safer way for participants to convey meaning for difficult, perhaps, sensitive subjects. If you seek a story in the form of a metaphor, then this request can form part of your initial invitation to participate. Participants may need some clarification as to what you are seeking, as they may not have thought of their work or life in this manner before (doing this effectively pairs the Stories strategy with the Structuring experience-focused strategy). In instances where you have not invited the creation

14.2

Strategies for Exploring People’s Handiworks

601

of a narrative or story but instead look for them in the context of other data being gathered or seek them from other social contexts entirely, you will need to depend heavily upon field notes in your research journal (and, where available, a digital/ video recording) to capture the essentials of the narrative or story for later analysis. Your goal would be to achieve as close to a verbatim record as possible, so writing the notes as soon as possible after encountering the narrative or story would be paramount to maximise accuracy/authenticity. Another approach to the Stories strategy is called the ethogenic approach (e.g., Cohen et al., 2011, Chap. 22), which looks at how people account for events they experience and/or actions they take in the world. The language used to construct such accounts provides insights into personal and social meanings. Accounts are typically elicited through a focus on social episodes or samples of experiences, which stimulate a kind of focused storytelling (you often stimulate the generation of the account from a participant using a stimulus phrase that the participant then builds their account from (e.g., “Tell me about when you had to make a difficult choice in your life …”). Autoethnography is a recently developed, very specific, type of self-reflective and highly analytical storytelling, where you become your own source of data in your own context (see, e.g., Anderson, 2006; Chang, 2016; Duncan, 2004; Jones, 2005). It is a narrative of the self and has emerged as a specific way of presenting one’s inner knowledge; it is the ultimate in ‘insider’ research. The autoethnographic approach requires a very high degree of openness, transparency and authenticity (uninfluenced by the need to create a false or self-serving image of oneself) and critical self-reflection. The (very difficult) trick is to sensitively maintain a balance between your researcher attitude (involving some degree of contextual distance) and your subjective experiences (on which you are reflecting). The interaction of memory with length of retrospective timeline also needs to be managed through assiduous and consistent attention to recording your own journal (thereby combining the Stories strategy with the Diaries/Journals strategy). The Stories strategy works well if combined with the participant observation (especially where story sharing between people in context is observed and recorded), the Diaries/Journals strategy, the Depth interaction-based strategy, Textual and/or Multi-media artefact-based strategies. In such pairings, the Stories strategy may take a dominant role or a supporting role. The autoethnographic approach is nearly always combined with the Diaries/Journals strategy where the writer/ researcher takes the role of the ‘self’ and the diary/journal is inward-looking as well as outward-focused because you are recording data on yourself in context. There are several systems you can use to effectively support the Stories strategy, including: • using your research journal for recording notes throughout the narrative/ storytelling process as well as recording your impressions and observations, after the fact; • using a digital recorder, camera, smartphone or video camera, if permitted, for capturing elements of stories as images/performances; and

602

14

What Data Gathering Strategies Should I Use?

• using icons, images, photos, props, symbols, or other tangible objects as prompts to support narrative creation and storytelling (in an Indigenous context, you must take care to negotiate the use of such objects, if they are to be used at your initiative). Bryman and Bell (2015, Chap. 22), Chase (2005), Clandinin (2007), Cohen et al. (2011, Chaps. 22 and 31); Czarniawska (2004), Elliott (2005, Chap. 2) all focus on the creation and use of narratives and accounts. Gabriel and Griffiths (2004), Kovach (2009, Chap. 5) and Solórzano and Yosso (2002) all focus on storytelling. Schmitt (2005) explicitly discusses the use of metaphors. Anderson (2006), Chang (2016), Duncan (2004) and Jones (2005) discuss the autoethnographic approach to narrative creation. Cohen et al. (2011, Chap. 31), Dick (2004), Rapley (2007) and Samra-Fredericks (2004) offer discussions focused on discourse and conversation analysis. Diaries/Journals Participant-Centred Strategy This participant-centred strategy encompasses diaries, journals and other self-monitoring and self-recording techniques where textual or multi-media data are generated by research participants acting, often out of your sight and hearing, as a quasi-researcher (what Charmaz, 2014, p. 47, referred to as ‘elicited texts”; see also Symon, 2004). The strategy may rely on the support of technology such as a laptop or tablet computer (for building blogs on the internet, managing a Facebook timeline, recording events against a calendar page or timetable or keeping an electronic version of a diary), voice recorder, smartphone or video camera, or may simply rely on written records kept by the participant according to some agreed upon schedule. Such strategies typically generate qualitative data and are entirely dependent upon the diligence of the participant in adding to and keeping the record. It is possible, however, to obtain quantitative data (e.g. time spent doing specific tasks, number of tasks undertaken, various types of health measurements) using the diary or journal method, which typically results in a log book that can take handwritten or electronic form. In some cases, the role of ‘self’ is taken by you, the researcher (as a ‘participant’ in your own research project) as when you maintain a personally contextualised and reflective research journal that can later be used as an additional source of data for contextualising decisions, choices, observations and reflections throughout the research journey or to help construct an autoethnography (pairing with a Stories participant-centred strategy and, perhaps, the Textual artefact-based strategy). The Diaries/Journals strategy can be implemented either under the positivist (especially where participants record quantitative measurements) or under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions (where participants record qualitative comments, narratives and/or stories). It can be used effectively within the Action, Case Study, Evaluation, Developmental Evaluation, Exploratory or Explanatory research frame, especially where tracking or mapping development and change over time or before, during and after experiencing an event, program, innovation or treatment, is a research goal. The longitudinal/chronological nature of dairies/journals means that longitudinal MU

14.2

Strategies for Exploring People’s Handiworks

603

configurations and hybrid longitudinal MU configurations are well-suited for this data gathering strategy. If there is an event, program, innovation or treatment implicated in the development/change process, then this strategy could be used in conjunction with experience-focused strategies, such as Manipulative, Non-manipulative or Immersive strategies. The Diaries/Journals strategy works synergistically with the Stories and/or Visualisations participant-centred strategies. Researcher insights and understanding of participants’ diaries/journals can be enhanced by combining their use with post-diary collection interviews. It can be an effective strategy in combination with the Measurements and/or Transformative data-shaping strategies (e.g., for recording tasks and activity characteristics, physiological data, such as heart rate, steps and distances walked and/or logging answers to standardised questions about activities, emotional states, feelings of stress, physiological measurements). This log book may be recorded manually by the participants at scheduled times or may be recorded automatically, if appropriate technological support is available. Diaries/journals can be valuable sources of qualitative interpretive data for understanding what participants think/do/feel/experience during a specific time period. Typically, you would initiate the self-recording activity via a formal request to intended participants to keep their own diary/journal of thoughts and activities for a specific period. Diary/journal keeping can be left open to participants as to what is included and in what form and detail or you can provide some guidance as to what things you are interested in seeing participants include in their record (perhaps encouraging them to incorporate photos, pictures, sketches, stories and/or mind maps into their records, especially if they find these to be natural devices to use; encouraging creativity should also help to reinforce commitment to consistently making entries). Some participants may be natural diarists or journal keepers and you may wish to request access to their writings (which will have pre-dated your interests, thus pairing with the Textual artefact-based strategy). Their diaries/ journals could then provide qualitative data to be analysed. However, you would have to be open and adaptive in your expectations, being willing to accept the content and formatting of records as given. This approach requires closer attention to a sampling strategy for deciding whose diaries/journals you want or need to access. When you maintain a research journal or set of field notes, you are taking the role of ‘self’ in ‘self-recorded’, as a participant in your own research journey. A research journal forms a generally useful source of information providing contextual background for different aspects of your research journey, irrespective of the pattern(s) of guiding assumptions you have adopted. Field notes are more commonly assembled in conjunction with other strategies, such as participant observation, under an interpretivist/constructivist pattern of guiding assumptions. When writing a research journal or field notes, you are actually creating data; data that can analysed and interpreted in their own right as well as in the context of other types of data, perhaps gathered using other strategies. Where your journal or field notes are in handwritten form, they may need to be transcribed into an electronic form for analysis (thus creating a synergy with the Transformative data-shaping strategy and

604

14

What Data Gathering Strategies Should I Use?

the Textual artefact-based strategy). Your research journal can also be critical for you to call upon if you are writing an autoethnography, providing contextual, reflective and personal details against a timeline. Using the research journal for this purpose shifts the focus from outward (understanding and making sense of what others do and experience) to inward (understanding self, as you experience and make sense of your own research journey). You should realise that participants will vary in their capabilities and motivations for maintaining a diary or journal. For some, this will not be a natural activity and, if so, participants should be given space for learning and developing their capabilities, which means you should downplay focusing on their earliest recorded entries. This data gathering strategy is highly dependent upon participants’ motivation and commitment to cooperate (including their willingness to reveal diary contents to you), their diligence in attention to the task, and their care and reliability in the diary recording activity. You need to take care to ensure that no incentives to falsify or impression manage their diary/journal entries are created. Effective safeguards and assurances, design of the recording medium, instructions/ expectations can help in this regard as can the provision of external rewards once their diary has been completed and submitted to you. Maintaining participant commitment without forcing (which may lead to inauthentic or incomplete entries) is critical and can be enhanced through periodic contact between you and participant. Participants need to know how much time they need to commit to maintaining their diary/journal before they return it to you and they need to be clear as to the schedule for completing diary entries (e.g., daily, weekly, time of day). You also need to make any expectations you have with respect to depth and level of detail of recorded information, including contextualisation of that information, clear. Missing data (e.g., missing entries) is always a risk, due to forgetfulness or lack of commitment on the participant’s part or lack of clarity of instructions/expectations on your part. If your research involves participants keeping a quantitative log book, accuracy will be a necessary requirement, as the contents of the record will eventually become the source of measurements for statistical comparisons. You may need to provide participants with specific equipment or instruments—such as stop watches, pulse rate meters or rating scales—to facilitate their log book entry activities. The quantitative log book variant of the Diaries/Journals strategy is not a strong stand-alone method but can be a useful initial step in exploratory investigations. Some training of participants may be required where quantitative measurements are to be gathered, especially in relation to which specific events are to be recorded, how to record them, what contextual details, if any, they should record and what units of measurement and levels of precision are required in entries. If wearable technology or diary-keeping software is to be used, participants will need to be trained in their use (and perhaps basic maintenance, such as changing or charging batteries, calibrating or resetting the device). Diary/journal design (see Symon, 2004), whether in hardcopy or electronic format, is something you should attend to as effective design will provide ease of use, lack of frustration and enough room to record information/write stories and

14.2

Strategies for Exploring People’s Handiworks

605

details/make drawings. Involving participants in the choice and design of their diary/journal, where qualitative entries are to be provided, could be a useful tactic. If quantitative data are to be recorded, the log book should provide clear areas to record measurements in the proper units. As researcher, you are ethically responsible for safeguarding who has access to the contents of a participant’s diary/journal while it is in your possession and what uses the information will be put to. This needs to be clarified up front before commencing your research. If you plan to access pre-existing diaries/journals (i.e., that pre-date your research), you need to carefully negotiate with the writer about who will access the diary/journal, how its contents/information will be protected (as well as the identity of the writer) and how you will access its contents. There are several systems you can use to effectively support the Diaries/Journals strategy, including: • using your research journal for recording notes throughout the Diaries/Journals setup and management processes as well as recording impressions, observations and participant feedback; • using software/hardware support and apps, such as: Livescribe (pen and special paper for taking notes/recording talk for automatic electronic storage), see https://www.livescribe.com/au/; EverNote (for recording notes, images, video, document links; Windows, Mac and Android platforms), see https://evernote. com/; My Personal Diary (for Windows), see https://www.camdevelopment. com/my-personal-diary/; LifeJournal (Online and Windows versions); see http:// www.lifejournal.com/; and/or • in the health area, various types of physiological monitoring or recording devices/apps can be useful (e.g., heart monitors, diet, exercise or activity monitors (e.g., Samsung’s S Health app, see http://www.samsung.com/au/apps/ mobile/s-health/; FitBits or other wearable activity-monitoring devices). Collis and Hussey (2009, p. 152–153), Elliott (1997) and Symon (2004) discuss participant-recorded diaries/journals. Emerson, Fretz and Shaw (2011), Montgomery and Bailey (2007) and Ortlipp (2008) all discuss issues surrounding the maintenance and use of reflective research diaries/journals/field notes/memos. Jones (2005) discusses the use of researcher diaries/journals in the context of writing an autoethnography.

14.2.2 Artefact-Based Strategies Artefact-based strategies focus on handiworks of potentially any form that may have been produced for other purposes at other times and places and by other people, as well as on artefacts generated specifically by you, as researcher. Here, the widest possible view of what counts as an artefact is taken, consistent with Plowright’s (2011) view. Plowright (2011, p. 92) indicated that artefacts may be

606

14

What Data Gathering Strategies Should I Use?

text-based (e.g., books, emails, letters, newspapers, web pages, magazines, brochures, leaflets, reports, blogs and other kinds of technologically-transmitted text, such as SMS and Twitter messages and Facebook comments, transcripts of interviews) or may reflect other types of sensory experiences and combinations of sensory experiences (e.g., visual and sound; visual, sound and kinaesthetic/spatial): • visual (e.g., photos, videos, YouTube clips, maps, illustrations and drawings, artworks, paintings, pottery, clothes, buildings and other architectural manifestations); • sound (e.g., stories, oral histories, music, television, films, radio, recorded or live streamed content); • kinesthetic/spatial (e.g., theatre, dance, plays, operas, concerts, gigs and community-based busking, treatments for improving health and well-being) • smell (e.g., food, drink, herbs and spices, medicines); and • taste (e.g., food, drink, herbs and spices, medicines). Any artefact can potentially serve as (non-human) data sources, without necessarily needing to connect with their creators or users (who may, in fact, no longer be amongst the living). Plowright (2011) stated that artefacts serve different purposes: carry/store information, present something, represent something and/or interpret something. Your task is to decide which artefacts, if any, may relevant to pursue to advance your learning with respect to your research questions, access those artefacts and then work to understand their purpose(s), their meanings and/or their significance relative to your research context(s). Under an interpretivist/constructivist pattern of guiding assumptions, you look for meanings communicated/embedded in sampled artefacts or in artefacts you yourself have produced. In some cases, your goal is to understand the artefact creator’s intentions and perspectives (looking inward at the artefact) and in other cases, your goal is to understand the perspectives on/meanings for the contexts and lives of people to which the artefact is relevant (looking outward from the artefact). Under the positivist pattern of guiding assumptions, you typically seek to quantify characteristics of sampled artefacts in order to identify relevant, perhaps hypothesised, patterns and relationships. There is an important bifurcation in the artefact-based strategies predicated on whether you have had input into the creation of the artefacts produced, gathered and analysed. For certain types of textual evidence that you may focus on, you are the creator of the artefacts, either in terms of writing a research journal or amassing field notes (pairing with the Stories or Diaries/Journals participant-centred strategy) or transforming recorded interviews or the audio content of other multi-media documents into transcripts or sets of quantitative indicators (pairing with the Transformative data-shaping strategy). For these kinds of textual evidence, you determine what is recorded, in what format and how detailed. You therefore form the central connection between the artefacts and their contents and meanings. Under an interpretivist/constructivist pattern of guiding assumptions, you may create all the qualitative text data that you will analyse. For your research journal or field

14.2

Strategies for Exploring People’s Handiworks

607

notes, transparency relates to the level and clarity of detail included in the artefacts and authenticity relates to how well the artefact captures what you intended to say or represent. Sufficiency resides in the completeness of the artefact content as it relates to the stories being conveyed as well as in the range of artefacts produced. Under the positivist pattern of guiding assumptions, you create the coding system that will be used to transform your data into artefacts containing quantitative indicators. Construct validity resides in how well the artefact content maps onto the constructs being measured and internal validity resides in your capacity to reduce or eliminate contaminating influences on that content (processes typically associated with ‘content analysis’, see, for example, Krippendorf 2004; Weber 1990). External validity is influenced by how well you maintain a consistent and unbiased focus on recording content and then extracting meaning from it across a range of situations (or interviewees, in the case of interview transcripts). For all other artefact-based strategies, other people typically produce and use or enjoy the artefacts and you typically have little or no opportunity to influence their content, production process or use. Their creation often predates the commencement of your research journey, serving purposes established by others; their existence must be taken as given. Such artefacts provide a ‘clean’ source of data, but one where the meanings, purposes and intentions of the author/producer/user can only be indirectly inferred from the artefact itself, unless direct access to the author/ creator/producer/user becomes possible. In these instances, all you can do is decide which artefacts will be sampled, how they will be accessed and how they will be used for research purposes. Sampling, i.e., choice, of artefacts to focus on, thus becomes a very important consideration and unfolding this choice process becomes a very critical aspect of sufficiency (under an interpretivist/constructivist pattern of guiding assumptions) or external validity (under the positivist pattern of guiding assumptions). When others produce the artefacts that you wish to access/study, it must be recognised that your task is to discover the intersections between each artefact and the research stories they are relevant to. This may require reading between the lines and drawing inferences about relevance and meaning that go beyond the original purposes of the artefact while, at the same time, revealing something about the context(s) in which the artefact emerged. Under an interpretivist/constructivist pattern of guiding assumptions, this often means using the artefact content to try to get inside the artefact creator’s head to glimpse the meanings they originally intended or meanings they are reflecting. Transparency is carried in the extent to which you are clear about how artefact content is accessed and used. Authenticity resides in how closely your inferences and interpretations align with the contents and purposes of each artefact and sufficiency relates to the relevance and diversity of artefacts chosen. Under the positivist pattern of guiding assumptions, construct validity resides in how well the information provided in each artefact maps onto specific measurement constructs that you intend to use. Internal validity resides in how well you can nullify arguments about alternative possible explanations for why the artefact content appeared as it did. External validity is carried in the quality of

608

14

What Data Gathering Strategies Should I Use?

the sampling process for choosing artefacts (or relevant portions of artefacts) to examine. Textual Artefact-Based Strategy The Textual artefact-based strategy involves sampling and analysing various forms of pre-existing or created written texts (e.g. presentations, commissioned research and other types of reports, journals, newspaper and magazine articles, annual reports, minutes of meetings, emails, media stories, transcripts of interviews, speeches) that you deem relevant to your research questions. You may assemble a sample of hardcopy texts (e.g., papers, manuscripts, books) and/or texts in electronic form (e.g., pdf documents, rich text format (rtf), Word or PowerPoint documents, text images), with the latter being far more common nowadays. Even in situations where ‘texts’ are accessed in hardcopy or non-textual format, you may convert them into an electronic text format. You may also create certain kinds of textual evidence in the form of a transformation or translation of qualitative data (e.g., interview transcripts, transcripts of speeches, translations of texts from other languages) from its original raw recorded format (pairing with the Transformative data-shaping strategy). Texts can be gathered in a manner consistent with either the positivist pattern (where it is typically referred to as content analysis and involves the transformation of qualitative data into quantitative data) or an interpretivist/ constructivist or other non-positivist pattern of guiding assumptions (where it may be referred to as document analysis or hermeneutics, for example). Every text you gather or create should be accompanied by contextual notes regarding its purpose, source, date, context of production, authorship, if relevant and so on. If the texts you gather constitute instances of previously published or unpublished research articles, reports, papers, theses and the like, then there are specific variations of the Textual strategy called meta-analysis (for quantitative research outcomes) or meta-synthesis (for qualitative research outcomes), which will be discussed under a separate heading below. For some types of research, texts (e.g., company or school annual reports or policy documents, minutes of meetings, media stories) may form background and contextual material for other data gathering strategies as well as the research context itself. This would be especially relevant in a Case Study, Action, Evaluation, Developmental Evaluation or Exploratory research frame. In such instances, the Textual strategy may comprise the first MU of a sequential MU configuration, part of a hierarchical MU configuration or background contextual material in a case-based MU configuration. If you produce texts in the form of transcripts from other data gathering activities (such as any of the interaction-based strategies), you might do so in the context of any research frame and as a consequence of virtually any MU configuration. Thus, we can see that, for some research purposes, texts may facilitate contextualisation (an elaborative or supplemental role) and, for other purposes, texts may constitute the main evidence you will use to address your research questions (a primary data role). It is important to note the distinction between the Textual strategy and the Diaries/Journals participant-centred strategy. The Diaries/Journals strategy yields texts that reflect on or record one’s own

14.2

Strategies for Exploring People’s Handiworks

609

personal experiences in the research context (whether in a participant or researcher role, i.e., a type of autobiographical orientation toward one’s research experiences) whereas the Textual strategy implements a much broader non-autobiographical orientation, at least with respect to experiences in the research context. When texts are sampled/collected from other sources (what Charmaz, 2014, p. 48, referred to as ‘extant texts’), you, as researcher, will not be involved in the creation, production or content of those texts and, very likely, their authors will not be immediately or directly involved in your research context (they may, in fact, no longer be alive). The production of such texts may, in fact, predate your research, may have been written to achieve very different goals and may even have been written in another language, thereby requiring translation (relevant perhaps in the Cross-Cultural research frame). Translation (for example, using Google’s automated translation process or a professional interpreter) is always a process that is filtered through the translator’s own cognitive processes and language expertise/ experience. This means that handling untranslatable concepts may require the translator to use expressions in the original language (creating a hybridised translation) and/or to use the nearest approximate meaning in the new language (which reduces translation accuracy). No matter what pattern of guiding assumptions you adopt, you must work hard to avoid potential biases in selecting texts to gather. Normally, this would require some type of sampling scheme (see Chap. 19) to ensure that the appropriate types of texts are gathered for your research purposes. However, access to documents and texts can be problematic, especially if the documents and texts are not on the Internet (as might be the case with certain company documents, minutes of meetings, older documents that predate electronic document storage, historical documents of various kinds and so on). You may have to invest effort and legwork to chase up texts that exist only in hardcopy format and, if they cannot be located, this rules them out of your sample. Certain texts and documents you might want access to might only be available on an organisational or institutional intranet, in which case, you will need to negotiate access (this may be especially important if you are seeking texts and documents to help you contextualise a group or a case study, for example). Government documents may or may not be publicly accessible and, if the latter, you may need to negotiate access or forgo access (if such texts are classified, for example). The Internet makes accessing many kinds of publicly available texts quite easy, but you need to establish a transparent basis on which to identify and select them for inclusion in your research. Such documents may be more difficult to contextualise with respect to their original purpose, authorship and authenticity. Any constraints on your access to the range and types of documents and texts you need for purposes of your research will, of necessity, alter your sampling intentions, thereby adversely impacting on your capacity to appropriately contextualise, generalise or claim sufficiency. If you adopt the positivist pattern of guiding assumptions for the Textual strategy, this will generally involve using a formal sampling plan (probabilistic, if possible) for selecting texts to analyse and using content analysis methods to code and eventually quantify text content (see, for example, Bryman & Bell, 2015,

610

14

What Data Gathering Strategies Should I Use?

Chap. 13; Hodson, 1999; Krippendorf, 2004; Weber, 1990). In such an approach, you would tend to focus on either manifest (surface content) or latent (deep content) themes reflected in the texts, but from an ‘outsider’s’ objectivist stance rather than from the author/producer’s perspective. Some key things to keep in mind about the positivist approach to the Textual strategy are: • You will need to rigorously plan and defend your sampling scheme for selecting texts. • You will need to plan, design (perhaps informed by a specific theory) and pilot test, in advance, the coding system you devise for labelling and representing content from the sampled texts (thus pairing this strategy with the Transformative data-shaping strategy). • You must attend to explicitly demonstrating inter-coder reliability and coding scheme validity (the latter is especially critical where latent content is being coded). If you adopt an interpretivist/constructivist or other non-positivist pattern of guiding assumptions for the Textual strategy, you will more likely be looking to unpack and display the author/producer’s perspective, rather than to simply illuminate isolated themes. If you adopt this approach, you would tend to look for deep structure, deep meaning and deep context behind the writing (see, for example, Bowen, 2009; Hughes & Goodwin, 2014a, 2014b; Rapley, 2007). For example, hermeneutics is one such interpretive approach, typically used for analysing historical documents (McAuley, 2004; Rowlinson, 2004). Thematic content is only one of many potential levels and angles of analysis that you could adopt in an interpretive/constructivist approach to the Textual strategy; others could include unpacking relationships, power dynamics and metaphors. Under certain patterns of guiding assumptions that have a critical orientation (critical social science, Indigenous or feminist), gatekeepers may safeguard important texts or documents that you would like access to and those gatekeepers may hold genuine fears about what granting you access might mean, culturally, legally and/or spiritually. In such cases, you would need to tread very carefully to negotiate access with the right gatekeepers, be open to the expectations they may hold of you if they grant access and honour any promises you give or obligations you accept in terms of how you treat the documents and the information they may contain/reveal if access is granted. Sampling in this approach may be purposive, systematic, opportunistic, or guided by what participants may offer up as documents to which they can provide access. Some key things to keep in mind about the interpretive/constructivist approach to implementing the textual evidence strategy are: • You will need to transparently describe your processes for identifying and selecting texts, facilitated by detailed annotations in your research journal about when, how, where, from whom and why specific texts were brought into the frame of your research. • You will need to transparently build up the system for coding, representing and displaying the author/producer’s perspective, based on the textual content itself (but perhaps informed by other data and contextual knowledge you have linked

14.2

Strategies for Exploring People’s Handiworks

611

to the author/producer; again, pairing this strategy with the Transformative data-shaping strategy). • You must take care to keep interpretations, accounts and stories about the texts as close to the textual data as possible to ensure maximal authenticity—this often means relying heavily on verbatim quoted material to help display convincing interpretations. There are several systems you can use to effectively support the Textual strategy, including: • using your research journal for recording notes throughout the text sampling process to ensure that each text is appropriately attributed to its source and is meaningfully contextualised, with respect to authorship, purposes and context of creation and reasons for inclusion; • using internet search engines, such as Google, Bing or Yahoo, to assist you in locating relevant documents such as company reports, media stories and other types of textual evidence; and/or • using a scanner for converting hardcopy documents to electronic form (scanning texts into a pdf format will transform them into a form that many qualitative data analysis software support systems will be able cope with). Bowen (2009), Bryman and Bell (2015, Chap. 23), Cohen et al. (2011, Chap. 12), Hughes and Goodwin (2014a, 2014b), Rapley (2007, especially Chap. 9) all discuss the Textual strategy in the form of document analysis, which is typically how this strategy is referred to under non-positivist patterns of guiding assumptions. McAuley (2004) and Prasad (2002) discuss the hermeneutic approach to using texts in research. Bryman and Bell (2015, Chap. 13), Krippendorf (2004) and Weber (1990) all discuss the Textual strategy from the positivist perspective, which is generally labelled content analysis. Onwuegbuzie, Leech and Collins (2010) discuss how to capture/represent nonverbal communication as textual data. Meta-analysis/Meta-synthesis Meta-analysis and meta-synthesis are important variants of the Textual strategy. Meta-analysis and meta-synthesis are approaches for sampling, assessing, theorising and generalising the learning amassed in a series of, most commonly, published research studies (books, unpublished research articles, theses and grey literature/ reports may also be incorporated). Thus, the texts sampled are the stories conveyed and displayed by researchers, typically in the academic literature. In short, meta-analysis and meta-synthesis provide ways to conduct and display systematic literature reviews. Meta-analysis focuses on research that gathers, analyses and interprets quantitative data and meta-synthesis focuses on research that gathers, analyses and interprets qualitative data. Both approaches provide avenues for systematically building up learning in a specific topic area that transcends individual research investigations, hence the term ‘meta’. Because of its focus on quantitative research, meta-analysis is primarily interested in research conducted under the guidance of the positivist pattern of assumptions and is, itself, considered

612

14

What Data Gathering Strategies Should I Use?

a positivist approach to integrating and generalising learning from the literature (see, e.g., Cooper, Hedges, & Valentine, 2009). Meta-synthesis, because of its focus on qualitative research, is interested in research conducted under the guidance of non-positivist patterns of assumptions and is, itself, considered an interpretivist/ constructivist approach to integrating and generalising learning from the literature (see, e.g., Noblit & Hare, 1988). Meta-analysis/meta-synthesis may be undertaken within any research frame as a pathway for understanding/summarising/generalising/theorising from relevant research literature. It is a particularly useful strategy for making sense of patterns and guiding learning from large numbers of research investigations on the same topic. This strategy may be implemented as a stand-alone research endeavour within a single MU configuration, where the meta-analysis/meta-synthesis is the intended outcome of the research project (this is the configuration employed for many published meta-analyses or meta-syntheses). Meta-analysis/meta-synthesis may also serve as a part of a larger investigation, whereupon, a sequential MU configuration is more likely to be employed, with the meta-analysis/meta-synthesis constituting the first or preliminary phase of the research, intended to set the stage and inform the remainder of the research process. A meta-analysis or meta-synthesis may also form a component of a hierarchical MU configuration, often in a supportive or supplemental role. Meta-analysis Meta-analysis focuses on published (most frequently, articles published in refereed academic journals) and, occasionally unpublished research articles and reports (see, for example, Cooksey, 2014, Procedure 8.8; Durlak 1995; Glass, McGaw, & Smith, 1981; Konstantopoulos & Hedges 2004; Schmidt & Hunter, 2014). The data for a meta-analysis, therefore, typically come from the stories written by researchers. Meta-analysis works only for synthesising research that produces quantified or quantifiable outcomes, meaning that, as a data gathering strategy, it is best suited for looking for patterns of outcomes and generalisations from positivist research investigations. Meta-analysis seeks to sample written research outcomes and to statistically analyse their findings in the search for more aggregated and generalisable patterns. A key quantitative concept associated with meta-analysis is effect size. Different systems for meta-analysis have differing statistical approaches to defining effect size (e.g. a standardised mean difference measure of effect size, an odds-ratio measure of effect size, or a correlational measure of effect size), but all approaches have roughly the same goal: to quantify, in some standardised fashion, the size of a group difference or a relationship between two or more measures, taking account of the size of the sample (N) in which that difference or relationship was observed in the context of a single research investigation. Once you have converted the statistical outcomes from individual research investigations into the selected standardised measure of effect size, you will be able to make various types of statistical comparisons between investigations. Figure 14.13a shows the sequence of events in the meta-analysis strategy: sampling of research articles,

14.2

Strategies for Exploring People’s Handiworks

(a)

(b)

Sample of ‘Quantitative’ Journal Articles

Meta-Synthesis Pathway

Coding of Journal Articles

Thematic Analysis of Articles

Coding Template

Domain of application

Year of publication _________ Decision problem area _______________

- Nursing - Aviation - Education - Consumer choice - Financial trading

Time-dependent decision Time-independent decision Low-impact risk decision High-impact risk decision

Negative influences on decision making - General/Unspecified Stressors - Time pressure - Emotional situation - Personal situation

Penalty for decision errors imposed Penalties for decision errors not imposed

- Family - Spouse

Stress measured by direct observation Stress measured by self-report instrument

- Professional situation - Job - Supervisor/Boss - Deadline

Sample size used in study ____________ Group comparison effect size _________ Correlational effect size ___________

- Past failures

…

Consequences - Post-decisional regret - Errors - Procrastination - Scrambling for solution - Health effects - Confusion - Changes in relationships

Statistical Analysis of Coded Articles Consequences of Being Wrong Penalities for Errors Not Imposed Measurement of Stress in Decision Context Impact Risk Associated with Decision

Sample of ‘Qualitative’ Journal Articles

may also be considered

Meta-Analysis Pathway

Time Sensitivity of Decision

613

Self-Report

Direct Observation

Low Impact-Risk Mean original r 0.11 0.21 Mean rz 0.110 se(rz) 0.080 Mean sample size 158.2 # of studies 47 Time-Independent High Impact-Risk Mean original r -0.06 -0.37 Mean rz -0.060 se(rz) 0.137 Mean sample size 56.3 # of studies 40 Low Impact-Risk Mean original r -0.28 0.07 Mean rz -0.288 se(rz) 0.115 Mean sample size 78.5 # of studies 41 Time-Sensitive High Impact-Risk Mean original r -0.29 -0.73 -0.299 Mean rz se(rz) 0.173 Mean sample size 36.4 # of studies 9

Penalties for Errors Imposed Measurement of Stress in Decision Context Self-Report

0.23

0.39

0.213 0.098

0.234 0.089

107.3 39

0.412 0.098

128.6 47 -0.52

-0.388 0.171

106.4 37 -0.71

-0.576 0.103

37.1 19

-0.887 0.185

97.8 37 -0.33 0.070 0.120

32.1 20 -0.66

-0.343 0.220

72.6 24

-0.793 0.165

23.6 27 -0.52 -0.929 0.112

82.9 8

Direct Observation

39.7 32 -0.86

-0.576 0.149 47.8 15

-1.293 0.234 21.3 8

Building up synthesis stories through: • Constructing a narrative & higher-order synthesis of learning across studies • Comparing/contrasting contexts, methodologies, data sources, meanings & interpretations Identifying commonalities & uniquenesses in • learning across studies Theory development incorporating learning • across studies

...

Fig. 14.13 Potential pathways for a meta-analysis and b meta-synthesis strategy (the table under the ‘Statistical Analysis of Coded Articles’ heading is taken from Cooksey, 2014, p. 452)

614

14

What Data Gathering Strategies Should I Use?

coding of research articles accompanied by calculation of relevant effect sizes and statistical analysis of codes and effect sizes. Some key issues associated with meta-analysis are detailed below. • You must carefully think through and defend your process for sampling/ selecting research articles and/or reports to include, because sampling biases (such as focusing only on research published in selected journals) can ruin your intentions to generalise. • You will need to argue for your choice of effect size measure that you plan to use in light of their advantages and disadvantages, and you must show that your choice is statistically appropriate for the types of outcomes being reported in the sampled investigations. You will need to make some strategic decisions very early on about what content of the sampled research investigations is to be coded and how (e.g. coding for specific features of the research design, methodology, type of journal article was published in, research quality, sampling scheme employed, population characteristics, laboratory studies versus field studies, experimental versus correlational research, studies in different countries or studies with single gender samples versus studies with multiple gender categories represented in the sample). The validity and inter-coder reliability of your final coding scheme will be important qualities for you to demonstrate (pairing with the Transformative data-shaping strategy). • You will also need to make a strategic decision about whether or not to include or exclude unpublished works, such as theses and dissertations, unpublished research, articles in the grey literature and consultant reports, in your meta-analysis sample, as failure to include such works could lead to a sampling bias that Rosenthal (1984) identified as the ‘file drawer’ problem (where unpublished studies typically outnumber published ones, and those that are published may be biased toward specific trends such as only reporting significant findings). • Your meta-analysis will generally be conducted with particular audiences in mind (which may include yourself, if you are using meta-analysis as a strategy to inform the conceptualisation, positioning and configuration of your own research as part of a sequential or hierarchical MU configuration). Meta-synthesis/Meta-ethnography Meta-synthesis (or meta-ethnography, as it was first known) was developed by Noblit and Hare (1988) as a way of interpreting and perhaps theorising about what has been learned in a sample of qualitative investigations (although quantitative research stories may also be considered). It can be very loosely considered as the qualitative analog of meta-analysis (Paterson, Thorne, Canam, & Jillings, 2001, for example, refer to it as qualitative meta-analysis). It has proven to be particularly useful in the medical, nursing and health sciences, but is much more widely applicable. Meta-synthesis takes an interpretivist/constructivist approach to working across a set of relevant qualitative investigations to build up a meaningful

14.2

Strategies for Exploring People’s Handiworks

615

perspective on what has been learned and probe whether shared concepts, processes, relationships, patterns and meanings in those studies can be identified and described. This then can lead to meta-theorising which may form part of the synthesis process. Synthesising is a comparative translational process whose goal, essentially, is to see if transportability can be established across the studies. Synthesis is not an aggregative generalisation process in the same way as a meta-analysis. Rather, through the comparative translation process, meta-synthesis attempts to construct an integrative perspective; one where both commonalities and uniquenesses among the studies are considered together in order to achieve a higher-order level of learning (one form of which may be a meta-theoretical account). Figure 14.13b illustrates a general meta-synthesis pathway where qualitative research stories are selected, the stories are read, coded, reread, interpreted and brought together in meaningful ways and synthesis emerges through a comparative translation process. The most challenging aspect of meta-synthesis is attempting to draw out meanings in service of an emerging theoretical story across qualitative investigations that likely did not have transportability as a research goal. Thus, you are seeking a transportable story where the original authors were not looking for one. This means that it can be very easy to read too much into what different investigations learn or to impose a pattern on findings that may not be adequately informed by the data. To minimise this risk, you must adopt a nuanced and balanced approach, in full awareness of any contextual differences that may exist between the different investigations you are synthesising (Noblit & Hare, 1988, offer one such approach). There are several systems you can use to effectively support meta-analysis/ meta-synthesis, including: • using Google Scholar, and other research-focused internet and library search engines and databases to locate articles for sampling (in the case of meta-analysis) or research stories to choose (in the case of meta-synthesis). • using Excel or other spreadsheet/statistical software to store quantitative codes for articles and other research outcomes you have reviewed in a meta-analysis; and/or • using a qualitative data analysis computer support package like MAXQDA, NVivo or dedoose to store the articles and other research outcomes you have reviewed in a meta-synthesis and to support your memoing, coding and synthesis processes. Cohen et al. (2011, Chap. 17), Cooksey (2014, Procedure 8.8), Cooper, Hedges and Valentine (2009), Durlak (1995), Glass, McGaw and Smith (1981—considered the classic original text), Konstantopoulos and Hedges (2004), Lipsey and Wilson (2001) and Schmidt and Hunter (2014) all discuss and illustrate positivist quantitative meta-analysis approaches, including the calculation of different kinds of effect sizes. Britten et al. (2002), Campbell et al. (2011), Paterson, Thorne, Canam, and Jillings (2001), Noblit and Hare (1988), Sandelowski, Docherty and Emden

616

14

What Data Gathering Strategies Should I Use?

(1997), Schreiber, Crooks and Stern (1997) and Walsh and Downe (2005) all discuss and illustrate interpretivist/constructivist meta-synthesis/meta-ethnography/ qualitative meta-analysis. Noblit and Hare (1988, pp. 26–28), in particular, present a framework of seven overlapping phases that may be revisited and refined as often as needed for a meta-ethnography. Suri (2011) discusses sampling considerations for meta-synthesis. Multi-media Artefact-Based Strategy The Multi-media artefact-based strategy involves the sampling and analysis of various forms of artefacts that are represented in a format other than simple written text (e.g., photographs, artworks, theatre performances, recitals, recordings, cartoons, videos, music, dance, films, plays, media releases, websites, postings on Flipboard, Google News and other multi-media news apps; image postings on Facebook, Snapchat, Instagram, Twitter, Pinterest, YouTube and other social media platforms). If we take a very broad view of the term ‘artefact’, we can extend the multi-media focus to encompass cultural symbols, objects, icons, rituals and any other artefact that has meaning for/is significant to people in specific contexts. The artefacts gathered using this approach will tend to emphasise a range of audio-visual and possibly other sensory representations (Banks, 2007; Plowright, 2011; Ross, 2001). Some multi-media artefacts, such as films, websites, media releases, YouTube clips, social media postings, news postings and speech recordings, may incorporate written text or spoken words as one medium in the mix. However, if you are interested in only the embedded written text or a transcript of the embedded spoken words in that artefact, this strategy effectively morphs into the Textual strategy, which means that you are deliberately sacrificing the multi-media nature of the artefact. Other multi-media artefacts (e.g. cartoons, photographs, artworks, performances, symbols, icons, rituals) can only be dealt with in their original form. Furthermore, some multi-media artefacts based on performances may be ephemeral (perhaps even one-off or unique), time-specific events (e.g., a play, dance, lecture or recital) that you must directly experience (pairing with the Participant observation-based strategy). In such cases, you will need to acknowledge this in your contextualisations of the artefacts and put procedures in places for recording your experiences. The Multi-media strategy can be undertaken in a manner consistent with either positivist or non-positivist patterns of guiding assumptions, and the paradigm-specific considerations will be largely the same as for the Textual strategy. Multi-media evidence gathering may be relevant in virtually any research frame, depending upon your research purposes. However, since cultural artefacts are also encompassed by this strategy, it can be particularly relevant to the Indigenous and Cross-Cultural research frames, but you would need to carefully negotiate the use of such artefacts for your research purposes. The Multi-media strategy may be used on its own in a single MU configuration or as part of a pluralist investigation that perhaps uses a simultaneous, sequential or hierarchical MU configuration.

14.2

Strategies for Exploring People’s Handiworks

617

Multi-media evidence typically focuses on sourcing/sampling artefacts originally produced for purposes other than your research, but where you bring that artefact into your research to help you address your research questions. It is possible to conduct research that synergistically combines the Multi-media strategy with the Diaries/Journals participant-centred strategy in instances where you invite participants to record their diary in a multi-media format (by taking photos or shooting video clips, perhaps using a smart phone) or the Stories participant-centred strategy in instances where you invite participants to record or perform their story in a multi-media context. It is also possible to conduct research that synergistically combines the Multi-media strategy with the Participant or Systematic observation-based strategy, where you, as researcher, record your observations in a multi-media format for later analysis. In an Indigenous or Cross-Cultural research frame, using this approach would need to be carefully negotiated and you must be prepared to be denied permission to record events and observations using cameras or sound equipment (there may, for example, be spiritual or cultural reasons why you cannot do such recordings). The Multi-media evidence strategy will present you with some unique challenges, depending upon the pattern of guiding assumptions you adopt. One challenge is embedded in the very nature of the artefacts—they rely on multiple sensory channels to convey meaning. You must therefore make strategic decisions about how those various channels should be handled, holistically or separately. If you adopt the positivist pattern of guiding assumptions for the Multi-media strategy, this will generally involve using a probabilistic or quota sampling design for selecting the multi-media artefacts to analyse. You will need to rigorously plan and defend your sampling scheme for selecting artefacts. In such an approach, you would tend to focus on either manifest (surface content) or latent (deep content) themes reflected in the artefacts, but from an ‘outsider’s’ objectivist stance rather than from the author/producer’s perspective. You could capture surface and/or latent content using a coding system you have designed, pre-tested and shown to be valid and reliable (pairing with the Transformative data-shaping strategy). Coding latent aspects will require much more validation effort to make the codes (and therefore the data) convincing. If you adopt an interpretivist/constructivist or other non-positivist pattern of guiding assumptions for the Multi-media strategy, you will more likely be looking to unpack and display the author/producer’s perspective, rather than to simply illuminate isolated themes. If you adopt this approach, you would tend to look for deep structure, deep meaning and deep context embedded in or reflected by the artefact. Thematic content is only one of many potential levels and angles of analysis that you could adopt in an interpretive/constructivist approach to the Multi-media strategy. If you adopt an interpretivist/constructivist or other non-positivist stance, you may try to deal with all modalities of the multi-media artefact as you experience them, a strategy less amenable to representing meaning through some type of pre-designed coding scheme. Instead, you would attempt to make a more holistic ‘reading’ of the artefact. In the end, however, the story you would tell about such artefacts would almost inevitably be textual in nature (e.g., in

618

14

What Data Gathering Strategies Should I Use?

your thesis or portfolio), which means that your display of meaning would have less richness than the original source of data. This can create problems that could potentially interfere with the authenticity of the perspective you are trying to present. If you want to achieve greater depth of understanding of social meaning, you could undertake a visual ethnographic approach, which reflects an “uptake of visual methods as an effective means to study the construction of social life through social practice” (Schembri & Boyle, 2013, p. 1251). Sampling in the Multi-media strategy, under non-positivist guiding assumptions, may be purposive, systematic, opportunistic, or guided by what research participants or other data sources may offer up as artefacts to which they can provide access (a kind of snowball sample). You will need to transparently describe your processes for identifying and selecting multi-media artefacts, facilitated by detailed annotations in your research journal about when, how, where, from whom and why specific multi-media artefacts were brought into your research frame. Under certain patterns of guiding assumptions that have a critical orientation (critical social science, Indigenous or feminist), gatekeepers may safeguard important artefacts that you would like access to and those gatekeepers may hold genuine fears about what granting you access might mean, culturally, legally and/or spiritually. In such cases, you would need to proceed very carefully to negotiate access with the right gatekeepers, be open to the expectations they may hold of you if they grant access and honour any promises you give or obligations you accept in terms of how you treat the artefacts and the information they may contain/reveal if access is granted. You will need to transparently build up your system for coding, representing and displaying the author/producer’s perspective, based on the multi-media content itself (but perhaps informed by other data and contextual knowledge you have linked to the author/producer; pairing with the Transformative data-shaping strategy). You must take care to keep interpretations, accounts and stories about the multi-media artefacts as connected to nature and/or content as possible to ensure maximal authenticity. In cases where you are dealing with Indigenous or cross-cultural contexts, this may mean you need to interview people involved with the creation, production, use, performance and/or caretaking of the artefacts you are accessing. There are several systems you can use to effectively support the Multi-media strategy, including: • using your research journal for recording contextualisation, observation and/or experiential notes throughout the multi-media sampling process; • using internet search engines or other online avenues for accessing multi-media recordings/postings on YouTube, Twitter, Facebook, Instagram, LinkedIn and other multi-media and social media support systems; and/or • using a camera, video camera or smart phone for assembling photographic and video evidence. Banks (2007), Cohen et al. (2011, Chap. 27), Harper (2005), Margolis and Zunjarwad (2018) and Ross (2001) all discuss visual research methods in general.

14.2

Strategies for Exploring People’s Handiworks

619

Margolis and Pauwels (2011) provide a handbook of visual methods. Bryman and Bell (2015, Chap. 28) and Fielding, Lee and Blank (2008) review e-research and online research methods. Sloan and Quan-Haase (2017) provide a handbook devoted to social media methods. Onwuegbuzie, Leech and Collins (2010) examines methods for studying multi-modal data. Finally, Pink (2013) and Schembri and Boyle (2013) discuss issues and processes associated with visual ethnography. Archival/Secondary Artefact-Based Strategy The Archival/Secondary strategy focuses on the sampling, retrieval and analysis of data sources such as organisational and institutional records, historical and other text-based records and archives, financial and other types of government, publicly or privately-maintained online or spreadsheet databases, and censuses and company annual reports (see, for example, Bryman & Bell 2015, Chap. 14; Cohen et al., 2011, Chap. 12; Cowton, 1998). Such data are gathered/maintained by people other than the researcher for purposes other than those of the researcher (data gathered directly by the researcher constitute primary data, see Veal, 2005). Archival data usually refers to collections of records and texts that are more historical and qualitative in orientation, which effectively pairs this strategy with the Textual strategy (see, for example, the National Archives of Australia, http://www.naa.gov. au/, which also offers online research tools and aids, https://www.archives.gov/ research/start/online-tools.html, or the Historical Archives of the European Union, https://www.eui.eu/Research/HistoricalArchivesOfEU). Such archives can be useful for historical, biographical, ethnographic and life history research, typically guided by a non-positivist pattern of assumptions. Note that the Textual strategies of meta-analysis and meta-synthesis, technically, depend upon archival data sources that store published and/or unpublished research texts (e.g., journal articles, conference papers, theses and dissertations, government report archives). Secondary data typically refers to maintained databases containing quantitative data (see, for example, OECD.Stat, https://stats.oecd.org/, or Principal Global Indicators, http:// www.principalglobalindicators.org/?sk=E30FAADE-77D0-4F8E-953CC48DD9D14735). Such data sources can be particularly useful for research, typically guided by the positivist pattern of assumptions (Smith, 2007), in the accounting, finance, education, management, political and policy sciences, geographical, operations research and economics disciplines. To get a feeling for what secondary databases and archives might contain, Fig. 14.14 provides screenshots of two publicly available and regularly updated online quantitative secondary databases, (a) IMF Database (http://www.principalglobalindicators.org/regular.aspx? key=60942001) and (b) OECD database (http://stats.oecd.org/Index.aspx? datasetcode=EAG_PERS_RATIO), and two examples retrieved from public record historical archives, (c) WWI Federal government archives (https://www. awm.gov.au/sites/default/files/exhibitions/1918/images/pr00027.jpg) and (d) immigration ship’s manifest (https://fallriverimmigrants.files.wordpress.com/2012/03/ cephalonia-manifest-1899-william-lomas.jpg). The Archival/Secondary strategy can be useful in a variety of research frames. Qualitative archival data sources may be useful in a Descriptive, Exploratory, Case

620

14

What Data Gathering Strategies Should I Use?

(a)

Secondary database – IMF data – exchange rates by country

(b)

Secondary database – OECD global education system data

Fig. 14.14 Screenshots of contexts from online public quantitative secondary: a financial database and b educational database Screenshots of contexts from online public qualitative archival databases: c war diary; and d ship’s manifest

14.2

Strategies for Exploring People’s Handiworks

(c)

(d)

Archival data – soldier’s war diary

Archival data – ship’s manifest

Fig. 14.14 (continued)

621

622

14

What Data Gathering Strategies Should I Use?

Study, Cross-Cultural or Indigenous research frame. Quantitative secondary databases may be useful in an Exploratory, Explanatory, Evaluation, Developmental Evaluation or Feminist research frame. A standalone single MU configuration is common in some types of educational, economic, econometric, accounting and finance research as well as for some types of historical and biographical research, where the research depends wholly upon secondary data sources. Where archival or secondary databases store data gathered at regular time intervals (e.g., daily, monthly or yearly), this can support a time-aligned longitudinal MU configuration. A sequential MU configuration is also commonly used, where the Archival/ Secondary strategy comprises the initial MU in the configuration. A hierarchical or case-based MU may use the Archival/Secondary data strategy as a supplemental or contextualisation strategy. Two major advantages of using archival or secondary data methods are: (1) substantial cost savings often accrue from not having to collect primary data and (2) the extent or reach of the data will often permit a wider scoping of your research project than would otherwise be feasible if you were left to your own data gathering resources (e.g., accessing more representative samples, capturing longitudinal trends across years, access to more obscure records). The second advantage is especially important if the units of analysis are in a highly-aggregated form (e.g. at company, industry or country level). A major difficulty, particularly with quantitative secondary databases, is that the accuracy of the data (unless they are independently audited) and the motivations of the data source creators/maintainers remain largely hidden from you, unless you are able to access background information on the database itself. There are some important questions you should ask about using any archival or secondary data source, some answers to which may be contained in the meta-data (i.e., information about the database and its contents, usually stored along with the database itself) that may accompany a particular data source. • For what purposes was the database originally established? Knowing this allows you to judge the potentially suitability of the database for your own purposes. • Who assembled/sponsored the database and are there costs associated with accessing the database? This goes to ownership of the data and relates to the original purposes for which the database was established. • For quantitative secondary databases, is there enough information available on the variables contained in the database, how they are defined and measured and how they should be interpreted (pairing with the Measurement data-shaping strategy)? Knowing this will help you to understand the nature of the measures you might wish to use in the database. Unless you can be assured of the nature and quality of all of the variables/measurements you plan to use, any model you construct and test could be called into question. Furthermore, if the entities populating the database provide their own data to the database builders/ maintainers (either on a voluntary basis or by virtue of a regulatory requirement), you need to be assured that each entity is providing the same measurements defined in the same ways.

14.2

Strategies for Exploring People’s Handiworks

623

• Is the database current and well-maintained and how often is it updated? This relates to the accuracy of the data contained in the database, but also to its reliability and consistency. This is especially important for databases that track variables/measurements across time. • Is the database searchable and, if so, what kinds of searches are supported? This addresses the functionality and user-friendliness of the database as well as the potential complexity/specificity of the searches that can be done. For example, Boolean search capability will allow you to combine key words and phrases and handle search inclusions and exclusions using Boolean operators such as AND, OR, NOT and NOR. • How were/are the data gathered, formatted, error-checked and uploaded/added to the database and by whom? This relates to the accuracy of the data contained in the database as well as to the methods used for gathering the data themselves. • Were the data sourced ethically with appropriate protections in place for the entities populating the database. The answer to this question should be ‘yes’ at two levels. The first level concerns the ethicality of data inclusion: are the data for any entity in the database included only with the informed consent of each entity. The second level concerns the ethicality of data access: if the database is accessed through a third party, did that party acquire ethical access to the data and is their version of the database current and unmodified? Ethicality of data access is critical to address because data harvesting without the consent of the entities or companies who provided the data or originally assembled the database is ethically reprehensible since such harvesting directly opposes the right to privacy. This has emerged as a particularly thorny problem in our modern digitally-connected world where software used to connect people (e.g., social media, telecommunications) can also record and store data, with or without users’ knowledge; with user’s knowledge and consent is the ethical standard to be met. However, this does not always happen, as was the case, for example, in 2014 when Cambridge Analytica, without authorisation, harvested millions of user profiles from Facebook in order to profile voters for political purposes, which created a mountain of legal, financial and consumer trust problems for Facebook and Cambridge Analytica to deal with. One work-around that is commonly implemented is ‘opting out’, where consent is automatically presumed unless a data source explicitly indicates they do not want their data harvested and/or on-sold to third parties; however, this workaround does not constitute informed consent, especially if the ‘opt out’ is in small print or not easy to locate. If you cannot answer ‘yes’ to the ethics question at both levels to your satisfaction, you should avoid accessing such data at all costs. Archival and secondary data methods require you to accept a very important assumption, namely, that a data source you intend to use provides the data necessary for evaluating your research questions. The issue is often one of completeness here. You will have no control over which data are gathered and how they

624

14

What Data Gathering Strategies Should I Use?

are represented in the data source. If there are important content, conceptual or measurement gaps in the data, there may be no way you to independently fill those gaps. This may force you to revise the scope of your study and perhaps even your conceptual framework. Thus, you must, of necessity, accept the contents of the secondary or archival data source as a given. In the case of a quantitative secondary database, this means accepting the operational definitions of the variables it contains and the unit(s) of analysis (e.g. company, geographic region, group, industry, country) represented in the data. There will normally be insufficient information accompanying such a secondary data source to permit you to verify either the validity or reliability of the measures, unless you have access to other data sources that can be used for triangulation purposes. In the case of a qualitative archival database, this means accepting their processes for acquiring the documents and the quality of any documents as stored (even if marginally readable in the case of handwritten records). Recently, there have been many advances in large-scale secondary database development, management, storage and use, giving rise to ‘data science’ (Provost & Fawcett, 2013). The resulting databases are often colloquially referred to as Big Data (see, for example; Boyd & Crawford, 2012; Gandomi & Haider, 2015; Walker, 2014). Such Big Data databases are typically assembled by governments or large companies (e.g., census bureaus, health departments, labour departments, education departments, large organisations, such as advertising agencies and marketing organisations), are typically quite massive and are often managed through a process called data warehousing. Their use in research has created a new field called knowledge discovery (see, for example, Fayyad, Piatsky-Shapiro, & Smyth, 1996), where warehoused databases are made publicly available to researchers in order to ‘mine’ the databases for previously unknown patterns and new knowledge. Tools for knowledge discovery in Big Data include machine learning and data mining, text mining (for qualitative databases), and predictive analytics (building up predictive statistical patterns and relationships using large-scale databases; Cohen et al., 2014, discuss the legal and ethical issues associated with the use of predictive analytics in the health care industry). Postgraduate researchers, appropriately trained, can certainly pursue opportunities to engage in knowledge discovery (see, for example, Graco, 2001, who searched for patterns in a sample of doctor shoppers from the Australian Pharmaceutical Benefits Scheme (PBS) database, maintained by the Australian Health Insurance Commission), but the need to address the questions posed earlier remains. There are several systems you can use to effectively support the Archival/ Secondary strategy, including: • your research journal for recording notes throughout the Archival/Secondary research process including meta-data concerning the nature of the database or data source itself; • access to electronic secondary databases (free or subscription-based) plus access to documentation supporting that database and describing all variables/ documents/records it contains (some archival data collections may be held in

14.2

Strategies for Exploring People’s Handiworks

625

hardcopy form in libraries) [On 5 September 2018, Google launched a beta version of a new dataset search tool for locating online datasets uploaded by researchers and organisations, see https://toolbox.google.com/datasetsearch]; • access to Excel or other spreadsheet software for recording/storing data sampled from a secondary database; • access to software for implementing data mining and predictive analytics, for example, RapidMiner (https://rapidminer.com/) or IBM SPSS Predictive Analytics (https://www.ibm.com/au-en/marketplace/spss-predictive-analyticsenterprise); and/or • access to software for implementing text mining (e.g., statistical analysis of texts), for example, Leximancer (https://info.leximancer.com/) or Wordstat (https://www.ibm.com/au-en/marketplace/spss-predictive-analytics-enterprise). Boslaugh (2007), Bryman and Bell (2015), Chap. 14, Goodwin (2012a, 2012b, 2012d), Hofferth (2005) and Smith (2008) all discuss and illustrate secondary data strategies, including discussion of ethical considerations. Corti, Thompson and Fink (2004) discuss approaches for using archived qualitative data. Hinds, Vogel and Clarke-Steffen (1997), Goodwin (2012c) and Irwin (2013) each review qualitative approaches to secondary data analysis. Cohen et al. (2011, Chap. 12), L’Eplattenier (2009), Hughes and Goodwin (2014c), McAuley (2004) and Rowlinson (2004) discuss archival/historical research. Cooksey (2014, Procedure 9.9), Han, Kamber and Pei (2012) and North (2012) discuss approaches to data mining. Grimmer and Stewart (2013), Gupta and Lehal (2009) and Kumar and Bhatia (2013) discuss text mining in various contexts and disciplines. Smith and Humphreys (2006) discuss the unique Leximancer approach to text mining.

14.3

Strategies for Structuring People’s Experiences

The domain, structuring people’s experiences, encompasses strategies that you can use to shape the data that you gather from participants into a quantitative or qualitative form suitable for analyses (i.e., data-shaping strategies) as well as several strategies for focusing directly on structured researcher-influenced experiences of participants or focusing on contextualised experiences they are having, or have had, about which you will be gathering data (i.e., experience-focused strategies). Figure 14.15 shows the 3-level expanded mindmap associated with this domain of strategies. There are three broad data-shaping strategies, two of which focus on shaping data gathered from or about research participants (Measurement and Transformative) and one (Generative) which focuses on the creation of quantitative data, without the direct need for participants. There are four different experience-focused strategies, three of which generally focus on scenarios where you, as the researcher, can exert partial or complete control over context to structure participants’ experiences (Manipulative, Structuring and Immersive) and one (Non-manipulative) where you must take the contextual events experienced by

626

14

What Data Gathering Strategies Should I Use?

Structuring People’s Experiences

Fig. 14.15 Expanded 3-level mindmap branches focusing on strategies for structuring people’s experiences (encompassing data-shaping strategies and experience-focused strategies) as well as some key considerations associated with each strategy

participants as a given (i.e., shaped by other people or other forces) rather than as something you have structured. The positivist pattern of guiding assumptions is more commonly associated with strategies for structuring people’s experiences for theory building and testing purposes and, under this pattern of assumptions, experience-focused strategies are always associated with one or more data-shaping strategies, normally to produce quantitative or quantified data. This is because theorising and evaluating externalised causal-effect relationships in the world is a central facet of the positivist pattern. However, the critical realist pattern may also be aligned with experience-focused strategies when understanding external causation of events is of interest. In either case, what is important is that you achieve as much control over context as is possible so that cause-effect relationships can be unambiguously isolated. However, it may be that you intend to pursue causal explanations but are hampered by constraints on your ability to achieve some or any control over context (especially where naturally-occurring events, completely outside your control, become your focus for causal explanation). In such instances, some control may be achieved via means other than control over context (e.g., through use of sophisticated statistical analyses) and this may assist in boosting your chances of drawing convincing causal conclusions. In any case, you exercise control in order to manage threats to sound causal inferences when you test your theoretical hypotheses and

14.3

Strategies for Structuring People’s Experiences

627

help you to rule out alternative plausible explanations for patterns and relationships you observe in the data you gather. It is important to realise, however, that experience-focused strategies are not the exclusive province of the positivist or critical realist patterns of guiding assumptions. Depending upon your goals, it may be entirely appropriate to employ one or more experience-focused strategies under the guidance of an interpretivist/ constructivist, critical social science, Indigenous, feminist or participatory inquiry pattern of guiding assumptions. Here, though, the concept of control gives way to different justifications for structuring people’s experiences, namely to understand how people react to and interpret experiences in a specific context or to help people unpack their experiences in a specific context more fully. For example, it is entirely plausible to carry out an interpretivist/constructivist investigation, yielding primarily qualitative data, in a context such as a simulation, game or task set up by the researcher. This reflects a goal of simply providing a context for experience which helps to focus both your attention and participants’ attention on the research issues of interest. Externalised causation is not the issue of interest in these cases; subjective perspectives (which may include internalised views of causation) are of more central concern.

14.3.1 Data-Shaping Strategies Data-shaping strategies generally focus on ways of ensuring that you gather or produce the appropriate kind of quantitative or qualitative data. Appropriateness would be judged through considerations of your research questions/hypotheses in the context of your research frame, researcher positioning, participants’ positionings and the pattern of guiding assumptions you are adopting. You would implement the Measurement strategy when you need to design processes for obtaining quantitative data in the forms and with the meanings that you need, thus shaping data at the time you gather them (most commonly associated with the positivist pattern of guiding assumptions). You would implement the Transformative strategy if you are gathering or have already gathered quantitative or qualitative data but need to change their form or type for analysis purposes, thus reshaping data either at the point they are gathered or after you have gathered them (may be relevant under a variety of patterns of guiding assumptions). You would implement the Generative strategy if you need to create data from scratch, without having access to or needing actual human participants (in effect, using computer-generated ‘participants’). The Measurement and Transformative strategies are seldom used on their own. Instead they are used in conjunction with a wide range of other strategies to support data gathering activities. The Generative strategy can be used in a stand-alone capacity, but often will be combined with the Manipulative experience-focused strategy through implementing a specific experimental design. Where you seek quantitative data, these strategies depend upon theoretically defensible or practically meaningful operational definitions for the measurement

628

14

What Data Gathering Strategies Should I Use?

process (more about this in Chap. 18). Under the positivist pattern of guiding assumptions and depending upon your goals, construct validity, content validity or criterion-related validity as well as reliability will be relevant properties for measurements to possess. You may either (1) empirically demonstrate that your measures possess specific and relevant types of validity and reliability (especially important if the measures are new) or (2) argue that your measures already possess such qualities based on previous research (especially important where you adopt measures created by others). Pathway (1) means that a more complex sequential MU configuration is required so as to include a distinct measurement validation stage prior to the main data gathering stage. Pathway (2) means you depend upon the convincingness of the literature surrounding the adopted measures to carry the day. This argument is much harder to sustain if the measures you adopt were validated in another country or culture because then you must argue for cross-cultural translatability, equivalence, meaningfulness and appropriate cultural (including religious) sensitivity as well. Certain data-shaping strategies (yielding qualitative or quantitative data) may be employed if you adopt an interpretivist/constructivist pattern of guiding assumptions. For example, you might use some quantitative economic measures to help contextualise your research context (e.g., to help you characterise the economic status of a suburb in a large metropolitan city as ‘disadvantaged’ for purposes of focusing your research sampling). You might use a questionnaire within the Survey research frame to gather qualitative data using entirely open-ended questions. In a pluralist investigation, you might use a Transformative data coding protocol to transform certain aspects of your qualitative data into quantitative measures (to help bolster arguments about prevalence of meanings, for example). As well, you might use a preliminary scheme or conceptual framework to commence your coding of qualitative data into higher-order categories and themes, with a view toward augmenting, extending, reshaping and refining that scheme/framework as more data come to hand (what we will describe as conceptually-driven coding in Chap. 18). Measurements Data-Shaping Strategy The Measurements strategy, in the social and behavioural sciences, focuses on designing and implementing procedures for quantifying the characteristics and behaviours of people and the systems they inhabit (Chap. 18 will dive more deeply into how you can ensure high quality measurement processes). Such measurement can occur at a myriad of levels of analysis, ranging from processes (e.g., neurological, physiological, psychological) internal to individual people to groups to organisations and institutions to societies and nations to the global community (Hand, 2004). The Measurement strategy is entirely congruent with the positivist pattern of guiding assumptions but may also be consistent with critical realist assumptions as well. Its goal is to provide the quantitative variables that you need to test hypotheses and/or search for patterns and relationships and, in this sense, measurement is most often used as a strategy to support one or more other data gathering strategies. The Measurement strategy can be useful in virtually any research frame, but is most strongly associated with the Descriptive, Explanatory,

14.3

Strategies for Structuring People’s Experiences

629

Evaluation and Survey research frames. It may also be useful in any particular MU configuration where quantitative data are required. We consider the Measurement strategy to be data-shaping because measurement processes produce quantitative data at the outset. What is measured are constructs or theoretical concepts/ideas that emerge in relation to research questions/hypotheses and those constructs may focus on the internal mental (e.g., what people or groups think and feel as conveyed by how they behave) and physical (e.g., metabolic rates, blood flow, nutrients, biochemical and neurochemical processes) states of people or on objects, people and other entities in the external physical world (which encompasses social, cultural, political, geographical and economic facets in addition to natural and ecological facets). Measurement requires some sort of instrumentation that converts an phenomenon into meaningful numbers and that instrument may be observational (e.g., what you can see, hear, feel, taste, smell, count, sort, and so on), technological (e.g., what you can read on a ruler, dial, clock, display, scale, meter or other measuring device) or self-reported (e.g., patterns of answers to questions on a questionnaire or of responses to some stimulus). When you employ measurements of any type at any level of analysis, you have a responsibility to either empirically demonstrate the quality of your measurements or argue for their quality. Chapter 18 discusses the four potential scales for measurement (nominal, ordinal, interval, ratio), several criteria for measurement quality, including various kinds of validity (construct, content, criterion-related, face), various kinds of reliability (internal consistency, test-retest, inter-observer) and the issue of measurement sensitivity. Some measures or indices may be formed as mathematical composites of other measures (as with economic indicators, financial ratios) and here you need to ensure that the processes for identifying/constructing the composite or index are credible, defensible and theoretically meaningful. Consider the range of possibilities (not exhaustive by any means) for measurement at different levels of analysis: • Physiological measures—encompass medical and health-related measures such as heart rate; galvanic skin response [GSR]; blood pressure; EEG wave patterns; chemical composition of blood and other substances; measurements from pictorial/graphical representations of physiological systems (such as X-rays, CT scans, MRIs and ultrasounds); eye movements (obtained through direct observation, often with some type of technological or mechanical instrumentation; see, for example, Webster, 2015). Some physiological measures may be intrusive, even painful, in that they require physical contact or technological interactions with a participant (e.g. taking a blood sample to measure cortisol, sugar or hormone levels; taking a blood pressure reading, taking an EEG or ECG reading, wearing an apparatus to follow eye movements) which may influence the quality of the measurement, because some participants may have adverse reactions to that measurement process. • Psychological measures—encompass tests and assessments of constructs such as learning, skill, aptitude, ability, intelligence, achievement, potential, task or job performance (may be observed by someone else or obtained by self-report)

630

•

•

•

•

14

What Data Gathering Strategies Should I Use?

as well as more indirect measures of constructs such as values, attitudes, preferences, sentiments, beliefs, motivations, personality, sociability, willingness to pay, risk taking propensity (typically obtained via self-report; see, for example, Boyle, Saklofske, & Matthews, 2015; Robinson, Shaver, & Wrightsman, 1991). Social and cultural measures—encompass measures of crime rates; incarceration rates; accident rates; suicide rates; life expectancy; life satisfaction; multi-cultural attitudes; quality of life; poverty rate; voting preference; social network characteristics (may be observed or obtained via self-report; see, for example, Boyle et al., 2015, Sect. VI; Gamst, Liang, & Der-Karabetian, 2011) Demographic measures—encompass measures of physical and situational characteristics of people and groups, including age, location of residence, height, weight, eye, hair or skin colour, ethnic background, occupation, nationality, educational attainment, income, marital status, family size (may be observed or obtained via self-report; see, for example, Carmichael, 2016; Murdock, Kelley, Jordan, Pecotte, & Luedke, 2016). The use of demographic measures is most consistent with positivist assumptions but may also be useful for characterising attributes of participants in research guided by a non-positivist pattern of assumptions. Physical measures—encompass measures of elapsed time or time taken to complete a task or process; counts of errors made; distance travelled, walked or run; calories burned; weight lifted; counts or measurements of product or process defects; task completion sequences; fuel used, air quality, water quality, pollution, waste, carbon footprint (may be observed, obtained via instrumentation or obtained via self-report; see, for example, Hand, 2004, Chap. 7; Williams, 2014). Some physical measures may be intrusive in that they require a participant to interact with technology (e.g. wearing a pedometer to count steps) which may influence the quality of the measurement, because some participants may have adverse reactions to the measurement process itself. Economic/financial measures and indices: encompass measures such as Gross Domestic Product (GDP); unemployment rates; tax rates; efficiency; effectiveness; inflation rates; interest rates; exchange rates; price; stock prices; Consumer Price Index (CPI); labour productivity; well-being index; Gini index (measure of inequality in wealth distribution) as well as financial ratios such as those for liquidity (e.g., ratio of assets to liabilities), efficiency (e.g., inventory turnover), profitability (e.g., operating margin) and solvency (e.g., debt to equity ratio) (may be observed, obtained via instrumentation or obtained via self-report; see, for example, Frumkin, 2015; Stengel & Chaffe-Stengel, 2012; Yamarone, 2017). Many of these measures are actually composite indices (often called ‘indicators’) formed by mathematically combining other more specific measurements. In some cases, economic and financial measures are expressed in units of a specific currency system such as U.S or Australian dollars, British pounds, Turkish lira and the like. Longitudinal time-aligned MU configurations are commonly employed for economic indicators, tracking the behaviour of economic/financial indicators/ratios over regular intervals of time (e.g., daily, monthly, yearly; see discussions in Stengel & Chaffe-Stengel, 2012, Chap. 2).

14.3

Strategies for Structuring People’s Experiences

631

Additional Considerations for Using Tests or Assessments You may be interested in measuring abilities, aptitudes, intelligence, understanding of concepts, learning outcomes, training effectiveness, skills and/or other cognitive domains where your goal is to measure achievement, capability or potential using some type of test or assessment task (for example, see Athanasou, 1997). Tests and assessments form one important category of psychological measures, but may also incorporate physical measures as well, especially where performance and skills are being measured. Such measures have the virtue of employing questions where the correct answers are known or can be argued for. This means they are only useful in situations where you can identify a reference content or skill domain and establish correct answers/behaviours. You can administer such tests in paper-and-pencil format, or via computer or Internet website. You can administer work sample or other types of situationally-specific assessments in the context of a classroom situation or a situational simulation such as an in-basket test for managerial competencies (thus pairing this strategy with the Immersive experience-focused strategy). Tests or assessments can be used in virtually any research frame, depending upon your needs, although you would need to do this with extreme care in the Indigenous and Cross-Cultural research frames. As well, they could serve a useful purpose in almost any MU configuration. However, if you use tests or assessments in any type of longitudinal MU configuration, you need to be aware of the risk of possible memory and other carry-over influences if you use the same version of the test or assessment on every data gathering occasion (such influences would likely have the effect of artificially inflating scores on subsequent occasions). If you are constructing a test or assessment from scratch, you will likely need to employ a sequential MU configuration, where your first phase will be to validate and refine your measurement processes using a separate sample from that you will use for phase 2, your main study. This, of course, will extend the timeline for your research. Tests or assessments can be usefully combined with any of the experience-focused data gathering strategies. As well, it may be productive to combine the tests and/or assessments with self-report questionnaires if your goal is to build up a well-rounded psychological, educational or managerial profile of your research participants. Content validity is critical for tests and assessments (more will be said about content validity in Chap. 18) and addresses whether the test items measure the correct content domain at the right level and with the right weighting. Item content should be determined by sampling the domain(s) of learning/knowledge/skills to be assessed. It is also possible to construct a test based on items with known statistical and psychometric properties (typically established using a psychometric approach called Rasch modelling, see, for example, Bond & Fox, 2015; Cooksey, 2014, Procedure 9.1; Lamprianou, 2008), so that the test has a predetermined level of difficulty and precision in measuring achievement or skill. In a computerised testing environment, it is even possible to build the test in real-time, selecting the next item to present to a participant based on their response to the immediately preceding item

632

14

What Data Gathering Strategies Should I Use?

(a strategy called ‘computerised adaptive testing’, see Fogarty, 2008). Since tests and assessments have the virtue of employing items where the correct answers are known, you can set and statistically defend standards and benchmarks (e.g. minimum competency), especially in cases where Rasch modelling has been used to construct the test. Tests and assessments can be designed to be norm-based (where scores are evaluated relative to some reference population of interest) or criterion-based (where scores are evaluated relative to a set of criteria or standards for achievement). Tests and assessments are not easy to construct and, for that reason, you may choose to adopt a test or assessment assembled and validated by a professional test development organisation like the Australian Council for Educational Research (ACER) in Australia (https://www.acer.org/), the Rogers Group in Australia (http:// www.rogersgroup.com.au/), the Educational Testing Service (ETS) in the US (https://www.ets.org/) or Psychometric Tests in the UK (https://www. psychometrictest.org.uk/). It is important to note that if you use a developed test or assessment in an Indigenous or Cross-Cultural research frame, you need to check whether any norms associated with interpreting scores on the instrument apply to the type of participants in your research. If there are no appropriate norms, you may need to find other ways to interpret any scores, including possibly establishing new norms. If you are developing a test to measure learning achievement (e.g., in a course, classroom or workshop context, perhaps in the context of the Evaluation research frame), you can develop such tests quite easily, but you need to take time and care in writing proper test questions (multiple choice questions, in particular, must follow sound development and validation practices; see, for example, Haladyna, 2004), ensuring that all intended learning domains are covered and ensuring that participants cannot obtain artificially high test scores simply by knowing how to take tests (a phenomenon known as ‘test wiseness’, see Evans, 2015) or by guessing. Where possible, you should also pre-test your test or assessment to ensure it is working the way you want it to. Additional Considerations for Using Self-report Questionnaires Self-report questionnaires encompass a class of measures where participants are asked to reflect some internal mental state in their categorical, numerical, graphical or written responses to stimulus questions that you provide as researcher (see, for example, Bryman & Cramer 2004; Punch 2003; Frazer & Lawley, 2000; Sapsford 2007). Self-report questionnaires generally encompass psychological measures, social and cultural measures and demographic measures. Some economic and financial measures may also be obtained by self-report means as well. Questionnaires are typically designed to provide directly quantified responses to questions using some type of numerical response scale. The administration of self-report questions to participants may be technologically mediated or supported (e.g. administered via email or online via an Internet interface such as SurveyMonkey, see Sue & Ritter 2012) or may rely on hard copy—as with mail surveys or evaluation forms. Self-report questionnaires are frequently used to measure psychological constructs (often called ‘hypothetical constructs’, see

14.3

Strategies for Structuring People’s Experiences

633

Hyland, 1981 for a useful discussion of types of constructs) such as attitudes, motivations, beliefs, values, intentions and preferences, which are considered to be ‘subjective’ measurements, thus not externally verifiable. However, questionnaires can also gather demographic measures, which are potentially externally verifiable, and can be used to classify participants in various ways. ‘Tests’ that measure various theoretical constructs associated with personality (e.g. the Myers-Briggs Type Indicator [MBTI], the NEO Personality Inventory, Holland’s Self-Directed Search for assessing vocational interests) would be classified as self-report measures. Construct validity is a critical quality for self-report questionnaires to possess, although this can be difficult for you to convincingly demonstrate (see Chap. 18 for more detail). Construct validity can be demonstrated statistically, using correlational and factor analytic approaches, which can assist you in making validity claims more convincing. However, you must plan for such a demonstration by ensuring that appropriate sampling and analyses are undertaken. Self-report questionnaires should transparently emerge from your translation of theoretical constructs into physical questionnaire items via the process of operational definition; greater transparency in your translation process will lead to a stronger foundation for construct validity. Commonly, self-report questionnaires measure multiple constructs (e.g. job satisfaction, organisational commitment, organisational citizenship behaviours) in the one instrument, and any one construct may comprise several dimensions or facets (for example, one version of the organisational commitment construct has three dimensions—affective, normative and continuance commitment). Self-report questionnaires, however administered, are useful for tapping large cross-sectional samples efficiently and for gathering a large volume of data within a relatively short time frame and in a reasonably resource-efficient manner. In general, it is fair to say that cross-sectional self-report questionnaires constitute one of the most commonly employed behavioural and social data gathering approaches in research today, typically within the Survey research frame. However, cross-sectional questionnaires are rather weak tools for supporting causal inferences (because of poor control over the sequencing [and values] of independent variables [putative causes] and dependent variable [putative effect] measurements) unless a longitudinal MU configuration or very sophisticated causal modelling techniques are employed. Self-report questionnaires used in the context of the Manipulative experience-focused strategy can play a role in disentangling cause and effect relationships precisely because in this strategy you have at least some control over relevant independent and extraneous variables. A simple triangulation strategy you can easily employ in self-report questionnaire research is to include open-ended questions in your questionnaire along with closed-response questions (e.g., rating scales, demographic questions). Such items give the participant the opportunity to write down their own thoughts and ideas, perhaps as a general comment, or in response to a more specific request for their own views. Their responses to such questions constitute a type of focused story (focused by their interpretation of the question content). So, for example, a leadership questionnaire which provides a number of specific rating items for measuring

634

14

What Data Gathering Strategies Should I Use?

different leadership constructs might conclude with a general open-ended question such as: “According to you, what is good and effective leadership?” (Muchiri, 2006). What participants write down can then be thematically or content-analysed and the results used to help support the story provided by quantitative data from the questionnaire. Many participants respond positively to being given an opportunity to express their own views, so a questionnaire that actually invites them to do so can be seen as more interesting and engaging and may actually improve response rates. Of course, it is possible to construct a questionnaire consisting entirely of open-ended questions, which would be a strategy more consistent with an interpretive/constructivist approach, if the questions were not too prescribed or delimited in focus. One drawback to the use of open-ended questions is that the option of whether to express anything—and if so, what—is left entirely up to the individual participant and their frame of mind as they complete the questionnaire. You should therefore expect some not-so-serious responses from people as well as some very startling, perhaps even offensive, responses. In some management questionnaires, particularly where anonymity has been assured, open-ended questions may invite ‘spleen-venting’ of a general or specific nature. In a positivist investigation, such responses would probably be discounted as ‘extreme’ or irrelevant. However, in an interpretivist/constructivist investigation, such responses may provide insights into participants’ perspectives and feelings at the time, thus constituting a valuable source of data. Additional Considerations for Using Physical Measures Using physical measures in social and behavioural research looks simple on the surface, but their interpretation can be complicated. If you measure time, you use a stop watch; if you count errors, you might use a manual counting device; if you measure distance or length, you use a ruler, measuring tape or, in some cases, GPS coordinate changes. However, the meaning you ascribe to such measures depends upon their construct validity, not so much with respect to the device being able to measure a desired physical quantity, but in terms of the inferences that you can defensibly make about what such measurements might be reflecting about people, psychologically, socially or health-wise. For example, you may measure, by observation, the number of defective products produced in a day on an assembly line in a quality control study. Physically, the count is what it is—the number of defective products observed. However, if you want to infer that the number reflects something about worker attitudes toward quality or that better training is needed for assembly line workers, these are entirely different matters. The same is true with physical measures of human ‘stress’ where the physical measure may be a blood pressure reading, galvanic skin response reading, cortisol level or pulse rate count, but the inference you might really be after is whether the measure reflects a person’s felt level of ‘stress’. Thus, construct validity becomes important for defending any inferences beyond the physical dimension being measured. In many cases, a single physical measure cannot stand on its own as a measure of a psychological construct like stress, ability or attitude; it must be one component in a suite of measures (some of which might be self-reported). You could then argue a case for construct

14.3

Strategies for Structuring People’s Experiences

635

validity based on how strongly correlated all the different measures in the suite are. Irrespective of the construct validity question, physical measures will typically be more reliable than self-report measures. Physical measurements are not subject to the range of response biases that can plague self-report measures. Thus, the measurements produced are less contaminated by participant-related influences. However, interpretational problems can occur from the point of view of the person doing the measuring. For example, observer biases are possible, especially if the observer must operate or trigger an instrument (e.g. such as a stop watch), read a dynamically moving scale indicator (e.g. a sphygmomanometer, measuring blood pressure), or be continuously vigilant (e.g. counting events or behaviours) to capture the measurement. In some cases, the measurement process can be completely automated, which can control for most observer biases but often at a greater cost to the researcher. There are several systems you can use to effectively support the Measurement strategy, including: • using your research journal for recording notes throughout measurement development, validation and/or adoption processes; • for tests and assessments: – using computerised adaptive testing and psychometric analysis systems and software to develop and administer your test or assessment (see, e.g., http:// www.assess.com/psychometric-software/?nabe=6511435425513472:1; http://iacat.org/content/cat-software). – using statistical software to analyse the psychometric quality of your test or assessment, via item analysis, reliability analysis or Rasch analysis (e.g., SPSS, SYSTAT, NCSS, MINISTEP/WINSTEPS, see discussions in Cooksey, 2014, Procedure 9.1). • for self-report questionnaires: – using a software package that can assist in designing and administering your questionnaire (e.g. SurveyMethods, http://www.surveymethods.com/index. aspx and Key Survey Software, http://www.keysurvey.com/). SurveyMonkey, http://www.surveymonkey.com/, provides an online data gathering tool that is easy to use. The Capterra website provides a listing of several vendors of survey design software packages, many of which offer free trial downloads, http://www.capterra.com/survey-software. – using statistical software, such as SPSS, SYSTAT or NCSS to conduct item analyses, factor analyses and related techniques to analyse the quality of self-report questionnaire measurements (see discussions in Cooksey, 2014, Procedures 6.5, 8.1 and 8.6). Hand (2004) provides a very useful overview of different kinds of measurement systems one might use in research. Keeves (1997, Sect. III) presents a range of contributions covering many facets of educational measurement, a number of which are relevant to the wider social and behavioural sciences. Shum, O’Gorman, Creed,

636

14

What Data Gathering Strategies Should I Use?

and Myors (2017), Reynolds, Livingston, Willson and Willson (2010) and Athanasou (1997) are three general texts that provide full discussions of psychological and educational measurement, testing and assessment. Lane, Raymond and Haladyna (2015) provide a thorough discussion of issues surrounding test development, whereas Haladyna (2004) discusses the development of multiple-choice tests in particular. Nunnally’s (1978) classic text covers a range of issues and processes associated with psychometric testing; Chap. 13 explicitly covers test of ability. Bond and Fox (2015), in their well-cited contribution, discuss the mechanics and applications of the Rasch model to measurement problems. Thornton and Kedharnath (2013) discusses issues surrounding the development and use of work sample tests in organisational settings. Nunnally (1978, Chaps. 14 (focusing on personality measures) and 15 (focusing on measures of sentiments)) and DeVellis (2016) discuss self-report scale development. The Boyle et al. (2015, Sect. I) handbook provides a useful exploration of issues associated with using self-report measures. Carmichael (2016) provides a thorough coverage of concepts, measures and methods for demographic analysis. Yamarone’s (2017) handbook focuses on economic indicators and Frumkin (2015) explores how economic and financial indices are constructed. Transformative Data-Shaping Strategy The Transformative strategy focuses on helping you to transform one form of data into another form. This data-shaping strategy often operates as an intermediate strategy between the data gathering strategies that yield your original data and your data analysis processes but can also play a role in more directly recording observations. The Transformative strategy encompasses the use of formalised recording protocols, templates, schedules, mathematical formulas or devices to record or code observations or segments of text or other multi-media content. Often, when data are gathered using another strategy, they may not immediately be in the most appropriate form or type for analysis and interpretation. This can happen under virtually any pattern of guiding assumptions and gives rise to the need to ‘re-shape’ the data to maximise their utility for your research. The Transformative strategy can be useful for supporting research conducted within an Action, Evaluation, Developmental Evaluation, Survey, Cross-Cultural, Indigenous, Explanatory, Exploratory or Descriptive research frame employing virtually any MU configuration. However, if you are creating a data coding template or protocol, you will typically need a distinct development phase in the research implemented in a sequential MU configuration. This strategy is almost always combined with the Structured observation strategy and/or Immersive experience-focused strategy (to develop/implement observational protocols/recording forms). It is also used in combination with the meta-analysis or meta-synthesis approach to the Textual artefact-based strategy to systematise, at least partially, the coding of journal articles and other documents. Finally, it is used in combination with the Textual and/or Multi-media artefact-based strategies to systematise, at least partially, the coding of texts, videos and other types of multi-media evidence. As well, the Transformative

14.3

Strategies for Structuring People’s Experiences

637

strategy is used to manage the process of producing transcripts from recorded interviews or other multi-media recordings. What defines the need for the Transformative strategy depends upon your guiding assumptions, the type(s) and form(s) of your original data, your research needs (including analytical requirements) and the stories you are trying to convey. There are four potential transformational pathways you can follow: transforming quantitative data into another quantitative form; transforming quantitative data into a qualitative form; transforming qualitative data into another qualitative form; and transforming qualitative data into a quantitative form. More than one pathway may be followed in any specific research endeavour, depending upon your needs. Quantitative ! Quantitative This pathway for the Transformative strategy is implemented when one or more quantitative measures, gathered in one form need to be transformed or recoded into other quantitative forms. Thus, data type is not changed, but the form and nature of measurements is changed. It is most useful for research guided by the positivist pattern of guiding assumptions. This pathway for transformation can be used to alter the distributional shape of a quantitative measure when its distribution does not match what is assumed to be the case for a specific statistical analysis procedure (e.g., non-normal data can be transformed using mathematical operations, such as log10, reciprocal, arcsin or square root, to produce a more normally-distributed variable). If you want to use nonparametric statistical procedures, such procedures alter the scale of measurement for the dependent variables being analysed by reducing them to ordinal ranks. Another common version of this pathway is to mathematically combine individual quantitative measures in specific ways to make new measures or indicators (as when a factor or scale score, a financial ratio or economic index is produced). In all cases, this pathway produces new quantitative measures that you can then use for building statistical models and testing hypotheses. If you follow this pathway, you need to be very clear as to your reasoning, which you can record in your research journal for later reference. Quantitative ! Qualitative (Qualitising) In this pathway for the Transformative strategy, you change data type as well as form and nature. Here one or more quantitative measures are transformed into a meaningful or higher-order qualitative category or thematic system or are used to define meaningful groups. This is often done when simplified descriptions/ interpretations for more complicated measures are desired (e.g., low, medium and high categories). The new qualitative categories may reflect inferences made about participants who are positioned contiguously along the quantitative scale of the original measure (e.g., on a series of questions about voting patterns and policy preferences, a researcher may classify a participant as being liberal/left- or conservative/right-leaning). For knowledge discovery purposes, there are certain statistical procedures, such as cluster analysis, neural network analysis or data mining (pairing with the

638

14

What Data Gathering Strategies Should I Use?

Archival/Secondary artefact-based strategy), that you can use to discover hitherto unknown categories of participants using a set of quantitative measures and, once discovered, you can create qualitative descriptions of each cluster or category using information contained in the patterning of their quantitative measurements. In general, qualitising is a tactic for enhancing interpretability and meaningfulness from the point of view of the research reader/user. It may also pave the way for clearer policy development and decision making in instances of applied or transdisciplinary research, because social policy can be targeted more readily to simply defined groups rather than to a heterogenous group of individuals. Qualitative ! Qualitative This pathway for the Transformative strategy is implemented when qualitative data are transformed or coded into a meaningful or higher-order qualitative category or thematic system (e.g., pairing with interaction-based strategies, or Textual or Multi-media artefact-based strategies; see also discussions in Chaps. 20 and 21). This pathway is often used in research guided by the interpretivist/constructivist or other non-positivist pattern of guided assumptions. A qualitative coding system may be emergent as in some forms of grounded theory, where subsequently gathered qualitative data are coded into the emergent categories (e.g., data-driven coding) as well as into any new categories that make sense. Additionally, data may be categorised into new higher order categories (what Saldana describes as a form of second-cycle coding). For some forms of qualitative research, you might use a pre-designed category system (often called a structural codebook or coding template, see Guest, MacQueen, & Namey, 2012, pp. 55–60) to code important contextual aspects/ features of texts to be analysed (e.g., attributes/characteristics of speaker(s) in interviews or focus groups, place, time, topic or question). A preliminary conceptual coding system may also be used to kickstart your coding process or to ensure that specific ways of looking at the data are not overlooked (e.g., a process termed concept-driven coding, see Schreier 2012, pp. 85–87, or template-guided coding, see King 2004) as long as you don’t assume the coding system to be fixed (i.e., it can evolve, expand or contract as you dig deeper into the data). These are types of deductive approaches to qualitative data management (distinguishable from inductive or data-driven approaches, see Schreier’s, 2012, Chap. 5 discussion of coding ‘frames’) where coding categories are at least partly formulated by the researcher prior to commencing analysis and may be theoretical, experiential, literature-based or logical in origin; such coding typically forms the preliminary stage of qualitative analysis, i.e., the first stage of data reduction. In the Indigenous research frame, structural codebooks and any preliminary apriori data coding and their interpretations should be discussed/shared with relevant Indigenous people/ elders. The qualitative ! qualitative transformation pathway also relates to the preparation of interview transcripts from digital or multi-media recordings (i.e., aural or multi-media data are transformed into textual data). When you create new texts via some transcription or translation process, you must first decide what kinds

14.3

Strategies for Structuring People’s Experiences

639

of information you wish to have represented in the transcripts (e.g., verbatim speech, stutters, ums, pauses, laughs, over-speaks, emotional emphasis, inaudible or unclear utterances) and how that information should be represented. Note that paralinguistic and chronemic features of talk can be captured from raw audio recordings, but not kinesic or proxemic features (which could, however, be captured in a video recording, yielding multi-media data). Transcript preparation needs to be consistent, to avoid problems with transparency and authenticity, and you need to ensure that all of what you intend to be recorded in the transcript is represented (more discussion of this process appears in Chap. 20). This means that you will need to double check each transcript against the original raw data recording. The qualitative ! qualitative pathway of transformation may also be useful for research guided by the positivist pattern of assumptions. For example, in content analysis or meta-analysis (pairing with the Textual artefact-based strategy), you may classify written responses or segments of research articles into pre-designed sets of categories. A key feature of the process here is that you create the coding protocol/schedule/template/codebook and pre-test it for validity and usability prior to commencing data gathering. Typically, in content analysis or meta-analysis, you would have more than one person code the same texts using the same coding protocol/schedule/template/codebook to facilitate checking for inter-rater (or inter-coder) reliability. Qualitative ! Quantitative (Quantitising) This pathway for the Transformative strategy can be implemented when qualitative data are transformed into counts, numerical ratings or rankings (e.g., rating the content of utterances according to the level of hostility evident in them; counting the number of phrases in transcripts that have been coded against a specific theme or category, thus pairing with the Textual or Multi-media artefact-based strategies). The interview schedule used in a structured interview constitutes one type of data transformation (vocalised answers to recorded quantitative answers) protocol, one which offers specific prompts for the data to be requested and recorded. For content analysis or analysis of open-ended questionnaire items, this transformation often follows original qualitative ! qualitative transformations to go one step further and produce quantitative measures that are appropriate to use in statistical tests and models. In the Manipulative and Non-manipulative experience-focused strategies, events, groups, treatments or conditions are often simply qualitatively represented as group memberships. However, for modelling purposes, these qualitative classifications need to be transformed into quantitative variables having the most appropriate and statistically interpretable form in the context of the model you are constructing and testing. In such cases, the Transformative strategy works by recoding the group membership into specific numerical values (e.g., using dummy coding, effect coding, orthogonal contrast or orthogonal polynomial coding—see, for example, the discussions in Cooksey, 2014, Fundamental Concept VI). The qualitative ! quantitative transformation is also implemented in combination with the Systematic observation-based strategy, guided by the positivist

640

14

What Data Gathering Strategies Should I Use?

pattern of guiding assumptions. Through the design and use of an observation recording protocol, qualitative, even holistic, observations can be numerically rated or counted for purposes of statistical analysis. When designing such a protocol, ease of use (minimising the risk of recording errors) for the observer is essential to ensure, especially if they are required to make judgments, as when rating deep/ latent meaning in observations. Efficiency of use also is essential to ensure for observational coding protocols in situations where observations are obtained under tight schedules or in rapidly-changing situations. Efficiency (and in some cases accuracy) can be improved if the structured observational protocol can be supported or automated using technology. It is important to establish the construct validity of the protocol, where your goal is to amass evidence showing that the protocol measures the intended dimensions of observed behaviour and/or the content validity of the protocol, where your goal is to ensure that the observational/coding dimensions tap relevant performance/behaviour aspects. In the Cross-Cultural research frame, care needs to be taken to ensure, as far as possible, that construct measurements to be recorded/coded using the protocol have equivalent meanings across the cultural boundaries. In the Indigenous research frame, the same concern applies, but with the potential additional implication that the content and interpretations of the protocol may need to be negotiated with/vetted and approved by appropriate Indigenous people/elders as well as shown to respect relevant cultural protocols (Chilisa, 2012, p. 52, illustrates the problems that can emerge if researchers do not follow this pathway so as to ‘decolonise’ their typical approach). For observational protocols, inter-rater (or inter-coder) reliability is important to establish. You may also need to conduct practice sessions with coders/observers so they can achieve proficiency in the use of the protocol, especially under dynamic conditions or situations where many observations must be made in a short span of time. There are several systems you can use to effectively support the Transformative strategy, including: • using your research journal for recording notes, specific details and reflections throughout the transformative processes you implement; • using transcription software or professional transcriber/transcription service to produce interview texts (see the additional discussions of the transcription process in Chap. 20); • using tablet, laptop, smartphone or web-based software for recording/storing data and codes (e.g., Noldus Pocket Observer, see http://www.noldus.com/theobserver-xt/pocket-observer; Mangold Obansys, see https://www.mangoldinternational.com/en/products/software/mobile-observation-with-obansys); Psychsoft ObservePrime for Android 2.0, see https://download.cnet.com/ Observe-Prime-2-0/3000-20411_4-77421741.html); and/or • using spreadsheet, statistical analysis and/or qualitative analysis software for setting up, recording and storing coding systems and codes (e.g., Excel, SPSS, Stata, MAXQDA, NVivo).

14.3

Strategies for Structuring People’s Experiences

641

Guest et al. (2012, pp. 55–64), King (2004), Schreier (2012, Chap. 5) each discuss structural codebooks/templates/frames for qualitative data recording, coding and analysis. Silvester (2004) reviews the attributional coding of qualitative data. Bryman and Bell (2015, Chap. 13), Cohen et al. (2011, Chap. 17), Lipsey and Wilson (2001, Chaps. 3 and 4) each discuss coding for content/text analysis, meta-analysis and meta-synthesis. Cohen et al. (2011, pp. 459–464), Gray (2014, pp. 422–426), Phellas, Bloch and Seale (2012, pp. 198–202) discuss coding schedules for structured observation data recording. Cohen et al. (2003, Chaps. 6 and 8) and Cooksey (2014, Fundamental Concepts II and VI and Procedures 7.8 and 8.2) provide conceptual overviews of quantitative data coding and transformation processes. Generative Data-Shaping Strategy The Generative strategy encompasses a set of approaches for creating data, using specific parameters and processes, to test social and behavioural models. Such models are evaluated using computer simulation rather than gathering data from human participants. While it might appear that creating data is ethically problematic, the difference here is that you are not ‘faking’ research data to support a research hypothesis, which is ethically reprehensible, you are openly and transparently creating artificial data so as to explicitly test some model or hypothesis of interest, using computer simulation processes (see, for example, Axelrod, 2007, and Gilbert & Troitzsch, 2005; note that computer simulations differ from the types of live-action simulations used in the Immersive experience-focused strategy, which do involve human participants). Use of the Generative strategy is consistent with the positivist pattern of guiding assumptions and can follow two different pathways. The first pathway is artificial data generation, such as Monte Carlo research, which involves the production of data, using random sampling algorithms, that exhibit specific statistical properties in order to test hypotheses about their behaviour. The second pathway is computational modelling, such as agent-based modelling, which involves the creation of a computer-simulated environment, based on theory, where assumptions, variables and parameters can be explicitly controlled and their effects on simulated entities precisely tracked. The Generative strategy is typically used in the Explanatory or Exploratory research frames. In the Explanatory frame, computer simulations are used to test theoretical propositions. In the Exploratory frame, computer simulations provide preliminary insights into what might be expected when human participants become involved in the research. The Generative strategy may be implemented within a single MU configuration, especially in the Explanatory research frame to test theoretical propositions. Furthermore, a sequential MU configuration could be used in computational modelling circumstances to develop and run simulations in the initial MU and then verifying and validating the simulation outcomes, using data from real participants/organisations, or from other models (a process called ‘docking, see, for example, Olaru, Purchase, & Denize, 2009). A sequential MU configuration might also be used within the Exploratory research frame, normally with the computer

642

14

What Data Gathering Strategies Should I Use?

simulation as the initial MU with the subsequent MU perhaps moving out into the field or into a laboratory to study real human participants, groups or organisations. Monte Carlo Pathway The Monte Carlo pathway (named after the famous casino in Monte Carlo, Monaco) involves sampling via random data generation (see, for example, Carsey & Harden 2013; Mooney 1997). With the Monte Carlo pathway, your goal is to decide which data distribution(s) and associated parameters are to be used to define the population(s) to be randomly sampled and the nature of the samples to be generated (e.g., how many observations to generate, how many groups to create, which data generation parameters to systematically vary). All Monte Carlo methods depend upon random number generators, which most statistical packages make available. Categorical as well as continuous data can be created using the Monte Carlo approach and any number of distributions can be simulated (e.g., uniform, normal, binomial). Once you have created the artificial dataset(s), you can then analyse them as if they were empirically gathered data. One major purpose for generating random data involves conducting experimental research explicitly to study the behaviour of data, generated with different characteristics, assumptions and/or generating algorithms, and seeing how those differences impact on the conclusions one might draw, using different analytical approaches (pairing with the Manipulative experience-focused strategy). As an example, Milligan (1981) conducted a Monte Carlo investigation to randomly generate 108 artificial data samples in order to evaluate different four cluster analysis methods and 30 clustering criteria and stopping rules. In this situation, Milligan could not feasibly gather data that had the appropriate properties he required to implement his desired experiment and test his hypotheses, so he artificially created the datasets he needed. The advantage of Monte Carlo research is that you can control virtually every aspect of the research from start to finish. What you need to be careful to do, though, is to be thoroughly transparent about all the decisions you made through the data generation process and anchor your choices to theory and/or the literature. Computational Modelling Pathway Computational modelling (see Taber & Timpone 1996, for an overview) allows you to set up and test theoretical models about the social, political and physical and/or economic world without needing to gather empirical observations. In some instances, the theoretical models are so complicated that it is not feasible or too costly for you to gather the necessary empirical data. However, you can partially circumvent this limitation by creating a simulated computational environment. Computational modelling may also allow you to test certain types of hypotheses that are infeasible to test in the real world. For example, suppose you want to explore the potential economic, housing, education and employment implications of different government policies for dealing with refugees seeking asylum or immigrant status. It would be unreasonable and probably unethical for you to conduct a social field experiment where you put a specific policy in place and gather data to evaluate the impact(s) of that policy. However, if you could construct a convincing

14.3

Strategies for Structuring People’s Experiences

643

computational model that appropriately captured the dynamics and important variables and relationships associated with the problem, then your social experiment could effectively be run within that computational or virtual world. Computational modelling generally involves structuring a theoretical model, generating the data needed to monitor the behaviour of the model and providing information useful for the validating both the model processes and its outcomes. We will examine two broad approaches here: dynamic systems modelling and agent-based modelling. Dynamic Systems Modelling System dynamics is an applied research discipline, tracing its lineage back to Jay W. Forrester at MIT in the 1950s (there is even a journal, devoted to systems dynamics research called System Dynamics Review, that traces its lineage back to Forrester). From that beginning, dynamic systems modelling evolved as a way of developing and testing complex simulation models. In the 21st century, dynamic systems modelling became integrated with systems thinking principles as well as complexity theory and new mathematic approaches for modelling complex nonlinear systems (see discussions in Maani & Cavana, 2007; Sterman, 2000). Such models are typically so complex that they cannot be constructed and implemented without the support of computer hardware and software. Systems dynamics models may be determinist in emphasis (where a system of equations calculates exact quantities for each step of the simulation process) or stochastic (where simulated random data are used as part of the modelling process; this is where Monte Carlo methods can play a role). Dynamic systems modelling tries to capture and simulate the behaviour of a defined system over time. The models are dynamic because they depend upon feedback loops over time just as real human systems do, using the concepts of stocks (supplies of necessary resources) and flows (movements of resources over time). The basic stages for constructing a dynamic systems model are: (1) develop a theory or conceptual framework for the model and its associated processes and anticipated outcomes (might require pairing with the Visualisation participant-centred strategy); (2) use the resulting framework to guide the development of the model itself (usually supported by an integrated computer programming environment such as the web-based package Insight Maker or computer software packages like STELLA, iThink, Vensim, or Powersim); (3) evaluate the behaviour of the model by running simulations and experiments; and (4) refining the model based on learning from Stage 3 as well as perhaps from external validation research (comparing model outcomes to observable outcomes in real systems, where and when such data can be sourced). Figure 14.16a illustrates what a dynamic systems model, constructed using the Insight Maker web-based platform, looks like (enclosed in the dashed frame) and how the behaviour of the system, when simulated over time, can be mapped using a behaviour over time graph (window at lower right). The model context is training to improve the speed and accuracy of quality control inspectors in factories and tries to capture how training effectiveness and training volume as well as psychological resilience and fatigue

644

14

What Data Gathering Strategies Should I Use?

(a) Simulation of a behavioural system using Insight Maker web-based software

(b) Simulation of a behavioural system using agent-based modelling via Netlogo software Fig. 14.16 Illustrations of the Generative strategy in action: a illustrates dynamic systems modelling; b illustrates agent-based modelling

14.3

Strategies for Structuring People’s Experiences

645

interact over time to influence job performance. The sliding bars on the right side of the large window provide the researcher with ways to alter some of these key variables and then replay the simulation, allowing for controlled experimentation (pairing with the Manipulative experience-focused strategy). What is important to understand about building these types of models is that they are critically dependent upon the nature and quality of the theory that gives rise to the model. The difficult part is identifying all the control parameters you will need to make the model workable as well as all the key variables and their functional relationships and feedback loops. Unless you do this well, the simulation will simply produce nonsense. What this often means is that you need to couple a deep exploration of the relevant literature with observations of the live system in action and/or interviews with experts. This implies using a sequential MU configuration with early MUs providing the contextual knowledge you need to construct and test the model in a subsequent MU. You will also likely need training not only in dynamic systems modelling itself, but also in using the computer environment in which you will program the simulation (note that modern-day software platforms offer graphical interfaces so that you can construct models without actually writing a computer program). Along the way, you will need to document all your thinking in your research journal, including the reasoning behind all the choices and assumptions you have made. When you are writing this type of research up, it will be critical for you to spell out all of the assumptions you needed to make in order to get the model working, how you defined and quantified all variables in the model and how and why you set up functional relationships and feedback loops that linked the variables over time. Bringing all this together means that the internal coherence meta-criterion becomes extremely important to attend to in a dynamic systems modelling investigation. Your story must be convincing but also must contain sufficient openness and extent of detail so that another researcher would be able to replicate what you had done. Agent-Based Modelling Agent-based modelling is another form of computational modelling (see, for example, Gilbert, 2008; Gilbert & Troitzsch 2005, Chaps. 8 and 9; Miller & Page, 2007; Railsback & Grimm, 2012). Agent-based modelling focuses on models involving simulated agents (e.g., individual people, teams, departments, organisations, nations, cars, planes, animals, trees, molecules) interacting with each other dynamically through time in a virtual world. This virtual world is controlled by model parameters and simulation conditions that you identify as researcher. Interactions between agents may constitute information flows (verbal, nonverbal, observational, decisions and choices, wealth) as well as relational connections and qualities (formal, informal, cross-cultural, conflict, trust, prior history, flocking, disease, culture). Gilbert (2008) showed that agent-based modelling has been useful for modelling collaboration and competition in industrial networks, social influences on consumer behaviour, team interactions (e.g. interactions of interdependent teams), opinion dynamics (how opinions spread through a population), urban behaviour (e.g. neighbourhood growth; segregation dynamics) and supply chain

646

14

What Data Gathering Strategies Should I Use?

dynamics (inventory management and movement between and amongst various organisations in a supply chain). Railsback and Grimm (2012, pp. 3–4) discuss an agent-based model, focusing on rabies control in Europe, which permitted explorations of different vaccination strategies and yielded predictions that led to a specific successful strategy being adopted. Agent-based modelling requires very precise theorising/conceptualisation and mathematical modelling within a computer support system, the most popular model development interface for the social sciences is probably NetLogo (Railsback & Grimm, 2012; Wilensky 1999). Figure 14.16b shows an illustrative agent-based model focusing on traffic movements in a city grid, where intersections were controlled by traffic lights (see Wilensky, 2003, for more details on how this model was developed). Agent-based computational models, in the NetLogo environment, use ‘turtles’ as the programmable entities that can be given attributes and assigned rules (often conditional) for how to move and behave in the simulated environment. In Fig. 14.16b, the ‘turtles’ are cars on the roads whose number and speed can be controlled as can the state of each traffic light. As the simulation runs through time, graphs plot key variables for monitoring the state of the environment and its agents. An agent-based model is conceptually and technically challenging to construct. It will likely require extensive background work in the literature, perhaps incorporating information from experts and potential users to verify the model’s conceptualisation, relevant variables and control parameters and its embedded processes. You will likely have to make simplifying assumptions about the ‘world’ you are about to construct, which concepts and relationships to incorporate and which you cannot include, what parameters are needed, what relationships need to be modelled, how variables should be represented/measured and what kinds of outputs you want the model to produce. Your assumptions will shape and delimit the specifications and characterisation of your model; any computational model is only as good as its underlying assumptions and model specifications. This imposes an extra burden of transparency on you, as researcher, to be very clear about the entire process for building and testing your model from conceptualisation to validation. Furthermore, your model’s performance will be critically dependent upon your anchoring assumptions and the choices for controlling parameters. This means that thorough experimentation, sensitivity testing and validation (wherever possible) of your computational models is essential, if the model is to be anything other than an academic exercise. In some cases, external validation of model outcomes may be difficult or impossible to obtain because you are unable to access the data you would need to do so. Docking has emerged as one way of trying to cope with this potential limitation, but to be successful, docking requires an appropriate alternative model to compare outcomes with—something that may not exist. What all of this means is that the internal coherence as well as extensional reasoning become extremely important meta-criteria to attend to. From an internal coherence perspective, you need to record sufficient information in your research journal that will facilitate you not only telling the story you need to tell, but also to facilitate potential replicability by others. If you create an agent-based model, but it does not resemble the system you are trying to model, either in composition or behaviour or

14.3

Strategies for Structuring People’s Experiences

647

both, then value for learning will be minimal. From an extensional reasoning perspective, empirical verification and validation of your model against other models as well as against the real world is important. This means it may become important for you to share your model with other researchers, so they can work with it and potentially evaluate it against other models. Thus, sharing your model may create docking opportunities. There are several systems you can use to effectively support the Generative strategy, including: • using your research journal for recording notes and decisions throughout your model building and simulation/testing processes and keeping those choices close to the literature, theory and/or logic on which they were based; • using a computer development and support platform such as Insight Maker (http://www.insightmaker.com/), Vensim (http://www.vensim.com/), STELLA (https://www.iseesystems.com/store/products/stella-architect.aspx), iThink (https://www.iseesystems.com/store/products/ithink.aspx), or Powersim (http:// www.powersim.com/) to help develop and test dynamic systems models; • using NetLogo (https://ccl.northwestern.edu/netlogo/index.shtml; there is a web version available as well, NetLogo Web, http://www.netlogoweb.org/) to help construct, test and evaluation agent-based model simulations (note that NetLogo is open source software, available free to researchers, and comes with an extensive library of pre-built models for learning purposes). Carsey and Harden (2013), Mooney (1997) and Wilcox (1997) discuss the potential uses for Monte Carlo simulation and other types of mathematical simulations. Sterman (2000), Richmond (2004) and Maani and Cavana (2007) discuss and illustrate system dynamics modelling and how it is linked to systems thinking (Richmond’s book is closely linked to the STELLA software package). Cohen et al. (2011, Chap. 19), Gilbert and Troitzsch (2005) and Axelrod (2007) discuss concepts and illustrations associated with virtual worlds and simulations. Gilbert (2008) reviews agent-based modelling and Railsback and Grimm (2012) provide a very useful resource book for learning to use NetLogo for constructing agent-based models. Miller and Page (2007) discuss the possibilities associated with using agent-based modelling to model complex adaptive systems and Olaru, Purchase and Denize (2009) demonstrate how docking can be used to validate an agent-based model.

14.3.2 Experience-Focused Strategies Experience-focused strategies create and implement or examine/evaluate a specific structuring or type of experience or activity for research participants. In essential, they establish the context within which or about which data gathering will focus. Under the positivist or critical realist pattern of guiding assumptions, you set the

648

14

What Data Gathering Strategies Should I Use?

stage for making externalised causal inferences by manipulating, where possible, the contextual environment or tasks in which participants are providing/generating information and/or data. Manipulation of conditions is a manifestation of control by design and allows you to ensure that causal variables change before effect variables are observed. If contextual manipulation is not possible, then causal variables may be measured, but at the cost of lessened capacity to infer causality. In some cases, you may not be able to create or manipulate the conditions, events or situations that participants experience, another agency may play this role. Other agencies encompass social or organisational/institutional agencies, government entities or natural environmental forces. It may also be the case that a social, cultural or physical event forms the basis of participants’ experience. In these latter instances, your researcher control over context is minimal, consequently weakening the strength of causal inferences even further. Experience-focused strategies thus range from non-manipulated experiences (imposed by agencies or forces other than the researcher) to completely manipulated experiences (where you control and manipulate all conditions) to structured experiences (where you provide a framework for participants to work within) to immersive simulation experiences (where you or another agency create an artificial environment for participants to experience). Under the positivist or critical realist pattern of guiding assumptions, experiences are structured in as tightly a controlled environment as possible (an experiment or quasi-experiment), if externalised causal inferences are to be established on the strongest grounds. This may be independent of any specific real environment (as with a laboratory study or a simulation). The extent of control over context and over experiences is key to the strength of causal inferences. Causally-oriented research may also capitalise on events unfolding the real-time in participants’ natural context (a non-experiment), but since such events are not under researcher control, causal inferences will be greatly weakened. When your research is guided by the positivist pattern of assumptions, experience-focused strategies are always paired with one or more data-shaping strategies. Single and simultaneous MU configurations may be used where cross-sectional samples are assembled and random assignment of participants to experimental conditions or treatments is possible. Longitudinal intervention time-aligned MU configurations (e.g., pre-test-post-test configurations with interventions) are also commonly used with experience-focused strategies. A sequential MU configuration may be used where a repeated-measures structuring of experiences, that is not time-aligned or time-dependent, is desired (such as taste-testing different brands of cola). Depending upon your goals, experience-focused strategies may be employed under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions. However, the experience is not a matter of positivist-like contextual control in pursuit of externalised causal inferences; rather it is a matter of inviting participants to experience or work within a specific context to convey/share their perspectives about those experiences with you. Fully randomised experiments are very seldom employed, but simulations, games, structured tasks (such as those involved in organising frameworks or process-tracing) can be quite useful

14.3

Strategies for Structuring People’s Experiences

649

structuring strategies for unpacking subjective experiences. This can be particularly useful in hierarchical embedded MU configurations or exploratory sequential MU configurations. Often, especially within the Case Study or Evaluation research frame, you may be interested in participants’ perspectives on natural (i.e., uncontrolled) contextual events they have experienced. Whatever experience-focused strategy is employed under interpretivist/constructivist or other non-positivist pattern of guiding assumptions, it will almost certainly be paired with another strategy that yields qualitative data, especially interaction-based strategies, the Participant observation-based strategy, participant-centred strategies as well as coupled with the Transformative data-shaping strategy. Manipulative Experience-Focused Strategy The Manipulative strategy is the prototypical strategy in the social and behavioural sciences for structuring people’s experiences in a research context under the guidance of the positivist or critical realist pattern of assumptions. Generally, the goal is to gather quantitative or quantifiable data to justify defensible inferences about the existence of external causal relationships. Under the positivist pattern of guiding assumptions, this strategy is always combined with one or more data-shaping strategies. In a true experiment, you manipulate and randomly determine all the conditions/events that participants experience, control for as many alternative plausible explanations and extraneous variables as possible and then measure the effects of those conditions/events on aspects of participants’ thinking and/or behaviour. The true experiment is the strongest data gathering strategy for making defensible inferences about external causation and is most likely to be achievable in a laboratory setting. Your ability to randomly assign participants to conditions/events is what differentiates a true experiment (where random assignment is used) from a quasi-experiment (where random assignment is not possible). Figure 14.17a shows examples of several types of true experimental design; Fig. 14.17b, on the right side of the diagram, shows more elaborate experimental designs that incorporate explicit control over extraneous variables. Note that all the true experimental designs shown incorporate one or more control groups to establish a comparative standard against which to gauge experimental treatment or condition effects and you would randomly determine which participants would experience those conditions (e.g., a placebo condition in a clinical trial for a drug or a ‘normal’ environment for people). If you are not able to fully determine which condition(s)/event(s) participants experience (such as when a demographic characteristic of the participant like gender or ethnic background is a causal factor of interest or when something or someone in the participants’ context prevents contextual manipulation), the experiment becomes a quasi-experiment. Thus, with a quasi-experiment, contextual manipulation can only be partially achieved, making it a weaker data gathering approach for drawing defensible causal inferences, relative to a true experiment. A quasi-experiment is more likely to be used in field settings or naturalistic conditions, where you cannot control contextual situations (e.g., which class a student is in or what company or department an employee works for). A quasi-experiment

650

14

What Data Gathering Strategies Should I Use?

a) Illustrative True Experimental Designs [R = participants randomly assigned to groups or conditions; X = intervention or treatment imposed by researcher; ~X = intervention/treatment-equivalent control experience provided by researcher O = observation occasion or point in time when dependent variable(s) are measured]

Experimental Group: Control Group:

Pre

O1 O1

Posttest only control group design

Pre

Experimental Group: Control Group:

Post

Pre

R R

Post

X

Experimental Group 1: Control Group 1: Experimental Group 2: Control Group 2:

R R R R

Latin Square design (controls for 2 nuisance variables, defining the rows and columns of the design)

Post

R R

Solomon four group design

b) Other Types of Experimental Designs

O2 O2

~X

X

O2 O2

~X Pre

Picture Frame Type

O1 O1

X

O2 O2 O2 O2

~X X

~X

Indep. Var 2

Factorial posttest only control group design Indep. Var. 1

Control Group

Experimental Group

Control Group

R

R

Experimental Group

R

R

c) Illustrative Quasi- Experimental Designs [Random assignment of participants to groups or conditions is not possible for at least one independent variable and/or at least one independent variable is measured, not manipulated] Non-equivalent static group design

Pre

Post

Experimental Group: Control Group:

X

~X

Non-equivalent control group pretest-posttest design Experimental Group: Control Group:

Wood frame

No frame

Gold frame

Decorativ e frame

Lighted from above

Painting B

Painting A

Painting C

Painting D

Lighted from below

Painting D

Painting C

Painting A

Painting B

Lighted from left

Painting A

Painting D

Painting B

Painting C

Lighted from right

Painting C

Painting B

Painting D

Painting A

Post Lighting Angle

Pretest-posttest control group design

Pretest-Posttest Control Group Design

Pre

O2 O2

Analysis of Covariance Design (statistically controls for the influence of a covariate on a dependent variable) Control Group Dependent measure

Covariate measure

Experimental Group 1 Dependent measure

Covariate measure

X

X

Experimental Group 2 Dependent measure

Covariate measure

X

Statistically remove (X) any association of the Covariate measure with the Dependent measure; then compare the three groups on the adjusted Dependent variable values

Post

O1 O1

X

~X

O2 O2

Interrupted time series (or single participant) design [if there is no intervention, X, then this becomes a standard time series design]

Fractional Factorial Design (specified fraction (½ in this example) of a fully factorial design, allowing estimation of certain effects while sacrificing information on other effects; reduces the size of the experimental design)

O 1 O 2 O 3 O 4 O 5 X O 6 O 7 O 8 O 9 O 10

Room Temp Normal

Treatment

Dependent Measurement

Single participant ABAB time series design [Alternating no treatment (A = baseline = control comparison level) with treatment phases (B)] Baseline Treatment Baseline

Humidity Low

High Humidity 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Measurement Occasions Time Line

Room Temp Hot

Background Noise

Loud Machinery

Background Noise

Loud Machinery

Light Level Normal

Participants experience

Not experienced

Not experienced

Participants experience

Light Level Bright

Not experienced

Participants experience

Participants experience

Not experienced

Light Level Normal

Not experienced

Participants experience

Participants experience

Not experienced

Light Level Bright

Participants experience

Not experienced

Not experienced

Participants experience

Factorial posttest only no control groups design Measured Indep. Var. 2

Measured Indep. Var 1

Category A

Category B

Category A Category B

Fig. 14.17 Synoptic snapshots of different experimental and quasi-experimental designs for the Manipulative strategy: a true experimental designs; b other more specialised experimental designs; and c quasi-experimental designs

also emerges in situations where independent variables (IVs) are defined by demographic or social characteristics of participants, such as their gender, age, socioeconomic status, income, years of experience or education level, ethnic background, religious background, since participants cannot be randomly assigned to a category of such IVs. Figure 14.16c, in the bottom left of the synopsis diagram, shows examples of several types of quasi-experimental designs. For some designs,

14.3

Strategies for Structuring People’s Experiences

651

control groups can be used, but you don’t have the ability to randomly assign participants to such groups. Both true experiments and quasi-experiments implement what is termed a ‘causal comparative approach’ (for example, see Johnson, 2001), generally involving IVs that define different ways of grouping participants for comparative purposes (e.g., different treatments or programs, different environmental conditions, different categories of a demographic variable such as gender, age or ethnic background). The group comparisons provide a pathway for teasing out cause-effect relationships between group membership and patterns of measurements on one or more dependent variables (DVs). The Manipulative strategy can be especially useful in the Explanatory research frame, but can also be very useful in the Evaluation, Exploratory or Survey research frames. It may also be useful in the Cross-Cultural research frame, but this would require much more effort and commitment of resources if cross-cultural comparisons are to be attempted. At a small scale, the Manipulative strategy could be useful in the Action research frame to evaluate the impact of specific local contextual interventions. It works well in intervention time-aligned longitudinal MU configurations as well as in single MU configurations and case-based MU configurations. For certain kinds of research, an experiment or quasi-experiment may be used in the second phase on an explanatory sequential MU configuration, following an initial phase which might use, for example, focus-group interviews (as is done in some types of marketing or consumer behaviour research, for instance). Tracking influences/changes over time (e.g., within intervention time-aligned or developmental time-aligned longitudinal MU configurations) leads to repeated measures experimental designs. Such designs are often used for double-blind randomised controlled trials (RCT), which have been considered the gold standard of medical research and clinical trials (see, for example, Kaptchuk 2001), especially where placebo (i.e., a non-treatment control condition) control has been implemented. For positivist-oriented causal-comparative research, the Manipulative strategy is always combined with one or more data-shaping strategies, Systematic observation-based strategy and/or the Structured interview interaction-based strategy. The meta-analysis approach within the Textual artefact-based strategy can be used in conjunction with an experimental design as can the Generative data-shaping strategy (especially for Monte Carlo simulation research). For an ethnomethodological experiment, synergies are typically achieved using the Participant observation-based strategy. In causal-comparative research approaches, the participant’s role is passive in that they participate in the experiment/quasi-experiment and simply provide the measurements that you select or design. Any information they volunteer or questions they raise (with respect to the research environment or conditions they are experiencing) on their own initiative is typically ignored. Any research, guided by the positivist or critical realist pattern of assumptions, that intends to draw inferences about externalised cause-effect relationships is necessarily involved in testing theoretical propositions and deduced hypotheses in a specific context, typically using a statistical model of some description. However, the statistical model, in and of itself, does not permit causal inferences to be drawn.

652

14

What Data Gathering Strategies Should I Use?

To defensibly draw conclusions about external causation, you need to meet three necessary and sufficient conditions: 1. Temporal priority of causes, where causal variables precede effect variables in time, that is, causes must change before effects can be observed to change. This is usually achieved through you, as the researcher, designing or arranging circumstances where causal variables are manipulated or measured before effect variables are measured (i.e., contextual design), in the context of an intervention time-aligned longitudinal MU configuration, for example. 2. Covariation of causes and effects, where if/when a causal variable changes values, the theorised effect variable is observed to change (thus, the cause and effect must co-vary). Here is where statistical models are most useful. They are constructed to demonstrate covariation between cause and effect variables (e.g., when the causal variable increases in value, the effect variable tends to increase or decrease in value). 3. Ruling out alternative plausible explanations, where other potential causal influences are removed as possible explanations for the covariations that have been observed. This is the central concern of the internal validity research quality criterion. How this happens is that you prevent a construct or variable that you are not interested in as a causal influence from changing at the same time as the construct or variable that is of interest as a causal influence (thereby avoiding a problem called confounding). Unless alternative plausible explanations for observed cause-effect covariations can be defensibly dismissed or actively negated, you cannot draw unambiguous causal inferences. This is the causality condition that the process of contextual control is primarily concerned with. The concept of control is central to research conducted under the positivist and related patterns of assumptions as it is through the exercise of control that you can minimise the most critical of alternative plausible explanatory influences. ‘Most critical’ is a judgment you make in trying to anticipate potential alternative plausible explanations which could interfere with achieving convincing outcomes. Sound control practices are all about choosing sensible and defensible approaches, in full recognition of what you stand to gain or lose by making each choice. From a statistical model building and testing perspective, contextual control creates opportunities to remove systematic influences from estimates of model errors, ideally reducing model errors to only unpredictable and uncontrollable random influences (see Chap. 18 for more on this issue). Many control approaches work to improve internal validity (i.e., capacity to unambiguously infer cause and effect) while others work to improve external validity (i.e., capacity to generalise). Table 14.2 provides more clarifying detail about processes of control in experiments and quasi-experiments (the Appendix: Clarifying Experimental/ Quasi-experimental Design Jargon provides a table that helps to clarify some of the jargon you may encounter in reports on experimental and quasi-experimental design research).

Control through measurement Control over context can be facilitated through measurement and quantification using careful design of instrumentation, tests and other approaches to obtaining measurements, i.e., using Measurement or Transformative data-shaping strategies. Such measurements effectively constrain the constructs/content that participants respond to as well as the formatting of those responses and ensure consistency across participants. Well-designed recording forms for observations and for the quantification of qualitative data (which often take the form of counts) can also help by shaping how gathered data are translated into quantitative form. Control through measurement works through how participants experience the data gathering process (e.g., completing instruments, questionnaires or online forms you design, interacting with sensors and other technology for recording specific types of measurements) and/or your experiences of the data gathering process (e.g., recording/coding/counting observations, transforming qualitative data into quantitative data, or creating quantitative data in specific data-generating environments). In terms of timing, control through measurement is imposed during (which shapes the data that you gather or generate) or after the data gathering process (which re-shapes data that you have already gathered) Control by design Control over context can be facilitated through effective design of contextual experiences. Control by design works by designing, manipulating or arranging the contextual experiences of different groups of participants. Groups may be differentiated according to whether they experience some change in their contextual circumstances (e.g., a new treatment or program, a change in environmental, social or institutional conditions, a drug dosage or some other experimental condition, that you determine, as the researcher). If you can control contextual conditions by design, this means that you are ensuring that the causal constructs of interest are used to define the groups or conditions. In addition, however, you may also explicitly track alternative plausible explanations (e.g., sex, age, level of experience) that could potentially influence dependent variables through explicitly incorporating them as independent variables in your research configuration (as definers of additional groups or conditions). Control by design has its impact while dependent variables are being measured and generally adds complexity to all components of the ‘Data Triangle’ (sampling, data gathering and analysis). For example, a pre-test-post-test intervention time-aligned longitudinal MU configuration helps to ensure temporal priority of cause and effect, in that the causal variable changes between the pre-test occasion and the post-test occasion (useful in research frames where your goal is to track changes over time). Use of a control group or placebo condition in the research configuration provides a reference condition against which to evaluate the effect of experimental treatments, conditions or groups. In some cases, control by design can be achieved by testing for the influence of one or more moderating or mediating variables. In situations where individual differences between participants create problems for causal inferences, you may decide to use an experimental design where each participant experiences all conditions associated with an independent variable, giving rise to what is termed a ‘repeated measures’ design. Instead of a control group containing a different sample of participants, each participant serves as their own control group, ensuring that their idiosyncrasies and individual differences remain constant across all conditions they provide data under. Figure 14.16b shows several experimental designs intended to facilitate control by design, through incorporating one or more extraneous variables (continued)

Table 14.2 Clarifying detail with respect to contextual design and control associated with implementing the Manipulative experience-focused strategy

14.3 Strategies for Structuring People’s Experiences 653

Procedural control Procedural control works by removing the opportunity for one or more alternative plausible explanations proposed as possible influences on the dependent variable before dependent variables are measured. Procedural control does not necessarily make your research configuration more complex. You can obtain control over certain alternative plausible explanations through the execution of specific procedures, such as random sampling of research participants, random assignment of participants to experimental or control conditions, counterbalancing of a set of experimental conditions experienced by the same participants, providing rest breaks where participation requires a sizeable commitment of time and effort and/or holding environmental and task conditions constant for all groups. Where you can determine (through random assignment) which participants experience which experimental or control conditions for all independent variables, you will have a true experiment. If you cannot randomly determine which participants experience which experimental or control conditions, for at least one independent variable, you will have a quasi-experiment. One important kind of procedural control is the ‘blind’ control. For example, in medical RCT research, a double-blind procedural control is typically required where neither the researcher/physician or the participants/patients know which condition they are participating in or which treatment they are receiving Control through statistical model specification This mode of control works by precisely specifying the statistical model that will be tested using an analysis procedure such as regression analysis, structural equation modelling; partial least squares or multi-level modelling (see Cooksey, 2014, for discussion and illustration of many of these procedures). Correctly specifying a model means specifying all the relevant variables to be included in the model and their inter-relationships (including causal relationships) as well as the error structures associated with that model (see, e.g., Hancock, 2004). Control through explicit statistical model specification generally requires very large sample sizes because of the estimation processes these more sophisticated analytical procedures employ. Control through statistical model specification has its impact before any data are gathered and, in fact, may influence the nature of the data to be gathered. It is generally not a sound practice to specify or modify a model once your data have been gathered and analyses have commenced as this is tantamount to conducting a ‘fishing’ expedition. If your model specifications are to be modified because of what you learned from analyses, the better practice is for you to obtain a new sample to test your modified model Control through statistical analysis Control through statistical analysis is closely related to control through statistical model specification in requiring a certain type of statistical model. However, control through statistical analysis works by statistically removing the influence of extraneous variables measuring alternative plausible explanatory causes (labelled covariates) from dependent variables after they have been measured but before the influence of the independent variables is assessed. Dependent variable measurements are statistically adjusted for the influence of any covariates and the adjusted measurements are then analysed with respect to possible independent variable influences

Table 14.2 (continued)

654 14 What Data Gathering Strategies Should I Use?

14.3

Strategies for Structuring People’s Experiences

655

It is important to realise that different approaches to control are often combined synergistically as when you employ a design involving several groups and participants are randomly assigned to each group (control by design coupled with procedural control). Control through statistical model specification is often combined with control through measurement in the Explanatory research frame using a statistical model testing approach like structural equation modelling or partial least squares modelling. In this situation, both modes of control have their impact simultaneously at the time of analysis. Control through measurement is sometimes combined with control by design and control through statistical model specification within the Survey or Explanatory research frame to create a conjoint experiment embedded within a questionnaire (as is often done in research on consumer choice or preferences). Sometimes the choice of one control approach creates circumstances where another alternative plausible explanation emerges as a potential liability and this means that another control approach must be put in place. For example, in a repeated measures design where participants experience all conditions (e.g., if you are a market researcher conducting taste preference tests), to negate the possible carry-over effects that repeated experiences (e.g., tasting) can create, you need to counterbalance or vary the ordering of experiences (e.g., products they taste) for different participants. Thus, control choices can sometimes create cascading or ripple effects. The practical side of thinking about experimental design involves making trade-offs between achieving a high level of internal validity in your research (i.e., capacity to unambiguously infer externalised cause and effect relationships) and achieving a high level of external validity (i.e. capacity to generalise the relationships and linkages found in your research context to some population of interest and to other places, times and contexts). Design decisions intended to maximise internal validity will almost invariably mean sacrificing a strong capacity to achieve high external validity and vice versa. Thus, these two types of validity, both of which are desirable features of ‘good’ positivist quantitative research, generally work against each other. There is no experimental or quasi-experimental design that can maximise both types of validity simultaneously. True experimental research designs afford the greatest possible control over extraneous factors while sacrificing contextual realism whereas quasi-experimental research designs sacrifice a high degree of control over context to obtain greater contextual realism and generalisability in the research. Here are some other things to consider as well: • Managing internal validity threats may require a more elaborate experimental design (e.g., including more IVs) and this may require a larger sample size. • If random assignment is not possible, individual differences may have to be controlled for in other ways (e.g., by measuring other IVs, by measuring and adjusting for covariates, or by using a repeated measured design). Using a repeated measures design, if not time-aligned, may then create the need to manage order effects, using a counterbalancing design like a Latin square. • If you employ any pre-test-post-test type of repeated measures design (which the intervention time-aligned MU configuration technically is, under the positivist

656

•

•

•

•

14

What Data Gathering Strategies Should I Use?

pattern of guiding assumptions) you must manage the risks associated with history, maturation and instrumentation effects, which creates the need for you to engage alternative control procedures (such as using parallel forms of measurement or using one or more control groups). Design complexity increases as the numbers of IVs (or manipulated levels/ measured values of IVs increases). In a factorial design, the number of effects to be tested (main effects and interactions) grows exponentially as the number of IVs increases linearly (e.g., 2 IVs in a factorial design creates 3 effects to be tested; 3 IVs creates 7 effects to be tested; 4 IVs creates 15 effects to be tested, 5 IVs creates 31 effects to be tested and so on). The more tests that you make in a single design, the greater your chances that you will wrongly conclude that one of those tests is significant. Sample size requirements increase as design complexity increases, unless some IVs can be used to define repeated measurements conditions using the same participants or matched participants (e.g., twins) or an incomplete or fractional factorial design (as in conjoint measurement experiments, see, e.g., Louviere, 1988) can be used. In laboratory or other types of experiments where you design tasks for participants to complete and you wish to make statements about how what you found might have relevance in the real world that participants live in, you may need to consider applying the Brunswikian principle of representative design (see, for example, Hammond & Wascoe, 1980). This means you would need to attend to the nature of the tasks/conditions that participants experience (i.e., that you create for them) so that they at least approximate the sorts of tasks/conditions they encounter in their lives outside of your research context. If the goal of your research is to maximise internal validity, then sound experimental design practices and choices provide the pathway. However, no amount of sophistication in statistical analysis can offset poor experimental design. Statistical analysis helps build the case for covariation and, in some cases, can help rule out alternative plausible causes, but it is sound experimental design that builds the strongest foundation required for concluding cause and effect relationships.

While the Manipulative strategy is most commonly associated with research guided by the positivist pattern of guiding assumptions, there may be very sound research-based reasons for implementing this strategy to manipulate people’s experiences in the context of interpretivist/constructivist guiding assumptions or perhaps critical social science, Indigenous or feminist guiding assumptions. However, we are not talking about importing the driving logic of positivism, where you attempt to exert sufficient control over context to warrant externalised causal inferences. Instead, your goal would be to use manipulated experiences simply as a way of focusing participants’ attention on what you (or some other relevant constituency, perhaps) wants or hopes to learn about. In that light, you seek to understand those structured experiences from participants’ perspectives. Thus, experience-manipulation, if done at all, would be loose, contextually-situated and

14.3

Strategies for Structuring People’s Experiences

657

relevant to participants in some way (which could be a positive or a negative or a complex way). What flows from this would be constructions, meanings and views that you co-create in conjunction with participants. The trick to maintaining authenticity in such circumstances is being constantly and analytically reflective on your contributions to that co-creation in addition to what participants bring to the table with their experiences and perspectives. Extensive field notes would be indispensable in such an effort. For example, suppose you are working within the Evaluation research frame and offer undergraduate students a choice to experience an educational program they have enrolled in one of two modes: in vivo (classroom-based) or on-line, making this essentially a quasi-experiment. In this scenario, your interest would be in understanding participants’ perspectives on, concerns about and reactions to their experiences of the specific delivery mode they chose. Under an interpretivist/constructivist pattern of guiding assumptions, it is possible to carry out a specific type of interpretive investigation, called an ethnomethodological experiment, which attempts to understand social order (pioneered by Harold Garfinkel in the 1960s). Here, the researcher deliberately intervenes in or manipulates a social interaction in order to disrupt social expectations and expose taken-for-granted assumptions for maintaining social order as the ensuing interaction subsequently plays out (see, for example, Heritage, 1984; Place, 1992). The researcher’s intention here is not to draw external causal inferences, but to bring to the surface social meanings that are typically not discussed or negotiated. In other words, the researcher deliberately violates social norms to see what gives rise to and maintains those norms. Thus, in an ethnomethodological experiment, the entire causal logic, relevant under the positivist pattern of guiding assumptions disappears, to be replaced by an interpretivist logic. Researcher intervention in context is deliberate and designed to elicit participant reactions that reveal that social expectations and social order have been violated. In such situations, this can yield very awkward and uncomfortable interactions and the researcher must be prepared for some blowback and, perhaps quite hostile, reactivity on the part of participants who get surprised by this research tactic. There are several systems you can use to effectively support the Manipulative strategy, including: • using your research journal for planning and recording choices made about design considerations, experimental manipulations/treatments to impose and control procedures to be employed; • where appropriate and you have access to the necessary resources, use of a laboratory where you can control environmental and observational conditions can help to maximise internal coherence (e.g., internal validity) of your research, but almost always at the expense of extensional reasoning (e.g., external validity); • using web-based software packages such as MouseLab (http://www. mouselabweb.org/), PsyToolkit (https://www.psytoolkit.org/), QuestionPro conjoint analysis software (https://www.questionpro.com/conjoint-analysis-

658

14

What Data Gathering Strategies Should I Use?

software.html) for designing conjoint experiments or Testable (https://www. testable.org/) to design, present, gather and store data from online experiments; and/or • using computer software programs such as SPSS, Stata, NCSS, Statistica and the like that generally offer procedures for computing optimised experimental designs, including incomplete block, Latin square and fractional factorial designs. Cooksey (2014, Fundamental Concepts VI and VII) explores some basic issues associated with the experiments/quasi-experiments strategy and Stuart and Rubin (2008) discuss best practices. There are a number of excellent in-depth discussions of the concepts and design principles associated with experiments and quasi-experiments (see, for example, Bryman & Bell, 2015, pp. 53–61; Cochran & Cox, 1957 (a classic text in the area); Cohen et al., 2011, Chaps. 10, 15 and 16; Cook & Campbell, 1979; Cook, Campbell and Peracchio, 1990; Keppel & Wickens, 2004; Kirk, 2013; Rosenthal & Rosnow, 1991, Chaps. 4–6; and Shadish, Cook & Campbell, 2001). Glass, Willson and Gottman (2008) review design considerations for time series experiments. Kaptchuk (2001) provides some useful insights into randomised controlled trials. Heritage (1984) and Place (1992) discuss and illustrate ethnomethodological experiments. Meyer (1995) reviews the use of natural and quasi-experiments in economics. Hammond and Stewart (2001, especially Chaps. 7–11, 19, 36) and Hammond and Wascoe (1980) discuss and illustrate the representative design of experiments. Non-manipulative Experience-Focused Strategy The Non-manipulative experience-focused strategy is implemented when you either (1) employ a correlational, instead of causal-comparative, approach where all constructs/variables relevant to your causal hypotheses are explicitly measured or (2) follow (or, in some cases, anticipate) the occurrence of an event (e.g., new social or environmental policy, regulatory scheme, social change program, conflict or war, natural or man-made disaster, the occurrence of which you would have no control over) in people’s life, cultural or work context in order to examine the impacts, side effects and longer-term consequences/implications of that event. For mode (1), your research would be considered non-experimental, involving no contextual manipulation of conditions/events for participants and generally using sophisticated statistical modelling techniques to test and fit hypothesised theoretical causal models to a set of data. It would most likely be guided by the positivist pattern of guiding assumptions. In a non-experiment, control by measurement (using data-shaping strategies) structures participants’ experiences only via the data gathering process itself; statistical model specification and statistical analysis are the tools used to enhance causal inference capacity. For mode (2), the context (or at least a portion of it) for the event becomes the research context and the people experiencing the event then become the intended research participants; your role as researcher is to connect with that context before, during and/or after the contextual event occurs. This mode of Non-manipulative strategy, focusing on naturally-occurring events that are

14.3

Strategies for Structuring People’s Experiences

659

Non-experimental Designs using latent variables and/or nested data structures [Conditions not manipulated; all variables are measured and are causally linked to specific latent variables/hypothetical constructs in specific ways via a structural model; model may or may not incorporate categorical variables for grouping/ differentiating participants; v = observed variable; u = measurement uniqueness; e = model error; IV = independent latent construct; DV = dependent latent construct] (a) Structural equation model (SEM) [covariance-based] u7 u8 u9 1

u1 u2 u3

1 1 1

v1 v2 v3

1 1 1

v4 v5 v6

(b) Partial least squares (PLS) model [variance-based] u10 u11 u12 1

1

DV1

v1 v2 v3

1

v 10 v 11 v 12

v4 v5 v6

DV2 e2

IV2

v7 v8 v9

v 10 v 11 v 12

DV1

DV2

IV2

1

(c) Multi-level (Hierarchical) model Level 3 Classroom/ teacher

1

Level 2 Student

1 2 … n1

Level 1 Occasion

IV1

1

1

e1 u4 u5 u6

1

v7 v8 v9

IV1

1

1

1

2

2

3

1 2 … n2

1

(d) Time series design

2

1 2 … n3

1

Y

T1 Y1

T2 Y2

X1 X2

X11 X21

X12 X22

Measurement Occasion T3 T4 Y3 Y4 ... X13 X23

X14 X24

... ...

Tn Yn X1n X2n

2

Fig. 14.18 Illustrations of different types of non-experimental designs: a structural equation design/model; b partial least squares design/model; c multi-level design/model; d time series design

occurring or have occurred, may be implemented under the guidance of virtually any of the patterns of guiding assumptions. Mode 1 Non-experiment (Correlational Approach) A mode 1 non-experiment (often referred to as the ‘correlational approach’ or a ‘correlational design’; see, for example, the discussion in Johnson, 2001) is achieved when you have no capacity or intention to manipulate conditions or experiences for participants. All independent and dependent variables are measured rather than manipulated, often in the context of a Survey or Explanatory research frame. This means that control through measurement, combined with control through statistical model specification and statistical analysis, are often the most viable tactics. Causal inferences are often sought but are generally weakest in a non-experiment. However, convincingness can sometimes be recouped using more sophisticated multivariate modelling techniques (e.g., in marketing, psychological, educational and organisational research where structural equation modelling, partial least squares modelling or multi-level modelling approaches are used; see Fig. 14.18 for illustrations). For a non-experiment (i.e., a correlational design), the causal factors may, in fact, be latent constructs, measured by a set of observed variables (e.g., ratings on questionnaire items). Both structural equation modelling (Fig. 14.18a) and partial least squares modelling (Fig. 14.18b) will permit testing of theoretically specified causal models. A multi-level model is used when you have

660

14

What Data Gathering Strategies Should I Use?

observations at different levels of analysis as shown in the example in Fig. 14.18c. Often a single MU configuration is used, especially within the Survey research frame. However, stronger causal inferences may be possible in a longitudinal MU configuration where IV data are gathered on the first occasion and DV data on the second occasion (to preserve the temporal ordering of causes and effects). The configuration of measurement occasions does not constitute a manipulation per se and leaves open questions about what might be happening between the two measurement occasions that could influence the DVs. It is also the case that many time series econometric designs (see Fig. 14.18d), where no intervention or treatment is imposed, fall into this category, including single (the Y row in Fig. 14.18d) and multiple time series (the Y, X1 and X2 rows in Fig. 14.18d, which may be related to each other through time), if you are analysing trends over time to create explanatory and predictive models (e.g., following changes in GDP over time, stock prices over time, unemployment rates over time). Here a longitudinal time-aligned MU configuration will likely be used to track one or more measures. For such models, all variables are measured instead of manipulated (often being sourced from secondary databases, thus pairing with the Archival/Secondary artefact-based strategy). If there is an intervention or treatment that has occurred at some point through the time series (e.g., public policy regarding interest rates or management of unemployment changes, not attributable to anything you have done as researcher), then the research would fall into the mode 2 non-experiment category for this data gathering strategy. In the correlational approach, there are several things you need to understand when you are configuring your research. • Generally speaking, correlational designs that use sophisticated multivariate statistical techniques require very large samples. This is so the statistical estimation procedures have a chance of providing good estimates of all parameters in the models being tested. A very rough rule of thumb is 10 cases/participants for every variable you measure; thus, if your model involves 40 variables/ questionnaire items, you would need a minimum of 400 participants in your sample, which could be resource-intensive to obtain. Sample size is somewhat less of an issue for partial least squares modelling than it is for structural equation modelling or multi-level modelling (see discussions in Cooksey, 2014, Procedures 8.7 and 9.5). • It is important to specify your model correctly in order to avoid problems with identifiability. What this means is that your model must be specified in such a way as to permit sensible estimates of all parameters and pathways to be made. What influences identifiability is the ratio of measurements in the model to sample size (sample size must be much larger) as well as the ratio of the number of parameters and pathways in the model to be estimated to the number of measurements available (measurements available must outnumber the parameters and pathways in the model), see discussions in Byrne (2010) and Cooksey (2014, Procedure 8.7).

14.3

Strategies for Structuring People’s Experiences

661

• For structural equation modelling, it is important to have at least three indicators or variables for every construct involved in the causal model (this practice also helps to ensure that your causal model is estimable). For complex models, this can mean that a great many questions need to be asked or ratings obtained, and this can lead to very long questionnaires and a complex pilot testing phase. For example, O’Cass (1998), in his PhD, used a hard copy questionnaire containing 19 pages to obtain the measurements for 179 items needed to test his theoretical marketing model focusing on consumer involvement. Consequently, you need to think about how to ameliorate the potential boredom and fatigue that accompanies filling in a lengthy questionnaire. Not attending to this potential problem can adversely affect analytic integrity and could influence your extensional reasoning capacity as well, particularly if this problem dramatically reduces your response rate (a highly likely outcome with long questionnaires). This is not quite as important an issue for partial least squares modelling, but still bears considering. • Multi-level modelling requires the use of a complex stratified sampling plan, where the strata are identified at several different levels of analysis (e.g., schools, classrooms and students). In order for a multi-level model to be estimable, there needs to be adequate sample sizes within each stratum at each level. This means that multi-level modelling typically requires very large sample sizes. Not attending to this potential problem can adversely affect analytic integrity. • Stable results when testing a complicated multivariate structural or multi-level model may not be achieved (usually because of anomalies in your data, such as non-normality), in which case you will need a back-up analytical strategy, and this may cost you in terms of power to make the kind of causal inferences you were hoping to make. Sometimes, data anomalies can be fixed using a quantitative to quantitative Transformative data-shaping strategy. • For time-series designs based on secondary data sources, missing or incomplete data for some time periods (e.g., data for one year might be missing) can be a problem, which means you will have to decide how best to handle the issue. There are approaches for estimating missing data, none of which has universal acceptance, so you would have to argue persuasively for the approach you plan to take. Another issue in time-series designs is ensuring that you have enough time periods included in order for a stable model to be estimated. In many cases, this number could approach 50 or 60 occasions (more feasible if your time unit is days or months rather than years, especially if you are relying on secondary data sources). Mode 2 Non-Experiment The mode 2 non-experiment strategy focuses on natural contextual events are beyond your control or will, as researcher, to influence. Your research, in such situations, is more likely to be opportunistic or solicited by parties interested in the impacts of the natural contextual event. Natural contextual events may be created or caused by human activities (e.g., social or economic policy interventions, social programs, social events, elections, a shooting/war/conflict, advertising programs).

662

14

What Data Gathering Strategies Should I Use?

Natural contextual events may also be created or caused by natural forces (e.g., earthquake, flood, severe weather events, seasonal changes, climate change, etc.) that are often unpredictable. Consider some examples: • New government legislation is passed requiring farmers to conserve native vegetation on their properties wherever possible, imposing stiff penalties if such vegetation is cleared without prior governmental permission. The researcher connects with farmers in order to gain insights into how this new regulation affects them and the contexts they live and work in (a research context similar to that explored by Sandall, 2006). • In an Indigenous community, a development application has been made to a local council which threatens land that has important spiritual meaning. Research may be conducted in order to more fully understand the Indigenous perspectives on this issue. • A destructive fire tears through a community destroying much property, injuring several people and killing hundreds of livestock. There are complaints running through the community that the emergency response process was not well-handled and that is why losses were so high. A researcher may wish to look into the community perspectives on the fire and emergency management process more fully and to compare/contrast those perspectives with the perspectives of relevant emergency services personnel. The mode 2 non-experiment strategy could be useful within a Survey, Case Study, Evaluation, Cross-Cultural, Indigenous, Transdisciplinary, Feminist, Descriptive, or Explanatory research frame, where your research focuses on the experiences, implications and impacts of that change or event. For such research, you could employ a case-based MU configuration (e.g., if the change or event occurs within a definable case context such as a community or organisation or the event itself defines the ‘case’). You might use a simultaneous MU, hierarchical MU or sequential MU configuration for situations where the change or event is not cased-based, but has a wider impact. You could employ an adaptive MU configuration if your research focuses on an ongoing event and you need to be flexible in your approach as the change or event unfolds. If your research focuses on a change or event that has already occurred, it would be considered to be a type of ex post facto non-experiment that seeks retrospective feedback and perspectives. The mode 2 non-experiment strategy will always be paired with other data gathering strategies, thus demanding a pluralist approach. Irrespective of the type of change or event being focused on, one of your responsibilities as researcher would be to ensure that you appropriately contextualise and describe the event, which will likely require you to implement a pluralist approach. In that light, we suggest that the mode 2 non-experiment be synergistically combined with one or more data-shaping strategies and/or one or more observation-based, artefact-based or interaction-based strategies. For example, contextualisation of natural contextual events created/caused by human activities (for example, the introduction of a government policy that raise the threshold for

14.3

Strategies for Structuring People’s Experiences

663

claiming childcare support for families with dual incomes) may require synergistic use of the Textual and/or Multi-media artefact-based strategies (e.g., tapping into news and social media data sources) coupled perhaps with an interaction-based strategy to connect with people (e.g., government ministers and advisors, stakeholders such as childcare workers, parents, employers) involved in the creation of or impacted by that event. More general considerations associated with mode 2 non-experiments include: • Ex post facto retrospective research needs careful attention to managing problems associated with memory and recall of the event or change under investigation. This problem becomes more prominent the farther back in time you ask participants to recollect. You may be able to use visual evidence such as photographs or physical objects to prompt a participant’s recall, but this need to be done sensitively in order to avoid biasing the participant’s memory recall. • Many changes and events that people must live through may be very stressful and emotional and all stages of your research need to be sensitive to this possibility when you examine one of those occurrences, from obtaining informed consent for their participation to publication of results. Under an interpretivist/constructivist or other non-positivist set of guiding assumptions, this is less of a problem because data gathering strategies will be geared toward capturing and dealing with such issues. Under the positivist pattern of guiding assumptions, this is more of a problem because emotionality and stress can adversely influence the quality of your measurements. What you ask and how needs to be very carefully considered. • In mode 2 non-experiments under the positivist pattern of guiding assumptions, high internal validity becomes very difficult to achieve because of your lack of control over context; instead external validity and statistical conclusion validity become more important to focus on. No matter what pattern of guiding assumptions you are following, it is important to realise that you may not get access to anyone who had experienced the change or event you are studying but since moved on or passed on because of it. This means that your sampling will invariably be somewhat biased. • We mentioned that change or event contextualisation is essential in a mode 2 non-experiment. However, in the case of human caused events, you may have difficulty contacting/gathering data from people who had a role in designing/ implementing the change or event in question. This can adversely impact your sufficiency arguments under an interpretivist/constructivist or other non-positivist set of guiding assumptions. There are several systems you can use to effectively support the Non-manipulative strategy, including: • using your research journal for recording contextualisation notes as well as notes regarding all data gathering activities;

664

14

What Data Gathering Strategies Should I Use?

• using sophisticated statistical software (such as SPSS, AMOS, Smart PLS, MPlus, eViews, Stata) to conducting the complex model building, fitting and testing processes required in a mode 1 non-experiment; and/or • using technological support such as digital recorders to record interviews or smart phone camera to record observations and video evidence. Johnson (2001) reviews ways of conceptualising non-experiments. For mode 1 non-experiments, Byrne (2010) discusses structural equation modelling, Bickel (2007) discusses multi-level modelling and Hair, Hult, Ringle, and Sarstedt, (2014) discuss partial least squares modelling. Bowerman, O’Connell, and Koehler (2005) and Enders (2014) reviews considerations associated with econometric time series. For mode 2 non-experiments, check out references associated with the data gathering strategies you pair them with. Structuring Experience-Focused Strategy The Structuring experience-focused strategy encompasses frameworks that employ structured thought- or experience-organising stimuli, tools and/or templates (e.g., decision support procedures and tools, scenarios, repertory grids, card or photo sorting, projective techniques using various kinds of stimuli) to guide or focus the experiences of research participants. These frameworks can be employed in a manner consistent with positivist or non-positivist assumptions, depending upon your research goals, the type of data you plan to collect, and how you plan to deal with those data. Your goal in using a structuring framework is to focus participants’ attention on a specific problem/issue or set of stimuli in a specific way in order to more fully investigate how they think about that problem/issue (which may range from very simple to highly strategic in nature) or understand participants’ reactions to/interpretations of those stimuli. Structuring Templates A structuring template is a task-configuring tool that you, as researcher, provide to participants or ask them to work within. Some examples of structuring templates for data gathering include: decision support procedures and tools such as decision trees and multi-attribute matrices (Brown, 2005; Pidd, 2009; yields quantitative data consistent with positivist assumptions); judgment analysis tasks (Cooksey, 1996; yields quantitative data consistent with positivist assumptions); scenarios (Maani & Cavana, 2007, Chap. 5; yields quantitative data consistent with positivist assumptions, but can yield qualitative data consistent with interpretivist/ constructivist assumptions); the Complex Dynamic Decision Perspective (CDDP, see Cooksey, 2000; yields qualitative data consistent with interpretivist/ constructivist assumptions); repertory grids (Cassell & Walsh, 2004; yields quantitative and/or qualitative data, but generally consistent with positivist assumptions), creative thinking and problem solving (e.g., Proctor, 2010) and the Five Whys (Senge, Kleiner, Roberts, Ross, & Smith, 1994; yields qualitative data more consistent with interpretivist/constructivist assumptions). Structuring templates may focus on real or hypothetical issues, events, situations or stimuli and may be used

14.3

Strategies for Structuring People’s Experiences

665

for individuals or in group settings. With planning-oriented templates like scenarios or decision support tools, data gathering may focus on hypothetical events or choices that have yet to occur. Structuring templates are a very useful strategy for investigating problem solving and decision making and can also be useful in helping to understand participant perceptions and thinking patterns. As such, a structuring template is a distinctly cognition-oriented applied research strategy and may be useful in an Explanatory, Exploratory, Evaluation, Case Study, Transdisciplinary or Action research frame as a way of gaining insights into participants’ thought and decision processes. A structuring template could be used by itself in a single MU configuration. In addition, a template may be used in a simultaneous MU configuration (e.g., gathering structured observations or a think-aloud protocol (to be discussed below) while participants actively use the template to solve a problem or make a decision) or in a case-based MU configuration as a way of studying problem solving or decision making within a specific case context. One important consideration attached to the use of a structuring template is that participants will generally need to be trained or at least have some experience with using or working with the template. This training requirement will add to the timeline of your research and there may be variations in how well individual participants cope with the template that training would not necessarily level out. Sorting Tasks Another type of structuring framework is the object sorting task (where the objects may be pictures, drawings, words or phrases, concepts or photographs). To implement this strategy, you would design or sample a set of stimuli to provide the objects for sorting and ask participants to sort them into piles, either according to some criterion that you provide (e.g. along a scale indicating degree of preference or liking) or by simply grouping them together in some way that makes sense to the participant. One variant of this sorting strategy is to have participants verbalise their logic as they sort (see process tracing methods below). Under the positivist pattern of guiding assumptions, the resulting relationships, embodied in the objects that are grouped together and distinguished from other piles, can be analysed statistically using multidimensional scaling or clustering methods to produce a map/graph of the dimensions along which the piles have been created (see, for example, Canter, Brown, & Groat, 1985; Kane & Trochim, 2007). If you do have participants verbalise as they sort things into piles, you could record their verbalisations and analyse the resulting qualitative data to uncover their perceptions, interpretations and logic behind the piles they create—an approach consistent with interpretivist/ constructivist assumptions. The concept mapping approach put forward by Kane and Trochim (2007) is a hybridised strategy that involves an initial qualitative phase where participants generate, via brainstorming, the concepts that they then sort into groups for statistical analysis and subsequent use in decision or policy making. Sorting tasks may be useful in an Exploratory, Cross-Cultural, Indigenous or Evaluation research frame as a way of gaining insights into how participants think about and organise their world and objects within it. In the Cross-Cultural or

666

14

What Data Gathering Strategies Should I Use?

Indigenous research frame, using a sorting task may provide a way to gain insights into culturally-based ways of perceiving the world. A sorting task would seldom be used by itself in a single MU configuration. Instead, such a task may be used in a simultaneous MU configuration (e.g., for gathering think-aloud protocols while participants actively sort the objects) or in a sequential or hierarchical MU configuration as a preliminary or supporting data gathering strategy. Projective Techniques Projective techniques constitute a different type of structuring framework. Such techniques employ physical stimuli—such as words, statements, pictures, photographs or drawings—for the express purpose of eliciting reactions in the form of interpretations, stories, comments, emotional reactions or evaluative ratings from participants (see, for example, Bond & Ramsey, 2007; Chrzanowska, 2002, Chap. 8; Malhotra, Hall, Shaw, & Oppenheim, 2008, pp. 138–142). As a set of data gathering strategies, they tend to be used more frequently in marketing, as well as in some cross-cultural research. A key feature of projective techniques is that they implement an indirect mode of structured data gathering where you, as the researcher, draw inferences about participant motivations, attitudes, beliefs and/or emotions based on their responses to intentionally ambiguous stimuli. The theory is that participants will project evidence of these internal states into their responses without you directly asking them about those states. Thus, projective techniques are dependent upon a minor deception of participants in order to work effectively. Confusingly, projective techniques are often implemented in a manner more consistent with positivist assumptions yet yield data more consistent with interpretivist/ constructivist assumptions. We say confusingly because, while most of the techniques will yield qualitative data, the interpretation of participants’ responses is not truly open. The researcher looks for specific things to appear in the data, rather than digging to understand the participant’s perspective in full. Another variation of projective technique is photo-elicitation, where photos are used to stimulate storytelling and narratives (pairing with the Stories participant-centred strategy; see Banks, 2007). Malhotra et al. (2008) indicated that there are four broad classes of projective techniques: association (e.g. word association tasks); completion (e.g. sentence or story completion tasks); construction (e.g. picture response tests or cartoon tests where the participants build a story around the stimulus being presented); and expressive (e.g. role playing where the participants try to put themselves in the ‘shoes’ of someone else). A major advantage of projective techniques is that, because of their indirect nature, they may provide insights into a participant’s state of mind which they otherwise might be unwilling to provide. Thus, these techniques are robust against social desirability and other impression management strategies that participants might use to cover up uncomfortable thoughts or feelings. On the flip side, projective techniques are notoriously difficult to validate, and suffer from relatively low levels of reliability. They may require the services of skilled interviewers and interpreters (e.g. a trained psychologist). In general, you would be advised not to depend too heavily on projective techniques as a single source of data and, in most cases, to only employ them when other

14.3

Strategies for Structuring People’s Experiences

667

avenues are not viable. Projective techniques may be useful in an Exploratory, Cross-Cultural, Transdisciplinary or Action research frame as a way of gaining insights into participants’ thought processes. A projective technique would not be used by itself in a single MU configuration. Instead, projective techniques are almost always paired with the Stories participant-centred strategy or the focused interview interaction-based strategy in a simultaneous MU configuration. Process Tracing Yet another type of structuring framework is process tracing. Process tracing takes several forms including techniques such as monitoring information acquisition and use and the think-aloud (verbal) protocol (see, for example, Ericsson, 2003; Ericsson & Simon, 1993; Payne, Bettman, & Johnson, 1993). These methods are most useful for investigating problem-solving and decision-making, most often at the individual level. They can be conducted consistent with either the positivist (where the data are transformed into quantitative form) or interpretivist/ constructivist patterns of assumptions (where the data are treated as qualitative interactions). However, the positivist slant tends to dominate, especially with respect to research on decision making. The process tracing strategy may be useful in an Exploratory, Explanatory, Evaluation or Action research frame, with the Explanatory research frame probably being the most commonly employed. Process-tracing strategies are frequently used in combination with other data gathering strategies in a simultaneous, hierarchical or longitudinal MU configuration. For example, certain types of decision making research are conducted by gathering verbal think-aloud protocols in conjunction with the Manipulative strategy (where specific task conditions are manipulated). For other kinds of problem-solving research, the process tracing strategy may be used as a stand-alone approach in a single MU configuration (as when expert chess players are asked to think-aloud as they plan their next move, see Ericsson & Simon, 1993, for examples). The think-aloud protocol approach can be useful in the development and pilot-testing of a complex questionnaire in a sequential MU configuration, where participants are asked to think-aloud as they work through the questionnaire (e.g., Trenor, Miller, & Gipson, 2011). The think-aloud verbal protocol process tracing approach was originally developed to study human problem-solving and decision-making, including trying to understand how people play chess or solve mathematical puzzles (see Ericsson & Simon 1993). However, the technique now enjoys much wider use as an approach for helping people display how they are thinking as they focus on a task. To implement the technique, you, as the researcher, invite participants to consider a specific problem (which may be real or hypothetical) and asking them to think out loud as they solve the problem, verbalising whatever they are thinking. Their verbal think-aloud talk is usually recorded and transcribed for later analysis, yielding a ‘think-aloud protocol’. The protocol can then be analysed either using a pre-designed coding scheme, consistent with a positivist approach), or more openly and thematically, consistent with an interpretivist/constructivist approach (thus pairing this approach with the Transformative data-shaping strategy and the Textual

668

14

What Data Gathering Strategies Should I Use?

artefact-based strategy). Think-aloud protocols can be produced with or without on-going input or prompting from you, the researcher, as appropriate to your research goals. The technique is limited by the extent to which participants have conscious access to the ways they have of thinking about a problem and by their potential susceptibility to social desirability or normative pressures (in which case, what participants say they are thinking may not agree with what they are actually thinking; see discussion in Nisbett & Ross, 1980, for example). To circumvent this limitation, you might pair think-aloud protocols with another data gathering strategy (such as the Visualisations participant-centred strategy, any of the interaction-based strategies or the observation-based strategies) to provide triangulating data. Think-aloud protocols have been used to analyse how doctors/nurses/ medical trainees/interns make diagnostic judgments and treatment prescriptions (e.g., Hammond, Frederick, Robillard, & Victor, 1989), how engineers evaluate highway design conditions (e.g., Hamm, 1988), how teachers make grading assessments of student essays (e.g., Cooksey, Freebody, & Wyatt-Smith, 2007), how weather forecasters make their forecasting judgments (e.g., Chi, 2006), and how consumers evaluate their preferences for specific products or services (e.g., Schkade & Payne, 1994). The think-aloud verbal protocol process tracing approach may also be used in combination with the Visualisations participant-centred strategy where participants are asked to verbalise as they create or construct a representation of something they have been asked to think about (e.g., Sandall, 2006). Monitoring information acquisition and use is a software/technology-mediated process tracing approach that is explicitly positivist in orientation. It involves the use of a software system (such as MouseLab, see Payne et al. 1993) to configure an experimental environment in such a way that all movements of a mouse or other control device, which a participant makes to uncover and examine information on a computer display relevant to solving a problem or making a (typically hypothetical) decision, can be tracked and stored for analysis (thus pairing with the Manipulative strategy). The software can measure many different aspects of decision behaviour including how much time has been spent looking at specific pieces of information, the order in which information is inspected, and how many times a bit of information was examined. You could couple the monitoring information acquisition and verbal protocol techniques together to produce a richer set of data for analysis. The major drawback of monitoring information acquisition is the artificial and constrained nature of the experience-structuring computer interface that participants must deal with. It remains an open question as to whether participants would actually process information for a real decision in the ways they demonstrate within the monitoring information acquisition technique. It is also possible to monitor information acquisition (as well as other visual learning tasks) using a technological innovation called eye-tracking. Here, participants use a wearable device that tracks their eye movements as they read a document or look at computer screen (pairing with the Measurements data-shaping strategy). This approach is useful for understanding how participants focus on and scan information, without necessarily requiring a think-aloud protocol to be produced. It may help to circumvent some of the problems associated with think-aloud protocols.

14.3

Strategies for Structuring People’s Experiences

669

There is another type of process tracing approach used in the social and behavioural sciences. Process tracing is used, especially in the political sciences, as a way of identifying, unpacking and clarifying the causal mechanisms underpinning social and political processes (see, e.g., Beach & Pedersen, 2013; Bennett & Checkel, 2014). It focuses on causal processes operating at the more macro-levels of society (e.g., institutions, governments, nations). What is interesting about this variant of process tracing is that it employs qualitative data gathering strategies such as ‘elite interviewing’ (see Tansy, 2007) to connect with key players in the political process being researched and almost invariably is used in a pluralist manner in conjunction with other data gathering strategies, such as the Non-manipulative experience-focused strategy and the Archival/Secondary artefact-based strategy, often within a Case Study or Explanatory research frame. In the end, the goal is to identify causal chains and linkages (including the key independent and dependent variables involved in them), which is consistent with the positivist or critical realist pattern of guiding assumptions (although Bennett & Checkel, 2014, in their Chap. 1 also discuss relationships with interpretivist/constructivist viewpoints). Beach and Pedersen (2013, Chap. 8) identified three variants of political process tracing, each labelled by its focal research emphasis: (1) theory-testing process tracing; (2) theory-building process tracing; and (3) explaining-outcome process tracing. They offer checklists for each type of process-tracing approach in their appendices. It is important to note that the think-aloud protocol version of process tracing is anchored in the present and focuses on individuals (protocols are generated while a participant is carrying out a task) whereas the political version of process tracing is necessarily retrospective in focus, examining events at a macro-systems level after they have occurred. This retrospective focus needs to be taken with a healthy respect for the potential influences of memory and self-serving biases in the stories that are told. One major drawback with using structuring frameworks of any type is that they may bound the problem or issue so tightly that issues outside the framework get overlooked, or in a way that is not congruent with participant values or their preferred mode of cognition, meaning that they will tend to struggle with using the framework and produce data that may be very suspect in quality. A pluralist investigation, combining the structuring strategy with other data gathering strategies, may be the most effective way to combat this drawback. Another potential drawback to be aware of is that structuring templates and projective techniques are usually created based on a specific theory and if that theory is incomplete or makes problematic assumptions, the usefulness of the framework could be greatly diminished. Both drawbacks could create problems with extensional reasoning for you if left unaddressed. There are several systems you can use to effectively support the Structuring strategy, including: • using your research journal for recording notes and observations throughout the data gathering process as participants work within the framework you have established for them;

670

14

What Data Gathering Strategies Should I Use?

• using an information acquisition and use process tracing software package such as MouseLab and MouselabWeb (http://www.mouselabweb.org/), or MouseTrace (Jasper & Shapiro 2002) to build an interactive computer interface for gathering and recording data; • using a digital recorder to record think-aloud protocols; and/or • using data recording media such as decision support computer interfaces (e.g., POLICY PC, http://www.exedes.com/PolicyPC.htm, for judgment analysis tasks, see Cooksey, 1996; or Expert Choice, https://expertchoice.com/, for more general decision tasks), printed templates, flip charts or whiteboards (especially useful when using structuring templates, particularly in a group setting; if whiteboards are used, use a smart phone camera to photograph the outcomes or use a whiteboard with printing capabilities). Boddy (2005), Catteral and Ibbotson (2000) and Walsh (1997) discuss projective techniques in the marketing and educational disciplines. Ericsson (2003), Ericsson and Simon (1993) and van Someren, Barnard and Sandberg (1994) discuss possibilities and considerations associated with think-aloud protocols. Payne, Bettman and Johnson (1993) provide extensive discussions of how to investigate decision making using information acquisition and tracing and link their discussions to the MouseLab software. Beach and Pedersen (2013), Bennett and Checkel (2014) and Checkel (2008) offer in-depth explorations of the political process tracing approach. Tansy (2007) relates this approach, not only to the possibilities of elite interviewing, but also to which sampling schemes might be useful to employ. Immersive Experience-Focused Strategy The Immersive experience-focused strategy provides goal-oriented participant experiences that are sometimes structured within a high-fidelity replica of some environment of interest, the parameters of which the researcher can control. For example, researchers into aviation ergonomics and flight crew behaviour may use extremely high fidelity (and quite expensive) flight simulators for jet aircraft or spacecraft (Lee 2005). Training and research on medical teams can now be conducted in the context of high-fidelity operating room or hospital ward simulations, complete with simulated patients (Small et al. 1999). There are also system modelling ‘flight’ simulators, which have less fidelity than complete environment simulators, but which comprise formal integrated dynamic systems for tracking and providing feedback on decisions and choices (see, for example, Pidd 2009, Chap. 9; Maani & Cavana 2007, especially Chaps. 3–6). With today’s computing technologies, it has become feasible to build highly realistic (‘immersive’) virtual worlds for the study of organisational and managerial behaviour (Pierce & Aguinis 1997). Web-based virtual worlds/games such as Second Life have also become potential environments in which research can be conducted (for example, see Mennecke et al., 2008; Rymaszewski et al., 2007). Games are typically of much lower fidelity, relative to simulated environments but are distinguished by the requirement that participants follow a set of rules that all players share. Simulations are much more open-ended in terms of permitted behaviours. The Immersive

14.3

Strategies for Structuring People’s Experiences

671

Table 14.3 Rough classification of types of simulations and games along two dimensions: fidelity and depth of immersion Fidelity Low Depth of Immersion

Shallow

Partial

Nearly Complete

Medium

Games (e.g., commons dilemmas (see Hine et al., 2009); the Beer Distribution game (http://web.mit. edu/jsterman/ www/SDG/ beergame.html), Island Escape Game, see Lane, 1995) Non-functional Mock-ups (e.g., working with lower quality mock-ups like a stage set or equipment, but with little or no functionality like the real-world environment)

Management ‘Flight’ Simulators/ Microworlds (e.g., People Express Airline, see Lane, 1995; Fire Chief dynamic decision making game, see Omodei & Wearing, 1995)

Role Playing exercises and games (see Cohen et al., 2011, Chap. 26; e.g., prisoner’s dilemma, Stanford Prison Experiment)

Computer Screen Virtual Worlds (see Cohen et al., 2011, Chap. 11; e.g., computer and online games, e.g., Second Life; ATC Lab Advanced air traffic control simulation, see Fothergill, Loft, & Neal, 2009)

Functional Mockups (e.g., working with high quality mock-ups that have at least some functionality like the real-world environment)

High

High-end Virtual Worlds (VR goggles/helmets); Fully-fledged environment simulations with simulated people (e.g., operating room or hospital ward with simulated patients, e.g., see Ker, 2006) Fully-fledged environmental simulators involving real people/actors (e.g., aircraft cockpit and spacecraft simulators (see Lee, 2005) or motor vehicle simulators offering full motion and 360-degree experience, car accident or disaster simulations that look, feel and function like their real-world counterparts( e.g., see Wybo, 2008)

672

14

What Data Gathering Strategies Should I Use?

strategy tends to be implemented in a manner consistent with positivist or critical realist assumptions using a wide range of quantitatively recorded data, although this need not be the case if players are interviewed in depth about their experiences or if team behaviours are observed or video recorded for later analysis, consistent with an interpretivist/constructivist pattern of guiding assumptions. The Immersive strategy may be useful in an Explanatory, Exploratory, Development Evaluation or Action research frame. It may also be possible to use a Survey research frame, but in an ex post facto manner where post-simulation or post-game participation reflections and retrospective perspectives are sought. The Immersive strategy is seldom used on its own, instead it is usually combined with other data gathering strategies. Thus, the Immersive strategy works best in a pluralist approach. For example, under the positivist pattern of guiding assumptions, simulation or game participation is often paired with the Manipulative strategy when you, as researcher, systematically vary certain parameters of the game or simulation, then observe or measure behaviours that occur and outcomes that are achieved (further pairing with the Measurement data-shaping strategy or the Systematic observation-based strategy). The Immersive strategy is often implemented within a simultaneous MU configuration, where data gathering via other strategies occurs contiguously with simulation participation. However, implementation within a sequential, hierarchical or case-based MU configuration could also be possible, depending upon your research goals. It is also possible to employ a longitudinal intervention time-aligned MU configuration where participation in the simulation constitutes the intervention and you track performance and experiences over time. Immersion can vary in depth from shallow, where participation is more surface level where lines between the research experience and real-world experience are minimally blurred, to almost complete, where participants may behave as if their research experience is a real-world experience and lines between the research experience and real-world experience are maximally blurred. Immersion can also vary from very low fidelity (i.e., low verisimilitude, where it is obvious that participants’ research experiences are not much like their real-world experiences, except with respect to certain features) to extremely high fidelity (i.e., high verisimilitude, where participants’ research experiences are almost indistinguishable from the real-world experience with respect to nearly all features). Table 14.3 shows how different types of simulations and games might be classified against the fidelity and depth of immersion dimensions. For some research goals, all you may require is the capacity to control some formal aspects and key variables of the tasks you are asking participants to perform, in which case, lower fidelity, shallower depth games may be best choice, where the larger environment within which the game is played is of little importance. For other research goals, embedding participants in an environment that matches, as closely as possible, one they would or could confront in the world outside of your research but one over which you still maintain control, would be the only ways to study the behaviours you are interested in. Simulations and games create artificial but engaging contexts within which to study behaviour at both the individual and the group level. The higher the fidelity of

14.3

Strategies for Structuring People’s Experiences

673

the simulation, the more immersive the experience for the participant and the more the research outcomes may be generalisable to real-life contexts. Simulations and games are often coupled with experimentation to create realistic yet highly controllable, manipulable research environments. The major downside, especially with respect to medium- to high-fidelity simulations, is the cost of the systems required and/or the effort required to construct and test them. They may also yield highly complex data that may demand complex analytical strategies. There are some important considerations to keep in mind if you want to implement the Immersive strategy: • Simulations and games are designed for providing structured experiences for participants that are safe from real-life consequences of making errors or incorrect decisions. This is why they are often used as learning and training environments for managers, teachers, doctors, nurses, pilots, air traffic controllers and military personnel. Their use as data gathering environments may add an observational or physical data record. Participants need to be advised of this possibility before they commence and give their informed consent for aspects of their performance in the simulation or game to be recorded and used for research purposes. • Participation in a simulation or game may create emotional stress for some participants, particularly if there is a power dynamic involved (as in prisoner dilemma games where participants play prisoner or guard roles). This risk increases where engagement/immersion in a role may create experiences that have emotional consequences. For this reason, participants should normally be debriefed after their participation so that any adverse consequences can be discussed and dealt with. • In role play games, people will differ with respect to their ability to “stand in someone else’s shoes” and this could have implications for the quality of the data you gather. • Some simulations and games may require a training phase before commencement of the experience proper, which will add to the timeline of your research. • If you are employing the Immersive strategy in conjunction with the Manipulative strategy, you must attend very closely to choosing which aspects of the simulation or gaming environment you want to manipulate and at what levels. You must be alive to what is realistic for people to experience. You do not want to create conditions that will inordinately stress people or place them in unrealistic scenarios. The higher the fidelity and deeper the immersion, the greater the risk of placing participants in situations they cannot adapt to or work within. Normally, you would not want to expose your participants to a no-win scenario in a high-fidelity simulation (e.g., an irrecoverable spin in an aircraft simulator). • Participants always have the right to withdraw their participation in a simulation/game at any time. You need to plan for what to do when this happens, especially in team-based simulations where losing a participant could adversely affect the entire exercise (e.g., in an operating room simulation).

674

14

What Data Gathering Strategies Should I Use?

There are several systems you can use to effectively support the Immersive strategy, including: • using your research journal for recording notes and observations throughout the process for developing and implementing a simulation or game and using it with participants for research purposes; • using appropriate materials/equipment/environment designed for the simulation or game you are using, usually obtainable by contacting the author of the game, owner of the resource or visiting an appropriate website; and/or • using appropriate computer software to implement as well as record data from a simulation (generally more viable for individual simulation participation). Cohen et al. (2011, Chaps. 11 and 26) discusses the use of virtual worlds and role playing in education. Lane (1995) provides a general background discussion on management simulations and games and includes an appendix describing several such simulations and games including: WFF’N Proof, The Island Escape Game, Beer Distribution game, Fish Banks, Ltd. and People Express Airline. Maani and Cavana (2007, Chap. 6) and Senge et al. (1994, Chaps. 87–90) discuss and illustrate microworlds and management flight simulators, tied to systems dynamics models. Shiratori, Arai and Kato (2005) contains a set of contributions focusing on simulation and gaming in the social sciences. The Sage journal, Simulation & Gaming (http://journals.sagepub.com/home/sag) is devoted to the use of simulations and games for research as well as educational and development purposes.

14.4

Key Recommendations

Throughout this chapter, we have emphasised the need to think carefully about each data gathering strategy, in terms of your research goals, their usefulness under specific patterns of guiding assumptions, research frames and MU configurations and their advantages and disadvantages. • If we could identify the single most important take-home message from this chapter, it would be this: Make clearly reasoned and well-defended choices of data gathering strategies as you are planning your research project. Know why you have chosen the strategies you have, understand what they will and will not allow you to do and say and realise that there are no perfect or uniformly preferred choices. Make your decisions, explain your reasoning, defend your choices and then tell your story in full recognition of the choices you have made and the consequences they produce. • Another point we would like to emphasise is that given the very diverse range of data gathering strategies available to you, it is easy to get carried away and assume that once your choices have been made and implemented, everything will go smoothly from that point. This assumption is false. Even the clearest, most well-reasoned and competently implemented data gathering choices can

14.4

Key Recommendations

675

back-fire or, even more likely, one or more of your intended data sources may pull the plug on their participation, perhaps forcing you to rethink your entire approach. These things do happen, but it need not be fatal to your research or to your candidature. What we recommend to our students is that it is important to have a Plan B at least sketched out for your research. Try to anticipate where your research could come off the rails and think about an alternative set of data gathering strategies that could get you close to your desired goals. Amongst the things that could happen to push your research off the rails are: – organisations, institutions, groups and/or participants who initially agree to participate suddenly withdraw their participation; – organisations, institutions, groups and/or participants you intend to sample do not agree to participate in sufficient numbers; – one or more of your data gathering strategies generates unintended side effects amongst organisations, institutions, groups and/or participants; – one or more of your data gathering strategies does not work in the way that you thought it would or does not yield the information you thought it would; and/or – you encounter dynamic changes in your life, or in the resources you can call upon to facilitate your research, which render one or more of your planned data gathering strategies non-viable. • To help you anticipate potential problems down the road, try a simple scenario thought experiment. Imagine you are conducting your study using your preferred data gathering strategies and your research is unfolding forward in time. Now try to imagine critical points in that unfolding where, if certain events occur of the sort highlighted above, you ask yourself two questions: (1) what might this event do to my research? and (2) what could I do to circumvent or perhaps prevent this from happening? Your answers to these questions should provide you with some insights as to what your Plan B should encompass. You will certainly not be able to anticipate every possibility, but if you can anticipate some of the more potent, large or probable ones, this would help you in your construction of Plan B. Here, we are only talking about Plan B in terms of choices of data gathering strategies. In Chap. 19, we will discuss the importance of adding a sampling dimension to your Plan B; namely, a Plan B for your selection of data sources. • Whatever data gathering strategies you employ, keep in mind the quality criteria relevant to your adopted pattern of guiding assumptions as well as the meta-criteria as these can help you focus on how best to implement the procedural aspects of any strategy. • We have highlighted the value that a pluralistic approach can add to research and the synergies we have pointed to in this chapter signal that in many areas. In some cases, pluralism is mandated by virtue of the nature of your research or your choice of data gathering strategy. In other cases, you might consider a pluralist approach to take advantage of the potential synergies it can offer.

676

14

What Data Gathering Strategies Should I Use?

Appendix: Clarifying Experimental/Quasi-experimental Design Jargon These contrasting concepts provide insights into the way that researchers, who implement the Manipulative experience-focused strategy under the positivist pattern of guiding assumptions, talk or write about certain features of their research. Between groups versus Within groups IVs A between groups IV has categories that define groups which contain different samples of participants (e.g., a treatment group and a control group). A within groups IV defines groups or conditions, all of which are experienced by each participant or by matched sets of participants such as twins or participants matched on key characteristics. A within groups IV includes intervention time-aligned conditions such as a pre-test and a post-test, giving rise to a class of experiments called ‘repeated measures’ designs) Factorial versus Nested designs A factorial design involves groups defined by least two IVs where each category of one IV is combined with each category of another IV, such that the groups exhaust all possible combinations (e.g., a quasi-experiment involving the IVs of gender, with 2 categories—male and female, and an experimental IV, with 2 categories—treatment condition and control condition, yields a 2  2 factorial design involving four distinct pairings of IVs (male-treatment; male-control; female-treatment; female-control). If you had a between groups factorial design with four IVs and each IV had 2 categories (or ‘levels’), you would have a 2  2  2  2 factorial design and that design would have 16 distinct groups of participants. A nested design involves groups defined by the categories of one IV being hierarchically embedded inside each category of another IV (e.g., an IV defined by year levels for classes of students at the primary school level is embedded within a second IV defined by specific schools). Nesting means, for example, that a year 6 class in one school cannot be considered equivalent to a year 6 class in another school (different teachers, different curricular emphases, different classroom environments, …), so that classes must be considered as nested within schools. Another type of nested design is a multi-level design, which compares samples defined by IVs at different levels of analysis (e.g., departments within organisations within industries) both within and between those levels Main effect versus Interaction effect For causal-comparative designs, a comparison of the groups or conditions defined by the categories (or ‘levels’) of a single IV comprises the main effect of that IV on the DV. The comparison of groups simultaneously defined by combinations of the categories of two or more IVs is termed an interaction. An interaction yields a conditional interpretation, where the pattern of relationship between one IV and the DV differs depending upon which category of another IV you choose to look at. Technically speaking, a moderator IV is an interaction IV. Where two IVs define an interaction, this is called a 2-way interaction; three IVs define a 3-way interaction and so on. In a factorial design, there are as many main effects as there are IVs in the design, all possible pairs of IVs form 2-way interactions, all possible triplets of IVs form 3-way interactions and so on. For example, if you had a factorial design involving 4 IVs (call them A, B, C, and D): there would be 4 main effects (A, B, C and D main effects), six 2-way interactions (AB (read as ‘A by B interaction’), AC, AD, BC, BD, CD interactions;), four 3-way interactions (ABC, ABD, ACD, and BCD interactions) and one 4-way interaction (ABCD) to test Incomplete or Fractional factorial designs (continued)

Appendix: Clarifying Experimental/Quasi-experimental Design Jargon

677

(continued) In some design circumstances, it may not be possible or feasible for you to include all possible factorial combinations of IV categories in a research design. For example, if you have four IVs, each with 3 categories, there would be 3  3  3  3 = 81 possible factorial combinations, which may be too many for you to find adequate samples to fill or to have participants rate or evaluate. As an alternative approach, you could employ an incomplete or fractional factorial design, which includes only a specific fraction or proportion of the possible combinations. In the previous example, a 1/3 fractional factorial design would require only 27 combinations instead of 81. The fractional combinations used are identified by sacrificing information about higher order interactions (e.g., three- and four-way interactions) in order to provide viable estimates of lower-order effects, such as main effects and two-way interactions (fractional factorial designs are often used in conjoint measurement designs, for example). One example of an incomplete design is a ‘Latin square’ design, which can control, using counterbalancing, for order effects between conditions or other extraneous/‘nuisance’ variables Manipulated (usually categorical/group-based) versus Measured IVs A manipulated IV is one where you can control who experiences a specific category of the IV (e.g., treatment or control conditions) using random assignment of participants to category. In contrast, a measured IV is one where you must take the IV as having a pre-existing value with respect to every participant and therefore you can only measure it (e.g., age, gender, ethnic background). Thus, in a true experiment, you seek to manipulate all IVs being evaluated whereas in a quasi-experiment, you generally have a mix of manipulated IVs and measured IVs

References Anderson, L. (2006). Analytic autoethnography. Journal of Contemporary Ethnography, 5(4), 373–395. Anderson, V. (1997). Systems thinking basics: From concepts to causal loops. Cambridge, MA: Pegasus Communications. Angrosino, M. (2007). Doing ethnographic and observational research. London: Sage Publications. Athanasou, J. A. (1997). Introduction to educational testing. Wentworth Falls, NSW: Social Science Press. Axelrod, R. (2007). Simulation in the social sciences. In J.-P. Rennard (Ed.), Handbook of research on nature inspired computing for economy and management (pp. 90–100). Hershey, PA: Idea Group Reference. Babbie, E. (2011). The basics of social research (5th ed.). Belmont, CA: Wadsworth Cengage Learning. Banks, M. (2007). Using visual data in qualitative research. London: Sage Publications. Barbour, R. (2007). Doing focus groups. London: Sage Publications. Beach, D., & Pedersen, R. B. (2013). Process tracing methods: Foundations and guidelines. Ann Arbor, MI: University of Michigan Press. Bechtel, R. B. (1967). Hodometer research in architecture. Milieu, 2, 1–9. Bennett, A., & Checkel, J. T. (Eds.). (2014). Process tracing: From metaphor to analytic tool. Cambridge, UK: Cambridge University Press. Bickel, R. (2007). Multilevel analysis for applied research. New York: The Guilford Press. Boddy, C. (2005). Projective techniques in market research: Valueless subjectivity or insightful reality? A look at the evidence for the usefulness, reliability and validity of projective techniques in market research. International Journal of Market Research, 47(3), 239–254. Boje, D. M. (1991). The storytelling organization: A study of story performance in an office-supply firm. Administrative Science Quarterly, 36(1), 106–126.

678

14

What Data Gathering Strategies Should I Use?

Bond, D., & Ramsey, E. (2007). Going beyond the fence: Using projective techniques as survey tools to meet the challenges of bounded rationality. In Proceedings of the 2007 Association for Survey Computing Meeting, The Challenges of a Changing World: Developments in the Survey Process, Southampton, UK (pp. 259–270). Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). New York: Routledge. Boslaugh, S. (2007). Secondary data sources for public health: A practical guide. New York: Cambridge University Press. Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27–40. Bowerman, B. L., O’Connell, R. T., & Koehler, A. B. (2005). Forecasting, time series, and regression: An applied approach (4th ed.). Belmont, CA: Brooks/Cole. Boyle, G. J., Saklofske, D. H., & Matthews, G. (Eds.). (2015). Measures of personality and social psychological constructs. Amsterdam: Academic Press. Boyd, D., & Crawford, K. (2012). Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, Communication & Society, 15(5), 662–679. Britten, N., Campbell, R., Pope, C., Donovan, J., Morgan, M., & Pill, R. (2002). Using meta ethnography to synthesise qualitative research: A worked example. Journal of Health Services Research & Policy, 7(4), 209–215. Brown, R. V. (2005). Rational choice and judgment: Decision analysis for the decider. Hoboken, NJ: Wiley. Bryant, A., & Charmaz, K. (Eds.). (2007). The Sage handbook of grounded theory. Los Angeles: Sage Publications. Bryman, A., & Bell, E. (2015). Business research methods (4th ed.). New York: Oxford University Press. Bryman, A., & Cramer, D. (2004). Constructing variables. In M. Hardy & A. Bryman (Eds.), Handbook of data analysis (pp. 17–34). London: Sage Publications. Buzan, T. (2003). The mind map book (Rev ed.). London: BBC Books. Buzan, T. (2018). Mind map mastery. London: Watkins. Byrne, B. M. (2010). Structural equation modelling with AMOS: Basic concepts, applications, and programming (2nd ed.). New York: Routledge. Campbell, R., Pound, P., Morgan, M., Daker-White, G., Britten, N., Pill, R., et al. (2011). Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technology Assessment, 15(43). Canter, D., Brown, J., & Groat, L. (1985). A multiple sorting procedure for studying conceptual systems. In M. Brenner, J. Brown, & D. Canter (Eds.), The research interview: Uses and approaches (pp. 79–114). London: Academic Press. Carmichael, G. A. (2016). Fundamentals of demographic analysis: Concepts, measures and methods. Switzerland: Springer. Carsey, T. M., & Harden, J. J. (2013). Monte Carlo simulation and resampling methods for the social sciences. Los Angeles: Sage Publications. Cassell, C., & Walsh, S. (2004). Repertory grids. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 61–72). London: Sage Publications. Catterall, M., & Ibbotson, P. (2000). Using projective techniques in education research. British Educational Research Journal, 26(2), 245–256. Charmaz, K. (2014). Constructing grounded theory (2nd ed.). London: Sage Publications. Chang, H. (2016). Autoethnography as method. London: Routledge. Chase, S. E. (2005). Narrative inquiry: Multiple lenses, approaches, voices. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 651–679). Thousand Oaks, CA: Sage Publications. Checkel, J. T. (2008). Process Tracing. In A. Klotz & D. Prakash (Eds.), Qualitative methods in international relations (Research Methods Series) (pp. 114–127). London: Palgrave Macmillan.

References

679

Checkland, P., & Poulter, J. (2006). Learning for action: A short definitive account of soft systems methodology and its use for practitioners, teachers and students. Chichester, UK: Wiley. Chell, E. (2004). Critical incident technique. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 45–60). London: Sage Publications. Chi, M. T. (2006). Laboratory methods for assessing experts’ and novices’ knowledge. In A. Ericsson, N. Charness, P. Feltovich, & R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 167–184). Cambridge. MA: Cambridge University Press. Chilisa, B. (2012). Indigenous research methodologies. Los Angeles: Sage Publications. Chilisa, B., & Tsheko, G. N. (2014). Mixed methods in indigenous research: Building relationships for sustainable intervention outcomes. Journal of Mixed Methods Research, 8 (3), 222–233. Chrzanowska, J. (2002). Interviewing groups and individuals in qualitative marketing research. London: Sage Publications. Clandinin, D. J. (Ed.). (2007). Handbook of narrative inquiry: Mapping a methodology. Thousand Oaks, CA: Sage Publications. Cochran, W. G., & Cox, G. M. (1957). Experimental designs (2nd ed.). New York: John Wiley & Sons. Cohen, I. G., Amarasingham, R., Shah, A., Xie, B., & Lo, B. (2014). The legal and ethical concerns that arise from using complex predictive analytics in health care. Health Affairs, 33 (7), 1139–1147. Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). New York: Routledge. Collis, J., & Hussey, R. (2009). Business research: A practical guide for undergraduate and postgraduate students. London: Palgrave Macmillan. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin. Cook, T. D., Campbell, D. T., & Peracchio, L. (1990). Quasi-experimentation. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 4, pp. 491–576). Palo Alto, CA: Consulting Psychologists Press Inc. Cooksey, R. W. (1996). Judgment analysis: Theory, methods, and applications. San Diego, CA: Academic Press. Cooksey, R. W. (2000). Mapping the texture of managerial decision making: A complex dynamic decision perspective. Emergence: A Journal of Complexity Issues in Organizations and Management, 2(2), 102–122. Cooksey, R. W. (2014). Illustrating statistical procedures: Finding meaning in quantitative data (2nd ed.). Prahran, VIC: Tilde University Press. Cooksey, R. W., Freebody, P., & Wyatt-Smith, C. (2007). Assessment as judgment-in-context: Analyzing how teachers evaluate students’ writing. Educational Research and Evaluation, 13 (5), 401–434. Cooksey, R. W., & Loomis, R. J. (1979). Visitor locomotor exploration of a museum gallery. Paper presented at the 50th Annual Meeting of the Rocky Mountain Psychological Association, Las Vegas, NV. Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2009). The handbook of research synthesis and meta-analysis. New York: Russell Sage Foundation. Corti, L., Thompson, P., & Fink, J. (2004). Preserving, sharing and reusing data from qualitative research: Methods and strategies. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 288–300). London: Sage Publications. Cowton, C. J. (1998). The use of secondary data in business ethics research. Journal of Business Ethics, 17, 423–434. Czarniawska, B. (2004). Narratives in social science research. Thousand Oaks, CA: Sage Publications. DeVellis, R. F. (2016). Scale development: Theory and applications. Los Angeles: Sage Publications.

680

14

What Data Gathering Strategies Should I Use?

Dick, P. (2004). Discourse analysis. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 203–213). London: Sage Publications. Duncan, M. (2004). Autoethnography: Critical appreciation of an emerging art. International Journal of Qualitative Methods, 3(4), 28–39. Durlak, J. A. (1995). Understanding meta-analysis. In L. G. Grimm & P. R. Yarnold (Eds.), Reading and understanding multivariate statistic (pp. 319–352). Washington, DC: American Psychological Association. Eden, C., & Ackermann, F. (2002). A mapping framework for strategy making. In A. Huff & M. Jenkins (Eds.), Mapping strategic knowledge (pp. 173–195). London: Sage Publications. Edmunds, H. (1999). The focus group research handbook. Lincolnwood, IL: NTC Business Books. Elliott, J. (2005). Using narrative in social research: Qualitative and quantitative approaches. London: Sage Publications. Elliott, H. (1997). The use of diaries in sociological research on health experience. Sociological Research Online, 2(2). Retrieved August 11, 2018, form http://www.socresonline.org.uk/2/2/7. html. Emerson, R. M., Fretz, R. I., & Shaw, L. L. (2011). Writing ethnographic fieldnotes (2nd ed.). Chicago: University of Chicago Press. Enders, W. (2014). Applied econometric time series (4th ed.). New York: John Wiley & Sons. Eppler, M. J. (2006). A comparison between concept maps, mind maps, conceptual diagrams, and visual metaphors as complementary tools for knowledge construction and sharing. Information Visualization, 5, 202–210. Ericsson, K. A. (2003). Valid and non-reactive verbalization of thoughts during performance of tasks: Towards a solution to the central problems of introspection as a source of scientific data. In A. Jack & A. Roepstorff (Eds.), Trusting the subject: The use of introspective evidence in cognitive science (Vol. 1, pp. 1–18). Exeter, UK: Imprint Academic. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: verbal reports as data (Rev ed.). Cambridge, MA: The MIT Press. Evans, W. (2015). Test wiseness: An examination of cue-using strategies. The Journal of Experimental Education, 52(3), 141–144. Farrington, D. P., & Knight, B. J. (1979). Two non-reactive field experiments on stealing from a ‘lost’ letter. British Journal of Social and Clinical Psychology, 18(3), 277–284. Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI magazine, 17(3), 37. Fielding, N. G., Lee, R. M., & Blank, G. (Eds.). (2008). The Sage handbook of online research methods. Los Angeles: Sage Publications. Finch, H., & Lewis, J. (2003). Focus groups. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice (pp. 170–198). Los Angeles: Sage Publications. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. Flick, U. (2014). An introduction to qualitative research (5th ed.). Los Angeles: Sage Publications. Fogarty, G. (2008). Principles and applications of education and psychological testing. In J. A. Athanasou (Ed.), Adult educational psychology (pp. 351–384). Rotterdam, Netherlands: Sense Publishers. Fothergill, S., Loft, S., & Neal, A. (2009). ATC-lab advanced: An air traffic control simulator with realism and control. Behavior Research Methods & Instrumentation, 41(1), 118–127. Frazer, L., & Lawley, M. (2000). Questionnaire design & administration. Milton, QLD: Wiley. Frumkin, N. (2015). Guide to economic indicators (4th ed.). London: Routledge. Gabriel, Y., & Griffiths, D. S. (2004). Stories in organizational research. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 114–126). London: Sage Publications. Galletta, A. (2013). Mastering the semi-structured interview and beyond: From research design to analysis and publication. New York: New York University Press. Gamst, G. C., Liang, C. T. H., & Der-Karabetian, A. (2011). Handbook of multicultural measures. Thousand Oaks, CA: Sage Publications.

References

681

Gandomi, A., & Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35(2), 137–144. Gilbert, N. (2008). Agent-based models. Thousand Oaks, CA: Sage Publications. Gilbert, N., & Troitzsch, K. G. (2005). Simulation for the social scientist (2nd ed.). New York: McGraw-Hill International. Gillham, B. (2005). Research interviewing: The range of techniques. Berkshire, UK: Open University Press. Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage Publications. Glass, G. V., Willson, V. L., & Gottman, J. M. (2008). Design and analysis of time-series experiments. Charlotte, NC: Information Age Publishing. Goodwin, J. (Ed.). (2012a). Sage secondary data analysis: Volume 1: Using secondary sources and secondary analysis. London: Sage Publications. Goodwin, J. (Ed.). (2012b). Sage secondary data analysis: Volume 2: Quantitative approaches to secondary analysis. London: Sage Publications. Goodwin, J. (Ed.). (2012c). Sage secondary data analysis: Volume 3: Qualitative data and research in secondary analysis. London: Sage Publications. Goodwin, J. (Ed.). (2012d). Sage secondary data analysis: Volume 4: Ethical, methodological and practical issues in secondary analysis. London: Sage Publications. Graco, W. J. (2001). Research into identification and classification of patterns of non-compliance in data using a doctor-shopper sample. Unpublished PhD thesis, School of Marketing and Management, University of New England, Armidale, NSW, Australia. Gray, D. E. (2014). Doing research in the real world (3rd ed.). Los Angeles: Sage Publications. Grimmer, J., & Stewart, B. M. (2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political analysis, 21(3), 267–297. Guest, G., MacQueen, K. M., & Namey, E. E. (2012). Applied thematic analysis. Los Angeles: Sage Publications. Guillemin, M. (2004,). Understanding illness: Using drawings as a research method. Qualitative Health Research, 14(2), 272–289. Gupta, V., & Lehal, G. S. (2009). A survey of text mining techniques and applications. Journal of Emerging Technologies in Web Intelligence, 1(1), 60–76. Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2014). A primer on partial least squares structural equation modelling (SEM). Los Angeles: Sage Publications. Han, J., Kamber, M., & Pei, J. (2012). Data mining: Concepts and techniques (3rd ed.). Waltham, MA: Morgan Kaufmann Publishers. Hand, D. J. (2004). Measurement theory and practice: The world through quantification. New York: John Wiley & Sons. Hancock, G. R. (2004). Experimental, quasi-experimental and nonexperimental design with latent variables. In D. Kaplan (Ed.), The Sage handbook of quantitative methodology for the social sciences (pp. 317–334). Thousand Oaks, CA: Sage Publications. Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed.). London: Routledge. Hamm, R. M. (1988). Moment-by-moment variation in experts’ analytic and intuitive cognitive activity. IEEE Transactions on Systems, Man, and Cybernetics, 18(5), 757–776. Hammond, K. R., Frederick, E., Robillard, N., & Victor, D. (1989). Application of cognitive theory to the student-teacher dialogue. In D. A. Evans & V. L. Patel (Eds.), Cognitive science in medicine. Biomedical modeling. Cambridge, MA: M1T Press (pp. 173–210). Hammond, K. R., & Stewart, T. R. (2001). The essential Brunswik: Beginnings, explications, applications. New York: Oxford University Press. Hammond, K. R., & Wascoe, N. E. (1980). Realizations of Brunswik’s representative design. San Francisco: Jossey-Bass. Harper, D. (2005). What’s new visually? In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 747–763). Thousand Oaks, CA: Sage Publications.

682

14

What Data Gathering Strategies Should I Use?

Heritage, J. (1984). Garfinkel and ethnomethodology. Cambridge, UK: Polity Press. Hinds, P. S., Vogel, R. J., & Clarke-Steffen, L. (1997). The possibilities and pitfalls of doing a secondary analysis of a qualitative data set. Qualitative Health Research, 7(3), 408–424. Hine, D. W., Montiel, C. J., Cooksey, R. W., & Lewko, J. (2005). Mental models of poverty in developing nations: A Canada-Philippines contrast. Journal of Cross-Cultural Psychology, 36 (3), 283–303. Hine, D. W., Gifford, R., Heath, Y., Cooksey, R., & Quain, P. (2009). A cue utilization approach for investigating harvest decisions in commons dilemmas. Journal of Applied Social Psychology, 39(3), 564–588. Hodson, R. (1999). Analyzing documentary accounts. Thousand Oaks, CA: Sage Publications. Hofferth, S. L. (2005). Secondary data analysis in family research. Journal of Marriage and Family, 67(4), 891–907. Hughes, J. & Goodwin, J. (Eds). (2014a). Documentary & archival research: Volume 1: Human documents – Perspectives and approaches. London: Sage Publications. Hughes, J. & Goodwin, J. (Eds). (2014b). Documentary & archival research: Volume 3: Human documents in social research. London: Sage Publications. Hughes, J. & Goodwin, J. (Eds). (2014c). Documentary & archival research: Volume 4: Archival research. London: Sage Publications. Hyland, M. (1981). Introduction to theoretical psychology. London: Macmillan Press. Irwin, S. (2013). Qualitative secondary data analysis: Ethics, epistemology and context. Progress in Development Studies, 13(4), 295–306. Jasper, J. D., & Shapiro, J. (2002). MouseTrace: A better mousetrap for catching decision processes. Behavior Research Methods, Instruments, & Computers, 34(3), 364–374. Johnson, B. (2001). Towards a new classification of nonexperimental quantitative research. Educational Researcher, 30(2), 3–13. Jones, S. H. (2005). Autoethnography: Making the personal political. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 763–791). Thousand Oaks, CA: Sage Publications. Jorgensen, D. (1989). Participant observation: A methodology for human studies. Newbury Park, CA: Sage Publications. Kane, M., & Trochim, W. M. K. (2007). Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage Publications. Kamberelis, G., Dimitriadis, G., & Welker, A. (2018). Focus group research and/in figured worlds. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (5th ed., pp. 692–716). Los Angeles: Sage Publications. Kaptchuk, T. J. (2001). The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf? Journal of Clinical Epidemiology, 54(6), 541–549. Keeves, J. P. (1997). Educational research, methodology, and measurement: An international handbook (2nd ed.). Oxford, UK: Pergamon. Kellehear, A. (1993). The unobtrusive observer: A guide to methods. St Leonards, NSW: Allen & Unwin. Keppel, G., & Wickens, T. D. (2004). Design and analysis: A researcher’s handbook (4th ed.). Upper Saddle River, NJ: Prentice Hall. Ker, J. S., Hesketh, E. A., Anderson, F., & Johnston, D. A. (2006). Can a ward simulation exercise achieve the realism that reflects the complexity of everyday practice junior doctors encounter? Medical Teacher, 28(4), 330–334. King, N. (2004). Using templates in the thematic analysis of data. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 256–270). London: Sage Publications. Kirk, R. E. (2013). Experimental design: Procedures for behavioral sciences (4th ed.). Thousand Oaks, CA: Sage Publications. Konstantopoulos, S., & Hedges, L. (2004). Meta-analysis. In D. Kaplan (Ed.), The Sage handbook of quantitative methodology for the social sciences (pp. 281–300). Thousand Oaks, CA: Sage Publications.

References

683

Kovach, M. (2009). Indigenous methodologies: Characteristics, conversations, and contexts. Toronto: University of Toronto Press. Kovach, M. (2018). Doing indigenous methodologies: A letter to a research class. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (5th ed., pp. 214–234). Los Angeles: Sage Publications. Krippendorf, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage Publications. Kumar, L., & Bhatia, P. K. (2013). Text mining: concepts, process and applications. Journal of Global Research in Computer Science, 4(3), 36–39. Kvale, S. (2007). Doing interviews. London: Sage Publications. L’Eplattenier, B. E. (2009). An argument for archival research methods: thinking beyond methodology. College English, 72(1), 67–79. Lamprianou, I. (2008). Introduction to psychometrics: The case of Rasch models. In J. A. Athanasou (Ed.), Adult educational psychology (pp. 385–418). Rotterdam, Netherlands: Sense Publishers. Lane, D. C. (1995). On a resurgence of management simulations and games. Journal of the Operations Research Society, 46(5), 604–625. Lane, S., Raymond, M. R., & Haladyna, T. M. (2015). Handbook of test development (2nd ed.). London: Routledge. Laukkanen, M. (1998). Conducting causal mapping research: Opportunities and challenges. In C. Eden & J. Spender (Eds.), Managerial and organizational cognition: Theory, methods and research (pp. 168–191). London: Sage Publications. Lee, A. T. (2005). Flight simulation: virtual environments in aviation. London: Routledge. Legard, R., Keegan, J., & Ward, K. (2003). In-depth interviews. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice (pp. 138–169). Los Angeles: Sage Publications. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage Publications. Louviere, J. J. (1988). Analyzing decision making: Metric conjoint analysis. Newbury Park, CA: Sage Publications. Maani, K. E., & Cavana, R. Y. (2007). Systems thinking, systems dynamics: Managing change and complexity (2nd ed.). North Shore, NZ: Pearson Education New Zealand. Malhotra, N. K., Hall, J., Shaw, M. & Oppenheim, P. P. (2008). Essentials of marketing research: An applied approach (2nd Ed.). French’s Forest, NSW: Pearson Education. Margolis, E., & Pauwels, L. (Eds.). (2011). The Sage handbook of visual research methods. Los Angeles: Sage Publications. Margolis, E., & Zunjarwad, R. (2018). Visual research. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (5th ed., pp. 600–626). Los Angeles: Sage Publications. Mathison, S. (1988). Why triangulate? Educational Researcher, 17(2), 13–17. McAuley, J. (2004). Hermeneutic understanding. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 192–202). London: Sage Publications. McDonald, S., Daniels, K., & Harris, C. (2004). Cognitive mapping in organizational research. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 73–85). London: Sage Publications. Meier, P. S. (2007). Mind-mapping: A tool for eliciting and representing knowledge held by diverse informants. Social Research Update, 52, 1–4. Mennecke, B., Roche, E., Bray, D., Konsynski, B., Lester, J., Rowe, M., et al. (2008). Second Life and other virtual worlds: A roadmap for research. Communications of the Association for Information Systems, 22(Article 20), 371–388. Retrieved August 11, 2018, from https://lib.dr. iastate.edu/cgi/viewcontent.cgi?article=1015&context=scm_pubs. Meyer, B. D. (1995). Natural and quasi-experiments in economics. Journal of Business and Economic Statistics, 13(2), 151–161. Miller, J. H., & Page, S. E. (2007). Complex adaptive systems: An introduction to computational models of social life. Princeton, NJ: Princeton University Press.

684

14

What Data Gathering Strategies Should I Use?

Milligan, G. W. (1981). A Monte Carlo study of thirty internal criterion measures for cluster analysis. Psychometrika, 46(2), 187–199. Minichiello, V., Aroni, R., & Hays, T. (2008). In-depth interviewing: Principles, techniques, analysis. Sydney: Pearson Education. Montgomery, P., & Bailey, P. H. (2007). Field notes and theoretical memos in grounded theory. Western Journal of Nursing Research, 29(1), 65–79. Mooney, C. Z. (1997). Monte Carlo simulation. Thousand Oaks, CA: Sage Publications. Muchiri, M. (2006). Transformational leader behaviours, social processes of leadership and substitutes for leadership as predictors of employee commitment, efficacy, citizenship behaviours and performance outcomes. Unpublished PhD thesis, New England Business School, University of New England. Murdock, S. H., Kelley, C., Jordan, J., Pecotte, B., & Luedke, A. (2016). Demographics: A guide to methods and data sources for media, business, and government. London: Routledge. Nisbett, R. E., & Ross, L. (1980). Human inference: Strategies and shortcomings of social judgment. Upper Saddle River, NJ: Prentice Hall. Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies (Vol. 11). Newbury Park, CA: Sage Publications. North, M. (2012). Data mining for the masses. Athens: Global Text Project. Retrieved August 11, 2018, from https://s3.amazonaws.com/academia.edu.documents/39657706/North_-_Data_ Mining_for_the_Masses_-_2012.pdf?AWSAccessKeyId= AKIAIWOWYYGZ2Y53UL3A&Expires=1533962896&Signature= 3h603Aq1BUPPSz2Svg0lC9RvwU0%3D&response-content-disposition=inline%3B% 20filename%3DData_Mining.pdf. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill. O’Cass, A. (1998). Reconceptualising and reconstructing consumer involvement: modelling involvement in a nomological network of relevant constructs: Casting the net wider or just fishing around. Unpublished PhD thesis, School of Marketing and Management, University of New England, Armidale, NSW, Australia. Olaru, D., Purchase, S., & Denize, S. (2009). Using docking/replication to verify and validate computational models. In Proceedings of the 18th World IMACS/MODSIM Congress, Cairns, Queensland (pp. 4432–4438). Retrieved August 11, 2018, from https://pdfs.semanticscholar. org/0edc/80bb4313dab4a0e0ee94a508a5a2883d4ebd.pdf. Omodei, M. M., & Wearing, A. J. (1995). The Fire Chief microworld generating program: An illustration of computer-simulated microworlds as an experimental paradigm for studying complex decision-making behavior. Behavior Research Methods, Instruments, & Computers, 27(3), 303–316. Onwuegbuzie, A. J., Leech, N. L., & Collins, K. M. (2010). Innovative data collection strategies in qualitative research. The Qualitative Report, 15(3), 696–726. Ortlipp, M. (2008). Keeping and using reflective journals in the qualitative research process. The Qualitative Report, 13(4), 695–705. Paterson, B. L., Thorne, S. E., Canam, C., & Jillings, C. (2001). Meta-study of qualitative health research: A practical guide to meta-analysis and meta-synthesis. Thousand Oaks, CA: Sage Publications. Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. New York: Cambridge University Press. Phellas, C. N., Bloch, A., & Seale, C. (2012). Structured methods: interviews, questionnaires and observation. In C. Seale (Ed.), Researching society and culture (3rd ed., pp. 181–205). London: Sage Publications. Pidd, M. (2009). Tools for thinking: Modelling in management science (3rd ed.). Chichester, UK: John Wiley & Sons. Pierce, C. A., & Aguinis, H. (1997). Using virtual reality technology in organizational behavior research. Journal of Organizational Behavior, 18(5), 407–410. Pink, S. (2013). Doing visual ethnography. Los Angeles: Sage Publications.

References

685

Place, U. T. (1992). The role of the ethnomethodological experiment in the empirical investigation of social norms and its application to conceptual analysis. Philosophy of the Social Sciences, 22 (4), 461–474. Plowright, D. (2011). Using mixed methods: Frameworks for an integrated methodology. Los Angeles: Sage Publications. Prasad, A. (2002). The contest over meaning: Hermeneutics as an interpretive methodology for understanding texts. Organizational Research Methods, 5(1), 12–33. Proctor, T. (2010). Creative problem solving for managers (3rd ed.). New York: Routledge. Provost, F., & Fawcett, T. (2013). Data science and its relationship to big data and data-driven decision making. Big Data, 1(1), 51–59. Punch, K. (2003). Survey research: The basics. London: Sage Publications. Railsback, S. F., & Grimm, V. (2012). Agent-based and individual-based modeling: A practical introduction. Princeton, NJ: Princeton University Press. Rapley, T. (2007). Doing conversation, discourse and document analysis. London: Sage Publications. Reynolds, C. R., Livingston, R. B., Willson, V. L., & Willson, V. (2010). Measurement and assessment in education (2nd ed.). Boston: Pearson Education International. Richmond, B. (2004). An introduction to systems thinking: STELLA software. Lebanon, OH: IEEE Systems. Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (Eds.). (1991). Measures of personality and social psychological attitudes. San Diego: Academic Press. Rosenthal, R. (1984). Meta-analytic procedures for social research. Beverly Hills, CA: Sage Publications. Ross, G. (2001). Visual methodologies: An introduction to the interpretation of visual materials. London: Sage Publications. Rowlinson, M. (2004). Historical analysis of company documents. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 301–311). London: Sage Publications. Rymaszewski, M., Au, W. J., Wallace, M., Winters, C., Ondrejka, C., & Batstone-Cunningham, B. (2007). Second life: The official guide. Hoboken, NJ: Wiley. Sandall, J. L. (2006). Navigating pathways through complex systems of interacting problems: Strategic management of native vegetation policy. Unpublished PhD thesis, New England Business School, University of New England, Armidale, NSW, Australia. Sandelowski, M., Docherty, S., & Emden, C. (1997). Qualitative metasynthesis: Issues and techniques. Research in Nursing & Health, 20(4), 365–371. Samra-Fredericks, D. (2004). Talk-in-interaction/conversation analysis. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 214–227). London: Sage Publications. Sapsford, R. (2007). Survey research (2nd ed.). London: Sage Publications. Schembri, S., & Boyle, M. V. (2013). Visual ethnography: Achieving rigorous and authentic interpretations. Journal of Business Research, 66(9), 1251–1254. Schkade, D. A., & Payne, J. W. (1994). How people respond to contingent valuation questions: A verbal protocol analysis of willingness to pay for an environmental regulation. Journal of Environmental Economics and Management, 26(1), 88–109. Schmidt, F. L., & Hunter, J. E. (2014). Methods of meta-analysis: Correcting error and bias in research findings. Los Angeles: Sage Publications. Schmitt, R. (2005). Systematic metaphor analysis as a method of qualitative research. The Qualitative Report, 10(2), 358–394. Schreiber, R., Crooks, D., & Stern, P. N. (1997). Qualitative meta-analysis. In J. Morse (Ed.), Completing a qualitative project: Details and dialogue (pp. 311–326). Thousand Oaks, CA: Sage Publications. Schreier, M. (2012). Qualitative content analysis in practice. Los Angeles: Sage Publications. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2001). Experimental and quasi-experimental designs for generalized causal inference (2nd ed.). Boston: Cengage.

686

14

What Data Gathering Strategies Should I Use?

Shum, D., O’Gorman, J., Creed, P., & Myors, B. (2017). Psychological testing and assessment ebook (3rd ed.). New York: Oxford University Press. Senge, P., Kleiner, A., Roberts, C., Ross, R., & Smith, B. (1994). The fifth discipline field book. London: Nicholas Brealey. Shiratori, R., Arai, K., & Kato, F. (2005). Gaming, simulations and society: Research scope and perspective. Tokyo: Springer. Sloan, L., & Quan-Haase, A. (Eds.). (2017). The Sage handbook of social media research methods. Los Angeles: Sage Publications. Small, S. D., Wuerz, R. C., Simon, R., Shapiro, N., Conn, A., & Setnik, G. (1999). Demonstration of high-fidelity simulation team training for emergency medicine. Academic Emergency Medicine, 6(4), 312–323. Smith, A. E., & Humphreys, M. S. (2006). Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping. Behavior Research Methods, 38(2), 262–279. Smith, E. (2008). Pitfalls and promises: The use of secondary data analysis in educational research. British Journal of Educational Studies, 56(3), 323–339. Smith, M. (2007). Research methods in accounting. Los Angeles: Sage Publications. Solórzano, D. G., & Yosso, T. J. (2002). Critical race methodology: Counter-storytelling as an analytical framework for education research. Qualitative Inquiry, 8(1), 23–44. Stengel, D. N., & Chaffe-Stengel, P. (2012). Working with economic indicators: Interpretation and sources. New York: Business Expert Press. Sterman, J. S. (2000). Business dynamics: Systems thinking and modeling for a complex world. New York: McGraw-Hill/Irwin. Stewart, D., Shamdasani, P. N., & Rook, D. W. (2007). Focus groups: Theory and practice (2nd ed.). Thousand Oaks, CA: Sage Publications. Stiles, D. (2004,). Pictorial representation. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 127–140). London: Sage Publications. Stuart, E. A., & Rubin, D. B. (2008). Best practice in quasi-experimental designs: Matching methods for causal inference. In J. W. Osborne (Ed.), Best practices in quantitative methods (pp. 155–176). Los Angeles: Sage Publications. Sue, V. M., & Ritter, L. A. (2012). Conducting online surveys (2nd ed.). Los Angeles: Sage Publications. Suri, H. (2011). Purposeful sampling in qualitative research synthesis. Qualitative Research Journal, 11(2), 63–75. Symon, G. (2004). Qualitative research diaries. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 98–113). London: Sage Publications. Taber, C. S., & Timpone, R. J. (1996). Computational modeling. Thousand Oaks, CA: Sage Publications. Tansey, O. (2007). Process tracing and elite interviewing: A case for non-probability sampling. PS: Political Science & Politics, 40(4), 765–772. Thornton III, G. C., & Kedharnath, U. (2013). Work sample tests. In K. F. Geisinger et al. (Eds.), APA handbook of testing and assessment in psychology, Vol. 1. Test theory and testing and assessment in industrial and organizational psychology (pp. 533–550). Washington, DC: American Psychological Association. Trenor, J. M., Miller, M. K., & Gipson, K. G. (2011). Utilization of a think-aloud protocol to cognitively validate a survey instrument identifying social capital resources of engineering undergraduates. In Electronic Proceedings of the American Society for Engineering Education Annual Conference and Exposition, Vancouver, CA (pp. 22.1656.1–22.1656.15). Retrieved August 11, 2018, from https://peer.asee.org/utilization-of-a-think-aloud-protocol-tocognitively-validate-a-survey-instrument-identifying-social-capital-resources-of-engineeringundergraduates. van Someren, M. W., Barnard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method: A practical approach to modeling cognitive processes. London: Academic Press. Veal, A. J. (2005). Business research methods: A managerial approach (2nd ed.). French’s Forest, NSW: Pearson Education.

References

687

Walker, S. J. (2014). Big data: A revolution that will transform how we live, work, and think. International Journal of Advertising, 33(1), 181–183. Walsh, J. J. (1997). Projective testing techniques. In J. P. Keeves (Ed.), Educational research, methodology and measurement: An international handbook (2nd ed., pp. 954–958). Oxford, UK: Pergamon. Walsh, S., & Clegg, C. (2004). Soft systems analysis: Reflections and update. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 334– 348). London: Sage Publications. Walsh, D., & Downe, S. (2005). Meta-synthesis method for qualitative research: A literature review. Journal of Advanced Nursing, 50(2), 204–211. Waddington, D. (2004). Participant observation. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 154–164). London: Sage Publications. Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage Publications. Webb, E. J., Campbell, D. T., Schwartz, R. D., & Sechrest, L. (2000). Unobtrusive measures (Rev ed.). Thousand Oaks, CA: Sage Publications. Webster, J. G. (2015). The physiological measurement handbook. Boca Raton: CRC Press. Wilcox, R. R. (1997). Simulation as a research technique. In J. P. Keeves (Ed.), Educational research, methodology and measurement: An international handbook (2nd ed., pp. 150–154). Oxford, UK: Pergamon. Wilensky, U. (1999). NetLogo. Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL. Retrieved August 11, 2018, from http://ccl. northwestern.edu/netlogo/. Wilensky, U. (2003). NetLogo Traffic Grid model. Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL. Retrieved August 11, 2018, from http://ccl.northwestern.edu/netlogo/models/TrafficGrid. Williams, J. H. (2014). Defining and measuring nature: The make of all things. San Raphael, CA: Morgan & Claypool. Wybo, J. L. (2008). The role of simulation exercises in the assessment of robustness and resilience of private or public organizations. In H. J. Pasman & I. A Kirillov (Eds.), Resilience of cities to terrorist and other threats (pp. 491–507). Dordrecht: Springer. Yamarone, R. (2017). The economic indicator handbook: How to evaluate economic trends to maximize profits and minimize losses. Hoboken, NJ: Wiley. Yin, R. K. (2011). Qualitative research from start to finish. New York: The Guilford Press.

Chapter 15

How Do I Handle Academic Integrity Issues?

15.1

What Is Academic Integrity?

In Chap. 10, we introduced two key aspects of research contextualisation: positioning of the researcher and positioning of participants. The former encompassed codes of professional conduct for researchers and the latter encompassed ethical guidelines and codes covering human participants. In this chapter, we will dig more deeply into these matters as they relate to the more general issue of academic integrity. What we will see here is that academic integrity can be seen simply as patterns of strategies for anticipating, adapting and/or responding to pressures and influences that arise from specific contextual sources and positioning stances. While, initially, most postgraduate students are simply concerned with getting ethics approval for their research, they soon learn that there is a host of matters that can create potential problems, both for them and for others, that would best be termed ‘academic integrity issues’. Academic integrity relates primarily to accuracy, honesty, safety, truthfulness and adherence to the highest standards in all aspects of research undertaken and reported. Academic integrity issues can arise not only with research contextualisation, positioning, configuration and data gathering, but also in the writing up, dissemination, and use of your research results. Issues such as maintaining confidentiality and ethical treatment of participants, storage of data, the appropriate recording and reporting of research findings, as well as concerns specific to the publication process in regard to plagiarism and authorship credit are implicated. As well, academic integrity encompasses how you relate to key stakeholders, groups and organisations that you are associated with or are reliant upon. Academic integrity involves not only you, as the researcher, but everyone with whom you interact as well as potential stakeholders in your research. For example, those potentially impacted by academic integrity issues are: • Research participants—participants in your research, whose privacy, dignity, safety and well-being must be protected. © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_15

689

690

15

How Do I Handle Academic Integrity Issues?

• Organisation(s)—the organisations, institutions, groups or communities from which your participants have been obtained or which themselves form the focus of your research. A wider view also includes the physical environment within which your research is conducted. Also implicated here are bodies that may be funding your research (perhaps directly or via a supervisor). Finally, there are implications if your research has impacts on or potential to impact on any organisations, institutions, groups or communities. • Your university—the institution at which you are enrolled in a research program, typically governed by a code of conduct, and the reputational effect that your research might have as well as issues of professional liability. • Discipline scholars—other academics within the discipline(s) you are researching. Minimising any potential negative impacts your research could have on the academic community within those disciplines need to be considered. This extends to members of your profession, which may be associated with a formal professional body such as the Australian Psychological Association, the Australian and New Zealand Academy of Management, American Educational Research Association, British Psychological Society and so on. Many of these professional bodies will have codes of practice which their members must adhere to in their research and professional practices. Doctoral candidates can often become associate or affiliate members of a relevant professional body, which adds value to your career prospects and extends your professional networks, but with the expectation of conformity with that body’s code of conduct. • Your employer—if you are employed while undertaking your research journey, then you must consider how your research could affect your employment conditions and relationships. This is especially a concern in research frames such as Action, Evaluation or Developmental Evaluation, where you may be researching or innovating within your own organisation. • You, the researcher—academic integrity issues that could relate specifically to you and impact on your well-being, reputation and career. The terms ‘academic integrity’, ‘research misconduct’ and ‘research ethics’ often appear together in the same context. They all relate to areas where postgraduate students need to be mindful of their professional responsibilities and the pitfalls which could result in a failed degree or a considerable black mark on their academic career, as well as the possibility of adverse effects on others. Research misconduct can be defined as the fabrication, falsification, plagiarism or deception in proposing, conducting or reporting results of research as well as deliberate, dangerous or negligent variations from accepted practices when carrying out research. Misconduct includes failure to follow established protocols, creating unreasonable risk or harm to organisations and/or individuals, and collusion or concealment of the misconduct of others. Misconduct can also include the intentional unauthorised use, disclosure, removal, or damage to, research-related property of another (e.g. materials, writings, data, hardware or software). Where research misconduct is intentional (i.e., the researcher knowingly varies from expected standards of research behaviour), the consequences can be quite severe and career-destroying.

15.1

What Is Academic Integrity?

691

There are many ways in which researchers may deviate from the standards for undertaking high-integrity research, including but not limited to: • fabrication of data or of results from analyses; • falsification or misrepresentation of results; • plagiarism or other misuse of the words, ideas or intellectual property of another researcher; • misleading ascription of authorship; • failure to declare and manage serious conflicts of interest; • falsification or misrepresentation to obtain funding; • conducting research without ethics approval; • risking the safety of human participants, or the well-being of animals or the environment; • gross or persistent negligence; or • wilful concealment or facilitation of research misconduct by others. Every university will have policies and documentation covering codes of conduct and what constitutes research standards, calling for high levels of academic integrity and adherence to ethical principles. These govern both research students and academic staff. Failure to adhere to these requirements can see a rather dramatic end to a career. One of the more notable cases of research misconduct is that of Diederik Stapel who in 2010 was accused of publishing 55 fraudulent papers and was fired from the University of Tilburg (Faria, 2018). It has been observed that with the pressure on doctoral students to make an original contribution to knowledge and the confronting of complex research processes, there might arise the circumstances where students adopt practices that compromise acceptable research standards (Mitchell & Carroll, 2008). As a postgraduate student, it is therefore recommended that you obtain the relevant policies pertaining to your university as well as reading up on research ethics. Failure to uphold relevant and applicable research codes of conduct could call an abrupt halt to your research and impose possible restrictions on the continuation your studies. It is also now not uncommon to require students to complete an Academic Integrity Module. Failure to complete the module or tutorial with a Pass could result in their registration being blocked for future terms.

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

When considering the issues of academic integrity in research, one is tempted to immediately bring the issues of plagiarism and appropriate referencing to mind. However, while certainly important, these are only the tip of the iceberg and it is worth looking at some of the more notable areas where you might be affected and what strategies you can enact to protect yourself.

692

15

How Do I Handle Academic Integrity Issues?

15.2.1 Intentional and Unintentional Harm As a researcher, you have a responsibility for ensuring that the privacy, safety, health, cultural/social sensitivities and welfare of those involved in your research are adequately protected. In addition, you need to ensure that your research is conducted in an environment that is safe for you, as well as for the participants and those who are impacted on, either directly or indirectly, by your research. When designing research and gaining ethics approval, the overarching concern is the prevention of harm. It is important to note that the risk, or potential for harm, is as important as actual harm. Harm can be any outcome from your research which adversely affects the interests or welfare of an individual or a group. Harm can come in many forms and can include, for example, discomfort, stress, anxiety, injury, pain, fatigue, embarrassment and inconvenience (Unitec Human Research Ethics Guidelines, http://www.unitec.ac.nz/epress/wp-content/uploads/2016/09/ unitec-human-research-ethics-guidelines.pdf). The types of harm may be categorised as psychological, physical, economic or social, as described below: • Psychological harm—your research must not humiliate, offend and/or create stress or anxiety for participants, both in the short term and possibly also in the long term. • Physical harm—a research intervention or other data gathering activities must not yield physical or health manifestations of harm. This is more common in animal and medical research and you will need to prove that no participants will be physically harmed in the course of your study and that participants will not even be in any discomfort. • Economic harm—the execution of your research must not result in any loss, e.g., damaging sales for the organisation or loss of income, money or property for an individual, group or organisation. • Social harm—a person’s social standing will not be affected as a result of your research, for example, disclosing a person’s sexual orientation (Polonsky & Waller, 2015). If you are undertaking experimental research, consider what the implications might be for the people in both the experimental and control groups. A classic example of this is the Milgram study in the 1960s where participants were asked to administer potentially lethal electric shocks as punishment for errors to individuals who were behind a screen participating in a learning task. The actors participating in the study as the shocked recipients would duly scream in pain. To be expected, the study caused considerable distress and psychological harm to participants. Will people assigned to the control group be harmed in any way, or will they miss out on anything that could be beneficial? If so, can this be justified? One notable research study in New Zealand, culminating in the Cartwright Enquiry, resulted in significant changes to general ethics approval processes in the health sector. The research was conducted at the National Women’s Hospital where, in an effort to prove that carcinoma in situ does not always progress to invasive cancer, a

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

693

number of women in a control group were left untreated. The experiment, conducted over a sixteen-year period, significantly compromised the health of many women and resulted in eight potentially preventable deaths. Suggested strategies are: • Consider all stages of your research—in order to avoid problems and potential litigation, it is appropriate to consider where harm could occur in your research. Once again, the harm may not necessarily be overt and/or intentional. Depending upon who your potential participants are, you may have to consider possible cultural and social impacts of your research as well as potential issues arising from power differentials or colonialist assumptions. This is especially important if you will be working with Indigenous participants, conducting research on Indigenous land or gathering/reporting/using Indigenous knowledge or stories (see Evans, Hole, Berg, Hutchinson, & Sookraj, 2009, for an exploration of Indigenous methodologies and research issues). • Relevance—your research should be adequately designed to meet the objectives of the project and, therefore, not be considered trivial or culturally insensitive by participants. • Use of Participant Information sheets—Participant Information Sheets describe, in the participant’s language, the essential points which any reasonable person would wish to know before agreeing to participate. For example, a Participant Information Sheet would answer questions such as what is your university affiliation, why you are undertaking the research, what you are researching, why you wish them to provide their views and opinions, what you intend to do with the information, how you will protect their identity and the information they provide, as well as who is supervising your research (see Appendix: Information to Include in a Participant Information Sheet). • Use Consent Forms—ensure informed consent forms are signed by all participants prior to commencement of their involvement in your research. The requirements on the consent form will be specified by your university. Consent forms usually have a brief explanation of the research and the ethics approval statement which states the date ethics approval was given (with an approval number, where relevant). If you are putting a draft consent form with your documentation for ethics approval, leave the date blank. The consent form should also contain your contact details as well as clauses in bullet form about confidentiality, participants’ right to exit the study at any time and what participants should do if they have any concerns or complaints. A copy of the participant information sheet and consent form should be retained by the participant. Note that exemptions from obtaining written consent might apply to mass-distribution or web-based questionnaires and very simple research procedures, for example, hearing tests or procedures where the participant’s ignorance of the intended research objective is essential (this latter must be exercised with extreme caution and with proper justification, as it essentially involves deception, to be discussed further below). For some questionnaires, particularly anonymously returned questionnaires, the return of the questionnaire can be reasonably taken as an

694

15

How Do I Handle Academic Integrity Issues?

indication of voluntary consent to participate. If this is the case, that fact should be clearly stated on the questionnaire and participant information sheet. Typically, research conducted based solely upon data obtained from secondary or archival databases does not require informed consent, especially if the database is publicly available. However, always check with your university’s requirements before assuming that you do not need to obtain informed consent for your specific project. The practice of not obtaining informed consent can have serious ramifications as the following case depicts: In 1932, the US Public Health Service, working with the Tuskegee Institute, began a study to record the natural history of syphilis in hopes of justifying treatment programs. The study initially involved 600 black men – 399 with syphilis, 201 who did not have the disease. The study was conducted without the benefit of patients’ informed consent. Researchers told the men they were being treated for ‘bad blood’, a local term used to describe several ailments, including syphilis, anemia, and fatigue. In truth, they did not receive the proper treatment needed to cure their illness. In exchange for taking part in the study, the men received free medical exams, free meals, and burial insurance. Although originally projected to last 6 months, the study actually went on for 40 years. In July 1972, an Associated Press story about the Tuskegee Study caused a public outcry that led the Assistant Secretary for Health and Scientific Affairs to appoint an ad hoc Advisory Panel to review the study. The panel found that the men had agreed freely to be examined and treated. However, there was no evidence that researchers had informed them of the study or its real purpose. In fact, the men had been misled and had not been given all the facts required to provide informed consent. The men were never given adequate treatment for their disease. The advisory panel found nothing to show that subjects were ever given the choice of quitting the study, even when this new, highly effective treatment became widely used. The advisory panel concluded that the Tuskegee Study was ‘ethically unjustified’ – ‘the knowledge gained was sparse when compared with the risks the study posed for its subjects’. A class action followed in 1973 with a $10 million out-of-court settlement reached in 1974 and a Health Benefit Program established. Wives, widows and offspring were added to the program in 1975 (U.S. Public Health Service Syphilis Study at Tuskegee (2015). Centers for Disease Control and Prevention, Department of Health and Human Services, http://www.cdc.gov/ tuskegee/timeline.htm).

15.2.2 Cultural Insensitivity People of different cultures hold differing beliefs, have different value systems and regard differing modes of behaviour as acceptable to them. If you are using an Indigenous or Cross-Cultural research frame, this becomes a very important issue for you to deal with. When dealing with matters which are often deeply personal and private, procedures for research can very easily cause offence, both to individuals and to ethnic groups, even though none has been intended (Health Research Council New Zealand, http://www.hrc.govt.nz/). It is now expected that due recognition is given to the increasing diversity of culture and religious beliefs which are now appearing in society, including those of Indigenous cultures. Practices and beliefs of an ethnic and/or religious nature must be fully respected and research

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

695

must be undertaken in a culturally sensitive and appropriate manner in full discussion and partnership with the research participants, whatever their ethnicity or religious affiliation, and the results of any investigation should be appropriately safeguarded and disseminated. In the case of Indigenous research, the issue of cultural sensitivity extends to seeking access to Indigenous community members by going through appropriate channels, involving tribal elders. When research involves participants from differing ethnic or cultural groups, even when small numbers from each group are involved, it is important that issues regarding cultural respect and safety are addressed. Where participants are sought from a particular social group, consideration must be given to the particular needs of those participants and how harm minimisation and knowledge safeguarding might best be addressed. It is also important to recognise that, in the case of Indigenous research, it is critical that you examine closely the positioning and guiding assumptions underpinning the research relative to the needs, expectations and cultural context of the people you would like to work with in your research, particularly if you are a white researcher working in an Indigenous context (Evans et al., 2009). Suggested strategies are: • Preparation—a prior understanding of cultural issues that may impact on the research and information should be sought. For example, the New Zealand Health Research Council Publication Guidelines provides detailed advice when researching in a Māori cultural context (http://www.hrc.govt.nz/sites/default/ files/Te%20Ara%20Tika%20Guidelines%20for%20Maori%20Research% 20Ethics.pdf). There is a similar set of guidelines in Australia (available from the https://nhmrc.gov.au/research-policy/ethics/ethical-guidelines-researchaboriginal-and-torres-strait-islander-peoples). Such guidelines form the general foundation for the Indigenous research ethics approval processes employed at most Australian universities, despite their original formulation to guide health research. Canada has its own guidelines for research involving First Nation, Inuit and Métis peoples (see http://www.pre.ethics.gc.ca/eng/policy-politique/ initiatives/tcps2-eptc2/chapter9-chapitre9/). A more general source guide for research ethics and Indigenous research can be found at the Indigenous Geography.net website (http://www.indigenousgeography.net/ethics.shtm). • Consultation—where a particular ethnic or cultural group is the focus of your research, consultation must be undertaken with appropriate parties. For example, the Australian NHMRC guidelines state the following with respect for any research involving Indigenous participants: Recognising and respecting diversity throughout the research journey helps to initiate, develop and sustain partnerships and relationships with Aboriginal and Torres Strait Islander Peoples and communities that are based on trust, mutual responsibility and ethics. When conducting research that includes both Aboriginal and Torres Strait Islander Peoples, researchers must consult and work with relevant stakeholders from both groups (NHMRC, 2018, p. 2).

• Uses of language—participants have the right to receive, in language that they will easily understand, information about proposed research in which they are

696

15

How Do I Handle Academic Integrity Issues?

being invited to participate. Where large numbers of participants from an ethnic group are being recruited, the involvement of a trained interpreter is highly desirable. If the number of participants from any ethnic group is small, the use of trained interpreters to read and discuss the information sheet with the participant may obviate the need for a printed translation of all research material (for example, see the New Zealand Health Research Council Pacific Health Research Guidelines, https://gateway.hrc.govt.nz/funding/downloads/Pacific_ health_research_guidelines.pdf). • Appropriate tailoring of data gathering strategies—in a Cross-Cultural research frame, you need to be aware of cultural/religious conventions in the cultural group from whom you are drawing participants. If, for example, you plan to conduct interviews, you need to gauge the appropriateness of matching interviewer and interviewee gender in the interview. In some Islamic cultures, a woman may not be permitted to interview a man and vice versa. In an Indigenous cultural context, it may not be possible to conduct one-on-one interviews with community members, you may be required to include a tribal elder in each interview. • Appropriate dissemination of research findings—you should carefully consider how the results of your research will be most usefully shared with and used by the participants as well as by the wider group/s to which the participants belong. Furthermore, for Indigenous research especially, just how research knowledge should be best shared with the outside world also needs to be carefully negotiated. If you disseminate research information or tribal knowledge in inappropriate, unapproved or untimely ways, harm may be caused to participants and good will may be destroyed. Consultation with the participant group should direct the you to the most appropriate processes in this regard.

15.2.3 Coercion An unethical practice in research is the use of coercion in order to increase participation rates. For example, saying “I need you to fill out this survey or I will fail subject X”, would be inappropriate as it places unnecessary social pressure on potential respondents (Polonsky & Waller, 2015). The offering of inducements to potential participants would also be considered a form of coercion, however, it is recognised that in some cultures the giving of a small gift as recompense for participation is deemed to be appropriate, for example, the use of ‘koha’ in Māori culture involves the passing on of a small amount of money usually to cover hospitality costs. When researching with Pacifica participants, the bringing of food (cake is acceptable) is also quite appropriate. In these circumstances, the offering is one of social practice and acknowledgement rather than of inducement, that is, they were going to answer your questions anyway. Research that is deemed particularly sensitive is where ‘vulnerable people’ or minors are involved as participants and their susceptibility to coercion is even stronger.

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

697

Suggested strategies are: • Be sensitive—coercion may not originate from the researcher directly. A boss may be coercing employees to be part of your study. You also need to be aware of situations where participants could perceive that they are being coerced. Thus, you need to be sensitive to where coercion could occur, both directly and indirectly, and take concrete steps to negate it. • Avoid inducements—unless culturally appropriate, be careful of any payment that would encourage an otherwise reluctant participant. Where payments are made, they should be a token gesture. It is important to recognise that this issue is continually being revisited, especially in marketing research, where participants may be paid for their participation or where participation (say, in a mail or web survey) may be rewarded by entry into a draw for a prize or by a contribution to a named charity. The issue remains debateable because, for many, there is a clear if fine distinction between ‘incentive’ to participate and ‘inducement’ to participate. Inducement implies at least indirect application of power in favour of participation whereas incentive implies free choice to either participate or not. It is appropriate to use monetary incentives for people who have joined an organisation, such as SurveyMonkey (see https://www. surveymonkey.com/mp/audience/?ut_source=megamenu) whose role is to provide a participant pool as well as survey technology for researchers, in areas such as marketing and consumer research. In such cases, people join these organisations on a volunteer basis, with an understanding that they will get paid for their participation in research. It is also a common practice for university research to be conducted with students who volunteer to participate in exchange for extra course credit. This situation is different from a situation where students may be required to participate in a research study in order to pass their course. The former situation would be classed as offering an incentive with participation being a choice (as long as grades are not dependent upon participation); the latter would constitute a form of coercive inducement.

15.2.4 Deception Deception can occur in the gathering, configuration, analysis and/or reporting of research. One important type of deception, call it ‘misleading participants’, occurs if a researcher represents their research as something other than what it is. This may take the form of deceiving participants as to: the true purpose of the research; the methods that will be used to gather the data; participants’ actual role in the research; the uses to which their data will be put; or any other action that limits, obscures or falsely informs participants’ understanding of what the research is actually about, what information they are really providing and how the research may impact on them. An example of the ‘misleading participants’ deception in organisational research is the case of the researcher who had participants undertake a financial puzzle task but provided individuals with false feedback about their performance

698

15

How Do I Handle Academic Integrity Issues?

(Staw, 1975). This deception was considered to be relatively mild compared with, for example, telling study participants that a personality test had revealed that they had latent homosexual tendencies or leading subjects to believe that they had administered potentially lethal electrical shocks (Stone, 1981). Another more notable study by Humphrey (2017) used ‘misleading participants’ deceit to gain information that would otherwise be inaccessible. Humphrey acted as a lookout in tearoom toilets used by gays for sex. While pretending to watch on behalf of the police, he was actually collecting data for his own research, which included recording car licence plate numbers, discovering where these men lived and later visiting the men he had identified in their homes pretending, once again, that they were randomly chosen to take part in a survey study. Clearly, there were multiple levels of deceit in this research (Humphrey, 2017). The ‘misleading participants’ deception also occurs when false assurances or no assurances are given to participants about how their data will be safeguarded or how information they provide might be used. Consider, for example, recording observations on people without their knowledge (e.g., covert observation) or using data recording for one purpose (e.g., store security camera recordings) for another purpose that people are unaware of (e.g., using the same security footage to make observations about consumer behaviour). In these instances, ethical principles are violated because people have not given their informed consent to participate nor have they enjoyed the right not to participate; in most cases, they will not even know that their behaviour is providing data for some researcher. Another type of deception, call it ‘misleading research audiences’, occurs if a researcher omits data or analyses data in ways that obscure or hide problems with data quality and, when reporting results, if they are written to favour a particular outcome or mask a problem. In other words, have the researchers been honest in telling the story of their research or have they presented a ‘tidier’ or more publicly or academically palatable version? This is technically a milder form of data falsification, not quite as serious, but still ethically objectionable, unprofessional and fraudulent. Martin (1999) discussed the issue of suppression of research data and focused on two sides of the problem. One side is where outside bodies (e.g., a tobacco company, a government, and chemical manufacturer or pharmaceutical company) put pressure on researchers to alter how they convey their research findings and even which findings they can or cannot report, i.e., deception for political purposes, where the researcher is not directly at fault. The other side is where researchers knowingly alter how they convey their research findings and/or which findings they report, in anticipation that some outside body might object if they were fully transparent. In this latter case, the researcher is directly culpable in deceiving research audiences, i.e., deception to avoid censure and adverse career impacts. ‘Misleading research audiences’ deception is a continually fraught area, which brings into the light a dark side of research practice. If you are conducting research being funded by a body that has a political or market-based agenda, you need to be aware of these kinds of pressures—this can be where the role of being a rigorous high-integrity researcher collides with one’s own human nature and survival instincts.

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

699

Suggested strategies for avoiding the various types of deception are: • Reinforcing ability to control their own participation—ensuring that all participants know that (1) they have the right not to participate at all and (2) they have the right to exit from the study at any stage. • Obtaining informed consent—ensuring that all participants give their informed consent to participate where ‘informed’ means spelling out very carefully and honestly what your research is about, how it may potentially affect a participant and what safeguards (e.g., confidentiality, anonymity, de-identification, when the raw data records will be destroyed) will be provided for any information they provide. • Using data only for the purposes signalled to participants—ensuring that all participants understand exactly what their data will be used for and who will have access to them. This is especially critical in organisational research where participants may be concerned that information or comments they provide may find their way back to their boss, adversely affecting their employment or work relationships. • Think strategically about funding/sponsorship/scholarship sources—Carefully consider the ramifications of seeking and accepting funding, sponsorship or scholarship for your research from a body or group that you know, or suspect has a political or market-based agenda. If your supervisor has such funding and your project would come under the umbrella of that funding, be sure to discuss the potential deception risks and pressures that could emerge and devise clear strategies for handling them.

15.2.5 Loss of Confidentiality or Anonymity Loss of confidentiality or anonymity is a serious problem in research and is worthy of some extended discussion. It can occur at numerous points in your research and, more notably, at times of data gathering, record-keeping, dissemination of your findings and disposal of research materials and may be relevant not only for individual participants but also for participant organisations. You need to take particular care in qualitative research because the kind of data gathered may require use of direct quotes, a personal reference or context which, with small samples, means that participant identities can easily be figured out. It is important to realise that promising confidentiality and promising anonymity are not the same thing. If you assure confidentiality, you must also indicate confidential to whom, meaning that someone will know a participant’s true identity and you are obliged to diverge who this is to the participant as part of the informed consent process. Absolute anonymity guarantees that no one, not even the researcher, will know a participant’s true identity. Partial anonymity guarantees that the researcher will take all necessary steps to hide the participant’s identity in the data file used for analysis as well as in any and all research outcomes. Whether you promise confidentiality, partial or

700

15

How Do I Handle Academic Integrity Issues?

absolute anonymity depends upon your research frame, configuration and data gathering strategies. For example, in the Survey research frame, a mailed or web-based questionnaire can be rendered absolutely anonymous through the use of ID numbers assigned to returned instruments. With interaction-based data gathering strategies, absolute anonymity cannot be guaranteed, so you must promise both confidentiality as well as partial anonymity, so as to safeguard participant’s identity in the database you create as well as in all research outcomes. It may be possible to come close to absolute anonymity if you assign a pseudonym to every person interviewed, destroy any link between that pseudonym and the participant’s identity (except in the memory of the interviewer or original data transcriber) and the pseudonym is the only identifier in any data records. In longitudinal research configurations, participants will have data gathered from them on multiple occasions, which means you must be able to track identities to ensure you can match data to the correct participant on each occasion. In such cases, confidentiality is the most important guarantee, but usually will be linked to partial anonymity as well. In sequential research configurations, you may use one data gathering strategy (e.g., a questionnaire) to serve as a screening/recruiting tool for selecting a subset of participants for a second phase of research (e.g., in-depth interviews with the questionnaire participants who opt into being considered for this second phase). If you promise anonymity, returned or recorded data gathering instruments (such as questionnaires, observational protocols and interview recordings) should not include information (e.g., their name, address, email address, phone number, and detailed description of person or organisation) that may directly, or by inference, identify an individual or organisation without their prior written consent. For some research contexts (e.g., workplaces, because of fears about what supervisors/ managers might learn) or research goals (e.g., if you will be asking sensitive questions, because of potential embarrassment, risk to social status or legal exposure for the participant), absolute anonymity may be the only way to encourage participation. If you promise confidentiality, you will need to retain identifying information in a specific and secure location accessible only by those who you have indicated will have access (i.e., not in the data file(s) used for analysis and reporting) and include any connection to identifying number or pseudonym. For example, which must be justified by the researcher, personal identifying information may need to be collected in order to follow up on responses made by individual participants. In all cases, your confidentiality/anonymity intentions should be clearly spelled out in the participant information sheet and agreed to in advance by the return of the questionnaire or signed consent form. In general, for postgraduate researchers, the only people who should have access to participant identities if confidentiality is promised are yourself and your supervisors. There may be instances, if you are doing a professional doctorate, where someone in the profession or workplace argues that they need to have access to participant identities as well and you will need to carefully consider the implications of any such request before you make the confidentiality promise. It is a severe breach of standards of research conduct and ethical guidelines if you fail to make good on your promise of confidentiality and/or anonymity.

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

701

Suggested strategies for ensuring maintenance of confidentiality or anonymity are: • Anonymity should be preserved—unless it is an overt part of your research configuration and agreed to by participants, you will not know the participant’s identity. For example, responses should be returned anonymously and there should be no coding or other means of identifying participants from those responses. Your records and reports should preserve the anonymity of participants and, if that can be done, promise absolute anonymity. If you can only safeguard participant identities when you write up any research results (including using any quoted material from participants) in research outcomes, then you can only promise partial anonymity. • Use clear protocols—ensure that clear research protocols are in place to ensure confidentiality/anonymity and protection of personal information. If you promise absolute anonymity, then all connections between participant identity and data records must be destroyed or never established in the first place. If confidentiality and partial anonymity are promised, then use a separate file to store connections between participant identity and anonymising labels that will be used. • Control of storage and access to data—it is necessary for researchers to ensure that arrangements for the storage of data are at least as secure as the source from which the data was obtained. Where confidentiality is promised, access to data should typically be restricted to postgraduates and their supervisors, and participant identifiers should be removed as soon as practicable. If you will opt to or are required to make your data publicly available, after a certain period of time, then any potentially identifying information must be removed before the database is publicly released. • Avoid personal identification—when writing up the data and particularly when publishing results, care should be taken that, unless it is specifically part of the research configuration, no participant, neither an individual nor organisation, can be directly or even by inference identified. This is the meaning of partial anonymity. • Retain & protect your data—research data and consent forms must be retained for a period of five years, unless your institution imposes different requirements. However, it may be that the conditions of informed consent to participate in a research procedure will require that the data generated be destroyed once the relevant research information has been coded and stored in a data file. If you use electronic files to store participant information, ensure these are password protected; if you use paper-based records, store them in a lockable filing cabinet. Avoid the embarrassment that one researcher experienced when it was found that the computer they had discarded still contained all their research files and participant information because while they had copied all of the data it had not been electronically wiped from the hard drive.

702

15

How Do I Handle Academic Integrity Issues?

15.2.6 Libel and Slander Often referred to as defamation, libel and slander is the act of harming someone’s reputation by making a false statement to a third party. Libel is by means of writing, print or other more permanent form, while slander is by means of spoken word or gesture (Humphrey, 2017). While this is not a common ethical issue to be considered by doctoral students, it is worth considering when discussing the writing up of your research. Two suggested strategies are: • Substantiate—when you make arguments, written or oral, contrary to someone else’s point of view, position or conclusion, ensure you have sufficient and concrete evidence to back up your own conclusions and claims. • Maintain a professional attitude—when you make critical comments, whether written or oral, on the research outcomes produced by others, ensure you maintain a professional attitude and target your criticisms to substantive content rather to personal characteristics, attributes or motivations.

15.2.7 Conflicts of Interest A conflict of interest can occur when there are any significant relationships that exist between you, as researcher, and other people and groups that intersect your life (e.g., research participants, workgroup, organisation, friends, family, colleagues, supervisor, examiner or sponsor) that could be perceived to create a risk of leverage, advantage or disadvantage either to the researcher or to those others. Where the relationship involves power being exercised, or potentially exercised, by one person over another (e.g., employee-employer, principal-teacher, doctor-nurse, supervisor-postgraduate, examiner-postgraduate), these considerations are particularly important. Issues of gender, ethnic group, indigeneity, religion and age may also be relevant here. Take appropriate steps to avoid conflicts of interest or the appearance of conflicts, particularly in regard to: • Roles—refrain from assuming roles in which your interests or relationships could reasonably be expected to impair your objectivity, competence or effectiveness, or expose persons or organisations with whom the relationship exists to harm or exploitation. • Disclosure—disclose relevant information and personal or professional relationships that may have the appearance of potential for conflict of interest. • Decision-making—carefully assess the potential for bias when making decisions affecting those with whom you may have strong conflicts or disagreements (Thody, 2006).

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

703

Strategies that you can use are: • Declaration—make very clear in your ethics approval process, and in your research outcome, any potential conflicts of interest and what possible impacts they might have. In your research outcome, this information could be covered in your methodology story as part of positioning yourself as a researcher or, if it becomes a significant and unforeseen issue, it may be mentioned in your limitations section. If you have the right to discuss a pool of possible examiners for your research outcome, make sure you identify any potential conflicts you might have with specific individuals and remove them from the pool. • Minimisation—consideration needs to be given and articulated, particularly in ethics approval documentation, as to aspects of your research configuration and data gathering strategies that will be utilised to minimise the effect or potential effects of significant relationships. It is important that you consider the issues and enact processes to limit their potential impact on your research. For example, if you are working with focus groups, allocations to groups should be made to ensure that a superior and a subordinate are not in the same group if it may materially affect the discussion and research outcomes.

15.2.8 Influential Funding One research integrity issue that is commonly mentioned in the popular press is where research funding has been received to support the research. This funding may then accompanied by the expectation that the funder can influence the research process and/or outcomes. This is a form of conflict of interest and occurs when a researcher is being funded, either through cash or in-kind support, by any external person or organisation, host institution, federal or state government department and/or a sponsoring agency. It is especially problematic if that agency is funding your research into products, services or environmental issues where they have a vested interest in that research showing specific findings (e.g., drugs, tobacco, agricultural pesticides/herbicides, climate change). All funding sources should therefore be fully disclosed and potential conflicts of interest or pressures to move or report the research in particular way carefully considered. Strategies you could use are: • Ensure transparency—make it clear at all stages of your research, i.e., framing, contextualisation and configuration stages, ethics approval, data gathering, write-up and dissemination, that your research is being funded by a third party. In this way, your findings will be considered in the light of the funding and expectations of the funding body. • Avoid influence—convincingness is the key to research quality and, despite the funding, you must remain independent and minimise the involvement or influence of the funding source. This may mean you (and your supervisors) need to devise strategies for dealing with any pressure to conduct your research in certain ways and/or present your findings in a specific light.

704

15

How Do I Handle Academic Integrity Issues?

• Check with the Research Office—if you sign any contracts for funding, it may restrict you in regard to ownership of any intellectual property generated during your research journey. You need to be very careful with such contracts. Most universities now have commercialisation offices, often contained within the Research Office or associated with it, so you should seek out advice. Once you get over the euphoria of getting funding, check the contract and read any fine print as there could be constraints.

15.2.9 Inaccurate Analysis and Reporting Once your research proposal has been approved, ethics approval granted, information sheets distributed, consent forms signed, and data gathered, you might be getting quite nicely lulled into a feeling that nothing can go wrong from here on in. However, the stakes are high, and students are bent on generating a convincing research outcome that demonstrates a substantial contribution. With that desire can also be the temptation to cut corners, and academic integrity issues can most certainly crop up in relation to the accuracy of your data analysis activities and reporting of research findings. Strategies you can employ are: • Avoid fabrication—do not fabricate data or falsify results in publications or presentations. • Full, open and transparent disclosure and reporting—in presenting work, report your key research decisions, methodological choices and findings fully. If you find some surprises, deal with these openly and critically (remember the handling of unexpected outcomes meta-criterion!). Also, if you needed to make certain assumptions in order to carry out your intended analysis or interpretation activities, detail these fully. As well, report on your research limitations fully and critically (remember the acknowledge of limitations meta-criterion). Rely heavily on your research journal to recall all necessary details. • Avoid hiding data omissions or data management activities—do not omit data that may be relevant within the context of the research question(s). If you must remove some data from consideration or alter them in any way (e.g., statistical outliers, invalid or ambiguous questionnaire responses, data transformations), be very clear with your reasoning, anchoring your arguments back to prior literature where possible. • Acknowledge the source of data sets—when analysing secondary data gathered by others, explicitly acknowledge the contribution of the initial researcher and/ or owner/maintainer of the database. This is especially important when you employ the secondary/archival data gathering strategy or a meta-analysis/ meta-synthesis strategy. • Enable verification—permit open assessment and verification by other responsible researchers, with appropriate safeguards, where applicable, to protect the confidentiality/anonymity of research participants.

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

705

• Admit errors—if significant unintended errors in the publication or presentation of your data have been made, take appropriate steps to correct such errors in the form of a correction, a retraction, a published Erratum or other public statement.

15.2.10

Not Keeping Appropriate Records

The retention of accurately recorded and retrievable research data is of considerable importance for the progress of scientific integrity. These records should include sufficient detail to permit examination for the purpose of replicating your research, responding to questions that may result from unintentional error or misinterpretation, establishing the authenticity of your records, and confirming the content and character of your conclusions (University of Pittsburg Guidelines on Research Data Management, http://www.provost.pitt.edu/documents/RDM_Guidelines.pdf). Strategies you can use are: • Maintain and continuously update your research journal—your journal is the record of your journey and the decisions you made along the way. Maintain it and keep it up-to-date. It will be an invaluable backstory for your research and the data gathered and used. • Determine responsibilities—early in your research, determine who will retain stewardship of the data you gather (recording, retaining and storage of research data). This is usually a fairly simple process when it is your own data, however, if you are part of a research team and move to another university, problems could occur. • Keep back-ups—for security and protection, always keep up-to-date back-ups of your data set. Nowadays, both electronic and hard-copy back-ups are good practices to enact. Store these in a safe place away from your normal workplace. • Be open and honest—other researchers may wish to use your data set for further data mining or to check and verify findings. It is important that you remain open to continued scrutiny of your material while also protecting the confidentiality/ anonymity of your participants.

15.2.11

Plagiarism

The interpretation of what constitutes plagiarism is sometimes subject to variation. However, the common theme is that plagiarism involves using someone else’s ideas, words or materials, directly or indirectly, without giving them credit. Any time you use ideas, words or material that relate to a specific source, you must attribute them to that source. When quoting or closely paraphrasing (restating), a full citation, commonly referred to as ‘attribution’, is required. It is recommended

706

15

How Do I Handle Academic Integrity Issues?

that you should explicitly cite others’ works and ideas, even if the work or ideas are not quoted verbatim or paraphrased. This also applies whether the previous work was published, unpublished or electronically available (Academy of Management, Code of Ethics, Point 4.2.1.1, http://aom.org/About-AOM/AOM-Code-of-Ethics. aspx#preamble). Plagiarism also applies to your own work and is referred to as self-plagiarism. Where you have published data or findings that overlap with work previously published elsewhere, you will need to cite those publications (Academy of Management, Code of Ethics, Point 4.2.1.2, http://aom.org/About-AOM/AOMCode-of-Ethics.aspx#preamble). In most universities, plagiarism is also deemed to be misconduct and is covered by a student misconduct policy. Research is about creating new knowledge, and plagiarised material is not new and relying on it as a publication strategy is essentially theft of another person’s intellectual property. Strategies you could use are: • Start off correctly—in order to minimise problems associated with referencing, it is appropriate to start early and start correctly, by entering your words into EndNote or other bibliographic software system, or your permanent record filing system (with the potential of yielding up to 30 pages of references) in your final research outcome. You will first need to determine which system, e.g., Author-Date, Chicago, Harvard or American Psychological Association (APA), your university (and, in some cases, department) favours for research outcome preparation. • Ensure accuracy—it is anticipated postgraduates will accurately identify, credit and reference the author(s) of any data or material they have used. This includes not only other people’s work, but also your own. If you quote someone directly, adapt or paraphrase what they say, you must include the page number(s) in the reference source from which you took the material, if that source is published in hardcopy. For electronic sources, page numbers may not be relevant, but you must provide the correct URL web address. • Use academic conventions—direct quotes are to be contained within quotation marks, unless they involve multiple contiguous sentences from the source (so-called block quotes), in which case, they are typically identified by indenting the block as a separate paragraph, rather than by quotation marks.

15.2.12

Attending to Correct Referencing

A good way to gauge attention to detail is by checking references. Inconsistencies between the body of work and the bibliography, missing references and poorly formatted references, are all indications of sloppiness that have a potential impact on the perceived quality of the research outcome itself. There are many reasons for ensuring appropriate referencing: • Defensive—to show that your research is justified by and related to other work in the same or similar fields;

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

707

• Archival—to record the sources for your own future research; • Altruistic—to provide the readers who want to carry out similar research with accurate and effective directions to the sources you used (effectively, your references can be a source for another researcher’s back-referencing strategy); • Promotional—citations are increasingly impacting on research assessment exercises; • Sycophantic—to win friends and flatter mentors, supervisors, examiners, and reviews when you cite their work; and/or • Protective—to avoid plagiarism by giving credit to the authorities whose work you have used (Petre & Rugg, 2010).

15.2.13

Copyright

Copyright will be more of a concern to you if you are lecturing and preparing material for students than if you are copying single items for your own study. However, you do need to be aware of copyright requirements, particularly if you are downloading full text material from a database into your bibliographic record system. Most developed countries will have a Copyright Act which will cover materials stored for educational purposes. The general guideline is that copies of a page or pages from a website may be stored by educational establishments provided that: • the material is displayed under a separate frame or identifier, identifies the author and source of the work and states the name of the educational establishment and the date on which the work was stored; • the material is restricted to use by the authenticated users, defined as participants in the course of instruction or institutional research for which the material is stored, who can access the stored material only through a verification process; and • the stored material is deleted within a reasonable time after the material has become no longer relevant for the purpose for which it was stored. All material you copy must be acknowledged, i.e., you must reference correctly all material to show where it has been taken from. Some websites will allow copying/printing of full articles or documents for educational purposes but there has to be a clear statement to this effect on their website. You may also write to the owner of the web site seeking permission to make multiple copies. If this is not available the guideline is that for articles or documents on the web you may copy 3% of a web page (i.e., what you can scroll down), or three pages, whichever is greater, as long as it is not the whole work. In many cases, your university will have subscriptions to publicly maintained banks of reference materials (e.g., Pro Quest, Emerald, APA PsychNet, EconLit, ERIC, JSTOR, Health & Society Database, Sage Journals). With a subscription generally comes permission for postgraduates,

708

15

How Do I Handle Academic Integrity Issues?

registered with the university, to download/print articles from sources covered by that database. Your library will be able to give you advice on copyright restrictions as they would be adhering to all copyright requirements. For understanding copyright restrictions in Australia, see Intellectual Property Australia Government, https:// www.ipaustralia.gov.au/about-us/about-this-site/copyright, and for publisher’s copyright requirements in the UK, see UK Copyright Licensing Agency, www.cla. co.uk. For a broader perspective see Copyright (New Technologies) Amendment Act (2008) World Intellectual Property Organization, http://www.wipo.int/edocs/ lexdocs/laws/en/nz/nz123en.pdf. Strategies you can use are: • Limit what you take—for hard copies, the general rule is that as long as you copy no more than 10% of a text, or one copy of an article from a journal, you are probably not in violation of copyright. For material that is obtained from the web, you may copy 3% of a web page (i.e., what you can scroll down) or three pages, whichever is greater. • Don’t forward pdfs—for material that you obtain from the web, the general rule is that you can download a pdf file, but you cannot email it to someone else. • Get clearance if needed—if you want to produce a number of copies for distribution in class, or for publishing in a textbook, copyright clearance will need to be obtained.

15.2.14

Use of a Professional Editor

By the time you near the end of your research outcome writing you will be getting to the point of being all too familiar with your work. English might also be your second language. Whatever your circumstances, a second set of eyes looking over your writing is often valuable to seek. This can come in the form of a professional editor who is paid to review and edit your research outcome. You need to be clear about the extent and nature of the help they offer in the editing of your research outcome. It is advisable to discuss this with your principal supervisor and to provide the editor with a copy of any guidelines that your organisation may use (for more information on editing services, see Institute of Professional Editors, http://ipededitors.org/). Strategies you could employ: • Look it up—obtain a copy of any editing practice policy or guidelines that your university may use, read it, and provide a copy to your copy editor. • Acknowledgement—when a research outcome has had the benefit of professional editorial advice, the name of the editor and a brief description of the service rendered should be printed as part of the list of acknowledgements. If the professional editor’s current or former area of academic specialisation is similar to that of the candidate, this also needs to be acknowledged.

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

15.2.15

709

Misappropriation of the Outcomes of Research

A key research integrity issue relates to the ownership of data and the outcomes of your research and you may also need to protect yourself from having your supervisor or others take credit for your work. Consider the following experience of a postgraduate student: Ann Green (not her real name) spent 7 years on her doctoral project at an East Coast university. In her mind, she had made a major breakthrough, the kind of discovery that could establish a career. When the results were finally published, she was missing from the list of authors. Her adviser–who, according to Green, had very little input in the research–had mysteriously risen to first author. Green’s only appearance came in the acknowledgment section, where she was thanked for her “generous advice”. Few people have ever worked so hard for a compliment. Over a decade later, she is still burning. “It was totally outrageous,” she says. “It wrecked my career. I went out into the world with no manuscripts behind me.” In the meantime, she says, her adviser has been cited over and over for her research. According to Green, he has also used her data to secure $5 million in grants (Woolston, 2002; https://www.sciencemag.org/careers/2002/05/when-mentor-becomes-thief).

It is an unfortunate reality that in some disciplines, the power relationship between senior (e.g., supervisors, mentors) and junior researchers (e.g., postgraduates, early career academics) can mean that the efforts of a junior researcher is claimed by a senior researcher, and this dynamic can be compounded if the research is funded by a grant to the senior researcher. Do not be fooled—this is credit theft and is an unethical research practice. You need to ensure that you get the credit you deserve for the work that you do and the findings you produce. Strategies to be used are: • Get agreement—you and your supervisor(s) should agree on intellectual property and publication authorship very early in your candidature. Some universities will offer a specific form on which to record such agreements. This is especially important if you are undertaking a professional doctorate where an innovation or process you design and conduct research on may be one outcome of your research. In such cases, the interests of various stakeholders, including the university you are enrolled in, the workplace or organisation within which you are situating your doctoral research and yourself, must be balanced. • Keep records—retain printouts of your computer files as well as electronic back-ups in order to be able to prove the extent of your contributions to the research. • Bring misappropriation instances to the attention of your university—if your work has been seriously misappropriated, bring the matter to the attention of your university. This may require a written submission to your university’s research ethics committee. Interestingly, as I write this, my university is currently dealing with such an issue in relation to two research colleagues. The ethics approval for their research has been withdrawn while the circumstances surrounding the misappropriation of research data is investigated. So effectively, their research has ground to a halt.

710

15.2.16

15

How Do I Handle Academic Integrity Issues?

Inappropriate Authorship Attribution

Having just mentioned misappropriation of research data, a related area concerns publication outcomes and accuracy around authorship attribution. One of the more contentious issues that a doctoral student can experience happens when papers are published in collaboration with their supervisor or supervisors. There is often a perceived intellectual and power imbalance and a student can, therefore, be reluctant to assert themselves. However, the reality is that while it is acknowledged that there is a power difference between yourself and your supervisor(s), there is no reason for you not to be assertive in protecting your intellectual property. There are well-established protocols regarding the use of postgraduate research work, however, sometimes the need to meet publication targets or the personal ambitions of a supervisor can result in behaviour that is less than ideal and is exploitative of the student. While your supervisor may mean well, they are under significant pressure in their own careers to achieve which may, unfortunately, prompt behaviour which is less than professional. It is for this reason that you will need to speak confidently to your supervisors or potential collaborators regarding your expectations for the use of the results of your study, the material you have written up in preparation for your research outcome, and any subsequent publications (as well as ownership of intellectual property). You may wish to provide your expectations in written form in order to obtain agreement from potential collaborators. From personal experience there is nothing more galling than having your work reproduced by a member of staff without acknowledgement. The feeling of being ‘ripped off’ is most unpleasant and one you wouldn’t wish to experience. Similarly, it is worth resolving difficulties rather than suffering the embarrassment of co-authors indicating that they did the lion’s share of work on the paper when, in fact, that was not the case. If you are naturally timid in these situations, you need to overcome this to secure the intellectual rights that are, in fact, yours. With regard to authorship credit, everyone who has been involved in the preparation of published material should be duly acknowledged based on their professional contributions. All persons designated as authors should qualify for authorship, and all those who qualify should be listed. Each author should have participated sufficiently in the work to take public responsibility for appropriate portions of the content. Authorship credit should be based only on: • substantive contribution to conception and design, acquisition of data, and/or analysis and interpretation of data; • substantive contribution to drafting the article or revising it critically for important intellectual content; and • final approval of the version to be published. All three of the above conditions must be met. Acquisition of funding, the gathering of data, discipline ‘norms’, or general supervision of the research group,

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

711

by themselves, do not justify authorship. Authors should provide a description of what each contributed. All others who contributed to the work who are not authors should be named in the acknowledgement, and what they did should be described. Increasingly, authorship of multi-centre trials is attributed to a group. All members of the group who are named as authors should fully meet the above criteria for authorship (see also Committee on Publication Ethics (COPE), https:// publicationethics.org/). Strategies you can use are: • Get agreement—if you or your supervisor think that some portion of your work might be publishable, agree on ground rules for publication as early as possible, and preferably before you have committed to a particular project. If you can’t agree at this stage, you won’t agree later. If it gets acrimonious, think about doing a different project with someone else (Cryer, 2006). Good things to agree on before you start: 1. 2. 3. 4. 5. 6. 7. 8.

where you will publish the paper; what you will do if it is rejected; who the lead author will be; what the order of authorship will be; what the next project might entail and who should be involved; who should write which sections; who should do the revision; and who should see the paper through the process (usually the lead author).

• Designate responsibility—one or more authors should take responsibility for the integrity of the work as a whole from its inception to the published article. • Your work, your name—when publishing with your supervisor and/or others, the typical protocol is that, as a student, you should anticipate being listed as the first author where the publication is substantially derived from your postgraduate research. There are disciplines where the norm is for the supervisor’s name to be listed first, but it is always useful to question this practice, as it runs counter to the logic that the person who did the most work on the project should be listed first. Of course, you can choose whether to follow this expectation, but that choice should be yours, not your supervisor’s. [As an example, Ray once supervised a PhD student, Jim, who ended up publishing 6 articles from his PhD, 5 of which Jim was first author on. For the 6th article, Jim said Ray should go as first author since he had suggested the unique analytical approach taken in a specific aspect of Jim’s PhD.] • Limit the number of authors—only list authors who have actually done the work or contributed to the paper in some way. • Avoid honorary authors—as a general rule, don’t use honorary authors in order to get reciprocal benefits on their papers or as a way of trading favours. This is considered to be unprofessional and is merely playing a numbers or tit-for-tat game. For example, at a European conference, Gael recently spoke to a data analyst in a tech company and he indicated that he used to work as an early

712

15

How Do I Handle Academic Integrity Issues?

career academic in a university. When she asked why he transitioned in his career to another job, he told me it related to a research integrity issue. Apparently, when he was completing his PhD, he and his supervisor were also finalising papers for publication from his PhD work. Just prior to submission of these papers, he was appalled to learn of the late addition of a new and unknown author. The explanation given by his supervisor was that the author was an old colleague and that his supervisor owed him a favour. Understandably, the young student was not impressed and withdrew from a future in academia. Honorary authorship is basically freeloading, unless all parties can agree beforehand as to the reasoning. There may be exceptional circumstances where an honorary authorship may be warranted. For example, if your principal supervisor passed away early in your candidature, you may choose to include their name as an author to honour their early contribution to your journey as well as to honour their memory. For more information on how to handle authorship disputes, see Publication Ethics, https://publicationethics.org/resources/guidelines-new/how-handle-authorshipdisputesa-guide-new-researchers.

15.2.17

Inappropriate Publications

Where a number of papers have been prepared in relation to a research outcome, care needs to be taken that there is no extensive overlap, and that you can prove each paper is an original piece of work. However, academics have, in the past, been given mixed signals about what types of manuscripts might be considered as ‘new submissions’. It is acknowledged that it is probably not realistic to ask faculty to engage in significant long-term comprehensive research programs, and expect them not to get more than one publication for their efforts, particularly when journals themselves won’t publish papers of more than 30–35 pages (Petre & Rugg, 2010). While it is common to rework and improve papers that have been previously published in conference proceedings, considerable care needs to be taken to ensure that each manuscript is ‘new’. In an effort to avoid confusion and provide clarity, most journals follow common protocols: • The journal will not consider for publication any manuscript that has been simultaneously submitted to, has recently been submitted to or will be submitted to another journal. • If a manuscript is not the first manuscript submitted from the research project, the authors are advised to disclose this information in a cover letter to the editor. • Make sure ‘new’ manuscripts are “materially different” from previous manuscripts published or under review. ‘Materially different’ requires subjective judgment, however, if you are using the same sample, or using the same variables, think carefully about whether the manuscript is ‘new’.

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

713

• Authors are free to submit descriptive or prescriptive articles based on research previously published as long as those articles fully credit the earlier publication and conform to other journal policies about multiple publications (in this way, you may be able to present the analytical data in one journal and a more practitioner-oriented paper in another). • When participating in a blind peer review process, authors are discouraged from including information within a manuscript that might inadvertently disclose their identity. Even the well-intentioned should not include references to articles in press in subsequent papers submitted for publication. Often postgraduates are unsure of when they should publish on their own and when they should publish with their supervisor. Feldman (2003) have recommended circumstances where sole authorship and joint authorship might occur. Sole authorship is likely where you are exploring unknown territory and where your supervisor is providing you with general advice on the path of exploration, without ever having considered the territory themselves. Joint authorship, which is the more common approach, is likely to result where your supervisor has equipped you and pointed you directly into territory and where you effectively provide the last piece to complete a much larger puzzle (Marshall & Green, 2007). Some institutions have a policy that, should a postgraduate not publish from their thesis, dissertation or portfolio for a period of two years after graduation, the material contained therein becomes available to the supervisor for them to publish from. The reasoning behind this policy is that often postgraduates are so “over” their PhD at the conclusion, they don’t ever wish to look at the work again, and the material languishes unpublished. As academia is about the dissemination of knowledge, the general sentiment is that this material needs to get out there. If the postgraduate student is not going to or is unable to take the responsibility, the supervisor should. In such cases, the supervisor should, as a general courtesy, discuss such intentions with the student and authorship could form one point of such discussion. Strategies to be used are: • Avoid churning—churning is when you produce multiple papers in relation to your research in which there is significant overlap. While it is not uncommon to reiterate the methodology of the study, each paper should have substantive elements of uniqueness. The easiest way to ensure this is to focus each publication on a distinct aspect of the research (e.g., the literature review, where you might have conducted a meta-analysis; the emerging theoretical framework if substantive; and the empirical results). • Single submissions only—only ever submit the paper to one journal at a time. • Get agreement—in respect to publishing with or without your supervisor, try to develop some protocols. Broach the subject by asking the general question at the onset of your study, “What are your expectations regarding publishing together and for you to publish independently?” You should also discuss authorship issues if you have multiple supervisors.

714

15

How Do I Handle Academic Integrity Issues?

• Resolve any difficulties—if you begin to resent your supervisor, this will erode your relationship and will cause considerable difficulty with the main purpose of your work, namely preparation of your main research outcome, so make sure this doesn’t happen.

15.2.18

Manipulation of Intellectual Property Policy

While a university usually owns all the knowledge assets created by staff in the course of their employment, some universities do not claim ownership rights to the intellectual property that results from a postgraduate student’s research program, but this is worth checking with your university. Within the front materials of a thesis, dissertation or portfolio, however, acknowledgement should be made of the supervision received and any assistance that has been provided in through the duration of your research journey (limit this, as you don’t have to thank the cleaner). The tricky part is where there may be some commercial interest in the intellectual property resulting from your postgraduate research, especially in a professional doctorate where an innovation or intervention program is involved. If this looks to be the case, the intellectual property rights need to be discussed between you and the principal supervisor, and often in conjunction with a representative from the research office of the university. If external funding has been received, the funder could also be an additional party. If you are undertaking a professional doctorate, situated in a workplace, organisational or industrial context, then there will be stakeholders in that context who might wish to claim a portion of the intellectual property associated with your research. This issue may become particularly important to address if a professional from that context is undertaking a co-supervision role for your doctorate. Prior to such discussions, you are advised to be well-informed, having read the intellectual property policies relevant to your institution. For instance, check your university’s intellectual property policy to see who can claim ownership rights if intellectual property is generated from your research, such as a training program which has commercial applications, a technological or social process innovation or even a research instrument that has commercial value. You will need to check to see who has ownership of the material as you may find that your university has a commercial interest in that intellectual property. Do not solely take your supervisor’s advice on this point; check the policy out for yourself. These policies are often quite detailed, so you may wish to ask a relevant administrative officer for further clarification. Strategies you can use include: • Look up the relevant policies—the intellectual property and commercialisation policies will be most relevant and may be contained in two or more policies, so you may wish to check them with your institution and find out who manages

15.2

• •

•

•

What Sort of Academic Integrity Issues Should I Be Aware of?

715

them. An example of such a policy can be viewed at Knowledge Assets and Intellectual Property Policy. University of New England, https://policies.une. edu.au/view.current.php?id=00117. Get advice—these policies can be a tad confusing, so it would be in your interests to seek guidance early on. Check the ownership of your thesis, dissertation or portfolio—in all likelihood, you will have ownership of this research outcome; however, you may wish to check. Similarly, if your research has been funded by either a private or public organisation, you will need to look at the wording of the contract to ascertain who has ownership. Consider restrictions—if there is a degree of commercial sensitivity or industrial security contained within your research outcome, there is a possibility that you, your supervisor (or your sponsor) could have your outcome classified as ‘restricted’. In this circumstance, it will only be read by the supervisor(s) and the examiners and it will not available for wider circulation nor held in the library. There is usually a formal procedure to request this form of classification. Examiners will, upon completion of the examination, return the documentation to your university. A variation on restricted classification, called ‘embargoing’ is where there is a time limit imposed before the release of your documentation to the library, similar to a statute of limitations. Respect cultural sensitivities—if you have Indigenous participants, your research will have to be developed in consultation with relevant Indigenous communities and elders. This will likely have implications for intellectual property and knowledge ownership, from an Indigenous point-of-view, and that point-of-view may not be congruent with Western cultural expectations. Here, the Indigenous view must be respected and privileged over the Western view, which typically means that what is learned from your research and how it is learned should respect Indigenous culture and values and provide benefit back into the Indigenous community (principle of reciprocity). In short, knowledge that is gained or created through the research is to be shared in appropriate ways, not simply used to advance the researcher’s purposes.

15.2.19

Financial Misappropriation

Often researchers have access to research funding, either from grants within the organisation or from external grant bodies or sponsors. In these circumstances, funds have been provided for the purposes of undertaking research and appropriate stewardship is expected. Where a researcher has received, or handled, funds (possibly your supervisor has received the grant rather than you), you are to ensure appropriate usage and provide a true account for the usage of the money and/or property entrusted. It is important to demonstrate fiduciary responsibility. I was somewhat surprised when I had an experienced academic asked me if he could use

716

15

How Do I Handle Academic Integrity Issues?

his research “wash up money” (the residual money left over from a 2-year research project) to pay for his children’s school fees that were coming up for payment. I naturally declined the request. My surprise was that he didn’t see this as a research integrity issue. Strategies you could use are: • Avoid personal gain—unless specified under the research funding arrangements, do not appropriate research funds for your personal use. • Ensure complete honesty—adhere to all financial policies and honest research practices, both in the use of funds and in keeping accurate and current records. • Check on wash-up money—the usage of money that remains at the end of a project, often called “wash-up” money, is particularly sensitive. Check with your university as to how such funds may, or may not, be used. In some cases, you (or your supervisor) may be required to return wash-up funds to the funding body.

15.2.20

Ethical Principles

As every piece of research is unique, rather than having a set of rules and regulations, it is now common to have a general set of ethical principles to guide researchers. Ethical principles act as overarching guidelines rather than a specific set of do’s and don’ts. There are a number of principles governing research you, as researcher, are required to adhere to: • • • • • • •

protect participants’ rights to confidentiality, privacy and anonymity; exercise duty of care and minimisation of harm and risk; demonstrate cultural and social sensitivity; engage full disclosure and transparency; demonstrate respect for intellectual property ownership; avoid conflicts of interest; and honour the principles of equity and fair treatment.

15.2.21

Ethical Approval

Why do you have to get ethical approval? It is now readily acknowledged that in studies where human beings or animals are being researched, the research project will be subject to the requirements for ethical approval. Ethical approval usually deals with a narrower set of issues than those we have discussed above and tends to focus on the specific research rather than broader academic integrity issues affecting the researcher. Ethical approval is often considered to be a cumbersome hurdle by postgraduate students. However, it

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

717

is an integral part of the research process and should be approached in an appropriate and transparent manner. Essentially, ethical approval requires postgraduate students to consider the potential ethical ramifications of their research prior to commencing it. As most research involves interaction with others, there is the potential for harm. Clearly, this is to be avoided and the primary reason for ethics approval is to ensure that you have considered what might be potentially harmful to those associated with your research, and that you have taken adequate measures to minimise that harm. As we have mentioned, the risk of harm is as important as actual harm, and harm is defined by the adverse effects that it might have on an individual or group. In many cases, the harm is unintentional but, nevertheless, does occur and can come in many forms. As discussed, harm can extend to physical (discomfort, pain, fatigue), psychological (anxiety, embarrassment, offence), economic (financial loss) and social harm (exposure, inconvenience). Not only does ethics approval aim to ensure good research practices but there is also an element of risk management. Under the insurance policies held by their university, postgraduate researchers are usually only covered for personal liability providing their research has ethics approval, is carried out according to the approved protocols and has been appropriately supervised. Where ethics approval has not been granted, it would be difficult to indemnify a student, a participant who is harmed in such a situation could seek legal redress, and the reputation of the university could take a hit. What does a research ethics committee actually do? The aim of a research ethics committee is to ensure that research, conducted under the auspices of the university, complies with ethical standards and international best practice, as well as ensuring research practices do not infringe laws, regulations or treaties. Research ethics committees meet regularly; however, they will normally only meet during term time and usually on a monthly or six-weekly timeframe. You will need to incorporate an approval submission deadline into your planning schedule and, once again, work backwards to determine when your ethics application documentation needs to be prepared. The usual practice is for no research to be implemented until ethics approval has been granted, so be prepared for delays and don’t let the ethics approval hold you up. Get onto other aspects of your research, such as further literature searches, drafting chapters and so on. Retrospective approval will generally not be given for any research already commenced. Where a committee has determined that your research is not being conducted according to the protocols approved, they can withdraw approval for the research. Failure to comply with approval procedure can result in serious disciplinary action. Often what may happen is that your research will be approved with conditions that you have to meet. For example, one condition with human participants that may be imposed is that a snowball sampling scheme may not be used (see Chap. 19). If you planned to use such a scheme, you may be required to modify it before approval will be granted. Another condition that may be imposed is to provide a reference to accessible resources/support (e.g., counsellors) in the Participant Information Sheet, should any inadvertent harm or psychological

718

15

How Do I Handle Academic Integrity Issues?

distress be experienced by participants. This may emerge from the committee’s reading of your application. If you have not anticipated the need, but the committee sees that need, they may decide to impose such a requirement. What is the usual process for gaining ethical approval? Each university will differ in its regulations and forms but, essentially, the process entails the following: • Complete and gain authorisation of all documentation in both hard and soft copy. There are standard forms and documentation that will be required, and they will typically be aligned with a code of ethics that the university subscribes to (e.g., in Australia, it is the National Statement on Ethical Conduct in Human Research 2007 (Updated 2018) (NHMRC, ARC, & UA, 2018) that all universities adhere to). The forms will require you and your supervisors’ signatures guaranteeing the accuracy of the information that is provided. • You submit all completed forms plus attachments (i.e., research instruments, information and consent forms) electronically to the secretary of the research ethics committee with a signed declaration page, a cover page for the application form and any additional information. Keep the inclusion of additional information to the minimum required as committee members are busy people and already have a lot to read. Reading of your application cannot commence until all documentation is received, so be sure to get it right the first time or you could be significantly delayed. • Your material may be distributed to a principal reader who may discuss the application with the applicant and/or supervisor, as appropriate. Depending on the processes of your university’s committee, the reader, or possibly two readers, will review and decide on your material and bring their decision to the committee for further deliberation. In some universities, especially smaller institutions, the entire ethics committee will read all applications. In some institutions, there may be an expedited pathway for ethics approval, where the project is non-controversial, and the chair of the committee may be empowered to approve straightforward applications outside of committee meetings. • Most committees make decisions by consensus or by a two-thirds majority only. The chairperson will ensure that members of the committee are free to participate fully in discussion and debate. Be aware that the committee will likely have some academic members who are not social or behavioural researchers. In our experience, such committees will have the most trouble dealing with interpretivist/constructivist approaches or critical approaches, so if your project adopts one of those patterns of guiding assumptions, your application will need to be very clear, with a minimum or paradigm-loaded jargon and a bit of attention devoted to defending your approach. • Committees can, and do, consult with external parties where they may seek expert advice (particularly on cultural issues). However, they are bound by principles of confidentiality and can only disclose the issue, not the identity of the researcher.

15.2

What Sort of Academic Integrity Issues Should I Be Aware of?

719

• Approval may or may not be granted. It is not uncommon for documentation to require resubmission, changes or additional information at which time an indication of where the application is lacking will be provided. You may then have to wait until the next meeting which could entail a delay of several weeks. What if you are an academic staff member at one university but are doing your PhD at another university? If you are an academic staff member at one university but a postgraduate student at another tertiary education institution in the same country, and ethics approval has been granted from the university where you are enrolled, a second application is not required. However, it would be a good idea to lodge a copy of the approval with your home university’s research ethics committee to ensure that you are covered by indemnity insurance in your own organisation, despite approved research liability normally resting with the institution approving the research. What if you are undertaking a professional doctorate or applied research within another organisation? If you are undertaking a professional doctorate or applied research within another organisation, you may have to go through another ethics approval process, undertaken by the industry or organisation who is a stakeholder in your research. This will be in addition to the ethics approval process you must go through at your university. For example, if you are doing research in a hospital or school, that hospital or school may wish to ethically review your project. If you are doing research in your own workplace, your employing organisation may wish to ethically review your project, before you are permitted to commence. What if my study changes significantly following ethics approval? Any significant departure from your project, as approved, particularly around issues of consent, confidentiality, sensitivity, sampling changes and/or potential harm to participants, must be notified to your research ethics committee and approval sought for the project variation before proceeding with the changes. What if my research has no research instruments and is, by its nature, evolving? It is acknowledged that interpretivist/constructivist and other non-positivist guiding assumptions limit the degree to which methods and tools can be explicit at the point of application (for example, in the Action research or Developmental Evaluation research frames where interventions/innovations are developed during the research process, in research that uses an evolutionary configuration or in research that uses unstructured or semi-structured qualitative interviews). In such cases, protocols describing the relationship of the participants to the applicants, and letters of support from the participants or their representatives, are usually required. As well, you will need to describe your data gathering strategies as clearly as you can but may need to spell out the reasons why you cannot be more specific or structured. If you are using a semi-structured interview methodology, a good practice is to provide a topical landscape (recall Sect. 14.1.1.3) of the issues you wish to cover in the interview (some students prefer to submit a mind map of the semi-structured interview landscape). For some evolutionary configurations, you may need to flag that your ethics application will need to be considered in stages,

720

15

How Do I Handle Academic Integrity Issues?

with the first stage being what the committee approves immediately, and that you will return an updated application, once you have reached the next stage of your research and have decided what you will do. What if I have a complaint about the committee or its processes? Complaints regarding an application and the process used by the research ethics committee in reaching a decision are always investigated by the research ethics committee. It is usual for the complaint to be made in writing. If you think that you have a conflict of interest or relational problem (e.g., a concern about potential bias) with one of the committee members, you are within your rights to contact the chair of the committee and request that that person not be involved in considering your application. What type of research is usually exempt from requiring ethics approval? The following types of research usually do not require specific ethics approval: • Research that does not involve human participants or animal subjects and is not foreseen to risk adversely affecting either humans or animals. • Research involving existing, publicly-available documents or data (e.g., analysis of archival records or secondary databases). • Evaluations conducted within the university as part of its quality assurance procedures. • Preliminary interaction or discussion where the exact research aims have not yet been formulated. • Research in which a single investigator is the subject of his or her own research (e.g., autoethnography), and where no hazardous outcomes are foreseen. • One-off interviews with public figures, for example, politicians, prominent authors. • When seeking a professional or authoritative opinion, except where this is part of a study of the profession or area of expertise.

15.3

Conclusion

Public trust in research can be dealt a severe blow when evidence of research misconduct is made public. For example, the renowned Norwegian researcher, John Sudbo, fabricated and falsified data in articles on oral cancer published in The Lancet and the New Zealand Journal of Medicine. Another case of fraudulent research involved a Korean stem-cell researcher, Wosuk Hwang, who published in Science and Nature (Van Der Weyden, 2006). Considerable attention and resources have, therefore, been focused on the ethics of research involving human participants (Simmerling, Schwegler, Sieber, & Lindgren, 2007). There are numerous texts to assist researchers (e.g., Altman & Hernon, 1997; Elliott, Fischer, Grinnell, & Zigmond, 2015; Faria, 2018; Homan, 1991; Kimmel, 2007; Macfarlance, 2009; Nicholas-Casebolt, 2012; Sieber & Tolich, 2013; Shamoo & Resnik, 2014; Seadle, 2017) and specifically in relation to detecting and

15.3

Conclusion

721

investigating research misconduct (Ben-Yehuda & Oliver-Lumerman, 2017), the management of data (Sibinga, 2018), online research (Woodfield, 2018), research ethics in the social sciences (Israel, 2015; Iphofen, 2017; Emmerich, 2018), in health (Steneck, 2007) and relating to publications (Jerjes, Hamoudi, & Hopper, 2018). The definition of what constitutes research misconduct casts a wide net. It has been noted that, in general, the Nordic countries and most countries of central and western Europe have national guidelines to address research misconduct and promote research integrity (Godecharle, Nemery, & Dierickx, 2013). Currently, the US, Canada, Denmark, Finland and Norway have overarching bodies to oversee and evaluate instances of research misconduct. In Australasia, the Australian National Health and Medical Research Council, the Australian Research Council, Universities Australia, and the New Zealand Health Research Council have all released codes for responsible conduct of research covering a range of potential ethical problems that a researcher may encounter (Van Der Weyden, 2006; for example, the Australian Code for the Responsible Conduct of Research is available at https://nhmrc.gov.au/about-us/publications/australian-code-responsible-conductresearch-2018). Essentially, researchers must ensure that their research configuration and associated sampling and data gathering strategies are adequate to allow ethically robust research to be carried out and to clearly demonstrate academic integrity. When considering ethical issues in relation to your research, there are two dimensions. The first is what the ethical issue might be, and the second is what actions can be taken to alleviate or avoid the potential ethical issue. For example, to ensure that participant coercion does not occur, the pre-emptive action is to gain prior informed consent. In identifying ethical issues, the principle of duty of care should be at the forefront of your deliberations and there should be extensive consideration of the potential for harm and risk, not only to your research participants but for everyone involved with your study, including yourself and other stakeholders. When undertaking actions to reduce or eliminate the impact of the ethical issue, a change in procedure, even if it causes you inconvenience or forces you to change your research plan, should be implemented if you hope to gain and retain ethics approval. Your university will have its own documentation for dissemination to researchers. This documentation usually outlines ethical issues as well as ethical approval processes. In a sense, ethical behaviour in research is not complicated. You should avoid actions or questions that can be viewed as threats to your participants’ health, values or dignity (Brown, 2006). If you are concerned about this area, look at cases that have previously been handled by the research ethics committee in your own university. They are not usually readily available but are archived and should be accessible for you. They will give you an indication of what has constituted unethical practices on the part of academics and postgraduate students in the department. Additional documentation may be needed to cover intellectual property and commercialisation. Acquire a copy of the intellectual property policies in existence in the institution where you are studying. Do not hesitate to ask the administrators for a copy of the policies. You may also keep an eye out for other guidelines for

722

15

How Do I Handle Academic Integrity Issues?

publishing with postgraduate students, as these may also be equally useful in guiding you in your interactions with co-authors. One further area worth mentioning is intervention and advocacy. What should you do if you hear about harmful, illegal or wrongful behaviour during research? A student’s research may stumble upon an unexpected outcome or reveal wrong doing. For example, an accounting and finance PhD student who was investigating an organisation, detected a significant fraud. While not an anticipated outcome, sensitivity in handling the situation and its potential impact on their research clearly needed to be considered and effectively managed. The advice here is not to go it alone. Bring any issues like this up with your supervisor(s). If the issue involves your supervisor, or your supervisor does not appear to be as concerned as you are, bring it to the attention of someone more senior, such as the Postgraduate Director, the Dean, or the chair of the research ethics committee. Academic integrity works both ways.

15.4

Key Recommendations

• Academic integrity relates to the entire range of research activities that you will be involved with, not just data gathering. • Ethics processes will vary somewhat from university to university, however, you have a responsibility to apply for and receive research ethics approval, where required, before you commence your research. • Given the irregularity with which research ethics committees meet, forward planning will be required so that a delay will not slow you down. If you wish to avoid hold-ups, become fully acquainted with the ethics policies, guidelines, processes and procedures relevant to your program and your university and with the timetable for ethics committee meetings. • Where there are commercial sensitivities, you may wish to look at having restricted access applied to your research outcome, that is, it will only be available to the supervisor and examiners, with a confidentiality agreement attached. The restriction may be for a specified time period or indefinite. In these circumstances, the examiners will be required to return their copy of your thesis, dissertation or portfolio after examination and will be requested not to copy or circulate it, nor discuss it with others. • If your research involves Indigenous participants, make sure you understand and implement ethical guidelines specific for those participants. These guidelines will speak to cultural sensitivities, respect, reciprocity and other values and will have implications for intellectual property and knowledge ownership in general. • Participants should have the option of participating or not in your data gathering process and should be informed from the outset that they can withdraw from participating at any moment they wish and can request you to destroy any data you have already gathered from their participation. • If you have promised anonymity to your participants, they should not be identifiable from the raw or published data, either directly or by inference.

15.4

•

•

• • • • •

Key Recommendations

723

Remember that confidentiality does not necessarily imply anonymity: Confidential means someone knows the identities and personal details of participants; anonymity means no one knows the identities and personal details of participants. Consider carefully what, if any, inducements you might be offering to potential research participants, and whether such inducements will influence the voluntary nature of participants’ involvement. Remember that an incentive to participate typically maintains a voluntary intent whereas as an inducement to participate could be perceived, by participants, as coercive. Research files may contain confidential information and it is essential that you ensure that such information is stored and dealt with appropriately. Access to the information must only be given to authorised persons. Consent forms should be held securely, with the data (for at least five years), until the latter are destroyed. Clarify who owns the ideas and outcomes for your research, the data you gather, and other issues surrounding intellectual property for research which is being externally funded, either directly to you or to your supervisor. Disclose sources of financial support and note any special relationship to any sponsor. Report accurately the results of your research and the scholarship of others by using complete and correct information and citations. It is important that ownership of the data and order of authorship are agreed upon prior to submission of a paper to a journal. It is ethically unacceptable to submit a manuscript to a second publisher before a decision has been received from the first.

Appendix: Information to Include in a Participant Information Sheet It is all too easy to manipulate a person’s informed and voluntary consent by exploiting their ignorance, fears and respect for experts or superiors. Applications for projects must therefore be accompanied by Participant Information Sheets that describe, in the participant’s language, the essential points which any reasonable person would wish to know before agreeing to participate in research, including: • The date the participant information sheet was produced. • The project title. • The names of the supervisor(s) for the project and the host institution for the research. • The name(s) of the researcher(s) who will actually make direct contact with the participants. • A statement of what the research is about and the purpose of the research. • An invitation to participate and a description of what they are being asked to do. • An explanation of how they were chosen to receive an invitation.

724

15

How Do I Handle Academic Integrity Issues?

• What the likely consequences are for them should they participate (describe what the discomforts, risks, and or time commitment could be). • What the benefits for them might be (indicate any likely benefits), including any incentive or reward to be provided for participating. • Indicate that they can receive feedback on the results (if desired, they can receive a summary of the results). • What will happen in this research (where, when and how frequently the data will be gathered from them). • Any special conditions of the research that might affect their participation (indicate whether there will be audio-taping or video-recording). • Indicate that there will be no disadvantages/penalties/adverse consequences related to not participating or to withdrawing from the research. • A statement of how privacy will be protected and how confidentiality or anonymity of information will be preserved (provide a schedule for the destruction of any personal identifying information, in the case of confidentiality). • What opportunity do they have to consider this invitation? (Give a time frame for them to consider). • How do they agree to participate in the research? (Need to complete the attached Consent Form). • Indicate that research ethics approval has been given (provide a copy of the University’s Research Ethics Approval Statement and/or approval number). • Who they can contact for further information or if they have concerns (provide a means, for example, a telephone number or email address by which participants are able to contact the researcher(s), the principal supervisor and or the secretary of the university’s Research Ethics Committee) (adapted from Participant Information Sheet Exemplar, Auckland University of Technology https://www. aut.ac.nz/research/researchethics/forms). Note: There are some cases where it is not appropriate to provide a written information sheet, for example, when dealing with young children or adults who are illiterate, it would be more appropriate to provide them with a verbal explanation. A Dialogue Statement of the verbal information that will be communicated should be provided. In order to be more engaging, an information sheet can be set up as a question and answer sheet by modifying the points above into questions, as if being asked by participants, and including responses from the researchers. It is also acceptable to present a consolidated information sheet and consent form, as long as the participants are provided with a copy of the document and have been given an opportunity to reflect on the document before indicating their consent.

References Academy of Management Code of Ethics. (2006). Academy of Management. Retrieved March 7, 2018, from http://aom.org/About-AOM/AOM-Code-of-Ethics.aspx#preamble. Altman, E., & Hernon, P. (1997). Research misconduct issues implications and strategies. Greenwich, CN: Ablex Publishing Corporation.

References

725

Australian Code for the Responsible Conduct of Research. (2018). National Health and Medical Research Council. Retrieved October 22, 2018, from https://nhmrc.gov.au/about-us/ publications/australian-code-responsible-conduct-research-2018. Ben-Yehuda, N., & Oliver-Lumerman, A. (2017). Fraud and misconduct in research. Ann Arbor, MI: University of Michigan Press. Brown, B. R. (2006). Doing your dissertation in business and management: The reality of researching and writing. London: Sage Publications. COPE. (2018). How to handle authorship disputes: A guide for new researcher. Retrieved October 21, 2018, from https://publicationethics.org/resources/guidelines-new/how-handle-authorshipdisputesa-guide-new-researchers. Copyright (New Technologies) Amendment Act. (2008). World Intellectual Property Organization. Retrieved March 7, 2018, from http://www.wipo.int/edocs/lexdocs/laws/en/nz/ nz123en.pdf. Copyright IP Australia. (2017). Australian Government. Retrieved March 7, 2018, from https:// www.ipaustralia.gov.au/about-us/about-this-site/copyright. Cryer, P. (2006). The research student’s guide to success (3rd ed.). Berkshire, UK: Open University Press. Elliott, S., Fischer, B., Grinnell, F., & Zigmond, M. (2015). Perspectives on research integrity. Washington, DC: ASM Press. Emmerich, N. (2018). Virtue ethics in the conduct and governance of social science research. Bingley, UK: Emerald Publishing. Evans, M., Hole, R., Berg, L. D., Hutchinson, P., & Sookraj, D. (2009). Common insights, differing methodologies: Toward a fusion of indigenous methodologies, participatory action research, and white studies in an urban aboriginal research agenda. Qualitative Inquiry, 15(5), 893–910. Faria, R. (2018). Research misconduct as white collar crime. Cham, Switzerland: Palgrave Macmillian. Feldman, D. C. (2003). When is a new submission “new”? Journal of Management, 29(2), 139– 140. Godecharle, S., Nemery, B., & Dierickx, K. (2013). Guidence on research integrity: No union in Europe. The Lancet, 381(9872), 1097–1098. Homan, R. (1991). The ethics of social research. New York: Addison-Wesley Longman Ltd. Humphrey, L. (2017). Tearoom trade: Impersonal sex in public places. London: Routledge. Institute of Professional Editors Limited. Retrieved March 7, 2018, from http://iped-editors.org/. Iphofen, R. (2017). Finding common ground consenus in research ethics across the social sciences. Bingley, UK: Emerald Publishing. Israel, M. (2015). Research ethics and integrity for social scientists (2nd ed.). London: Sage Publications. Jerjes, W., Hamoudi, R., & Hopper, C. (2018). The power of research best practices and principles in research integrity and publication ethics. Amsterdam: Kugler Publications. Kimmel, A. J. (2007). Ethical issues in behavioural research: Basic and applied perspectives (2nd ed.). Malden, MA: Wiley. Knowledge Assets and Intellectual Property Policy. (2015). University of New England. Retrieved March 7, 2018, from https://policies.une.edu.au/view.current.php?id=00117. Macfarlance, B. (2009). Researching with integity. New York: Routledge. Marshall, S., & Green, N. (2007). Your PhD companion: A handy mix of practical tips, sound advice and helpful commentary to see you through your PhD (2nd ed.). Oxford, UK: How to Books. Martin, B. (1999). Suppressing research data: Methods, context, accountability, and responses. Accountability in Research, 6(4), 333–372. Mitchell, T., & Carroll, J. (2008). Academic and research misconduct in the Phd: Issues for students and supervisors. Nurse Education Today, 28(2), 218–226.

726

15

How Do I Handle Academic Integrity Issues?

NHMRC. (2018). Ethical conduct in research with Aboriginal and Torres Strait Islander Peoples and communities: Guidelines for researchers and stakeholders. Canberra: The National Health and Medical Research Council. NHMRC, ARC, & UA. (2018). National statement on ethical conduct in human research 2007 (updated 2018). The National Health and Medical Research Council, the Australian Research Council and Universities Australia. Canberra: Commonwealth of Australia. Nicholas-Casebolt, A. (2012). Research integrity and responsible conduct of research (building social work research capacity). New York: Oxford University Press. Pacific Health Research Guidelines. (2014). The Health Research Council of New Zealand. Retrieved March 7, 2018, from https://gateway.hrc.govt.nz/funding/downloads/Pacific_health_ research_guidelines.pdf. Petre, M., & Rugg, G. (2010). The unwritten rules of PhD research. Open up study skills (2nd ed.). Maidenhead, UK: Open University Press. Polonsky, J. M., & Waller, S. D. (2015). Designing and managing a research project: A business student’s guide (3rd ed.). Los Angeles: Sage Publications. Seadle, M. (2017). Quantifying research integrity. San Raphael, CA: Morgan and Claypool Publishers. Shamoo, A., & Resnik, D. (2014). Responsible conduct of research (3rd ed.). Oxford, UK: Oxford University Press. Sibinga, C. (2018). Ensuring research integrity and the ethical management of data. Hershey, PA: IGI Global. Sieber, J. E., & Tolich, M. B. (2013). Planning ethically responsible research (2nd ed.). Thousand Oaks: Sage Publications. Simmerling, M., Schwegler, B., Sieber, J. E., & Lindgren, J. (2007). Introducing a new paradigm for ethical research in the social, behavioural and bio-medical sciences: Part 1. Northwestern University Law Review, 101(2), 837–859. Staw, B. M. (1975). Attribution of the “causes” of performance: A general alternative interpretation of cross-sectional research on organizations. Organizational Behavior and Human Performance, 13(3), 414–432. Steneck, N. (2007). ORI introduction to the responsible conduct of research (Rev. ed.). Washington, DC: Office of Research Integrity. Stone, E. F. (1981). Research methods in organisational behavior. Glenview, IL: Scott Foresman & Co. Thody, A. (2006). Writing and presenting research. London: Sage Publications. The Tuskegee Timeline. (2017). Centers for Disease Control and Prevention. Retrieved March 6, 2018, from https://www.cdc.gov/tuskegee/timeline.htm. U.S. Public Health Service Syphilis Study at Tuskegee. (2015). Retrieved October 21, 2018, from https://www.cdc.gov/tuskegee/index.html. Unitec Human Research Ethics Guidelines. (2014). Unitec Institute of Technology. Retrieved March 6, 2018, from http://www.unitec.ac.nz/epress/wp-content/uploads/2016/09/unitechuman-research-ethics-guidelines.pdf. University of Pittsburgh Guidelines on Research Data Management. (2009). University of Pittsburgh. Retrieved March 7, 2018, from http://www.provost.pitt.edu/documents/RDM_ Guidelines.pdf. Van Der Weyden, M. B. (2006). Preventing and processing research misconduct: A new Australian code for responsible research. Medical Journal of Australia, 184(9), 430–431. Woodfield, K. (2018). The ethics of online research. Bingley, UK: Emerald Publishing Limited. Woolston, C. (2002). When a mentor becomes a thief. Retrieved October 6, 2018, from https:// www.sciencemag.org/careers/2002/05/when-mentor-becomes-thief.

Chapter 16

How Should I Shape and Defend My Proposal?

16.1

Preparing a Research Proposal

16.1.1 Why Do I Have to Do a Research Proposal? I just Want to Get on with the Study! There is a common misperception among postgraduate students that the real work of research only begins when you start gathering data. Given that doctoral research is a process, data gathering is in reality only one small part of the entire process. While data gathering produces tangible outcomes, the real work actually occurs in conceptualising, contextualising, framing, positioning and configuring your research, so a critical and absolutely formative milestone is the development of the research project itself in the form of the research proposal. The proposal, which will be required in the early stages of your candidature, is, essentially, a detailed plan of your anticipated postgraduate research journey in the context of the degree program you are enrolled in. You will have discussed all elements of this with your supervisor(s) and, in the proposal, your intention is to communicate your clarified thinking in regard to your research topic and the approach you will be taking. In many universities there will also be a requirement for you to provide a verbal presentation of your proposal for comment, feedback and critique to other students, academic staff and, where appropriate, other interested stakeholders. The aim of your proposal is to present and justify the relevance of your topic and how you plan to conduct your research (Walker, 2014). Your proposal presentation will provide an overview sketch of what you plan, condensing the relevant points of your proposal into a short (typically 15–20 min) talk. Essentially, you are providing ‘what’ you intend to study in the form of the topic and research questions and ‘why’ (contextualisation and positioning, anticipated contribution(s), potential limitations), and ‘how’ (guiding assumptions, research frame, research configuration, sampling and data gathering and analysis strategies) you will obtain the answers to your questions. Producing the proposal is the © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_16

727

728

16 How Should I Shape and Defend My Proposal?

‘thinking’ part; the ‘doing’ part is actually conducting the research once approval of your proposal has been received. Your goal is to convince relevant stakeholders that your planned research has a high likelihood of achieving its goals, will add value as an original contribution, will demonstrate your capabilities as a competent researcher, and, most importantly, will likely lead to successful completion of the postgraduate program in which you are enrolled. The aim of a research proposal can be seen from two perspectives: one is to engage you in thinking systematically through the entirety of your project, and the second is to give confidence to your supervisor(s) and other relevant stakeholders (including your academic department) that you have a feasible and convincing plan (at this stage, not necessarily the ability) for conducting your research. As Peters (1997) has commented, one needs to keep in mind that that the process of writing a proposal has several complementary purposes. The proposal is a research plan, an evaluation, a trial run at writing, a contract with your department and other stakeholders, as well as a sales pitch. Writing a research proposal is actually a ‘hard ask’ as it requires you to consider and virtually decide on all aspects of your research before you actually carry it out. You need to consider all aspects and stages of your research project and document your intended approach in each of the stages. Your proposal will answer questions such as: What prior research has been undertaken? What are the relevant contexts for your research? What positioning stances are relevant for you to take into account? What research question(s)/hypotheses do you want to address? Why is the research important? What are your underpinning/guiding assumptions and intentions? Which research frame is helping to shape your approach? What theoretical approach will you pursue (if any)? What sampling strategies will you use? What data gathering strategies will you use? And, what analytical approaches will you undertake? If you take the attitude, “I will work it out when I get there”, you will convince no one; you must critically consider all aspects of your research prior to actually commencing the study. Writing good research proposal will be beneficial, not only for clarifying your thoughts on your research journey, convincing supervisor(s) of your ability to perform, and demonstrating your writing ability, but also for developing a skill which will be particularly useful later in your career. The format and thinking required for external grant fund applications is similar to that of a research proposal, so learning how to do it well will be useful for a career in academia, or possibly even in industry. It is likely this will not be the only time in your life when you will be submitting a proposal of this type. If created well, your proposal should succinctly convince the reader that the problem is significant, worthy of your time and effort and that you have a realistic approach to addressing the question. In general, you will find that a well-maintained research journal will be an indispensable friend as you write your proposal. A further benefit of the proposal preparation process is that, by laying out what you intend to do and offering it up for critique and feedback, you afford yourself a great opportunity to identify any potential problems that may be inherent in your study, problems which, if not resolved now, may hold you up later. Critique and feedback

16.1

Preparing a Research Proposal

729

may not always be negative as we have seen several students benefit from comments that suggest an opportunity to add value to their research in a way that they had not anticipated. So, be open to review and evaluation at this early stage. Your supervisor(s) do not want you to fail and rectifying problems now is a lot easier than further on. In fact, the process of generating a proposal may also stimulate your thinking about a Plan B—contingency plans if your proposed data gathering strategies or sampling strategies do not come off as you intend. You may not be expected to spell out your Plan B in the proposal itself, but you could certainly map it out in your research journal as you formulate your proposal document. One final general observation to make about proposals is that they may take somewhat different shapes and have somewhat different content depending upon the type of research degree program you are pursuing. While our discussion to follow below will largely accord with typical proposals for a PhD or master’s thesis or dissertation proposal, where relevant, we will point out where some variations might occur if you are undertaking a professional doctorate research program (especially where a portfolio is the intended final research outcome rather than a thesis or dissertation) or PhD by publication.

16.1.2 When Is a Research Proposal Usually Provided? The stage at which a proposal is required differs between institutions, with some universities requiring a preliminary proposal at the time of application that is further embellished, shaped and polished through the early enrolment period. Other universities may require submission of a proposal in the first six months after enrolment, or after all course work has been undertaken. For others, the proposal is not required until a significant period of reading and reflection has been undertaken and the literature thoroughly reviewed. Your program regulations will indicate when your proposal is required and, once this is in your planning document, you then need to work back a few months to allow for preparation and critique from your supervisor(s). It has been suggested that you should devote the same amount of time to writing a proposal as you would to a final course paper (Peters, 1997). Most research proposals have 6,000 to 10,000-word limits; however, don’t worry about the length, the bigger problem will be trying to stay within the word limit. Keep in mind that this is not your actual research project or research outcome, this is just one of the first stepping-stones, so be cognisant of the fact that there will be much more to do to complete the entire project. If you become a perfectionist at this stage, the longer you spend on the proposal the longer the delay in getting onto your actual project. Also remember that a proposal does not cast your research in stone. As we have argued throughout this book, unexpected things and things out of your control happen along the research journey and you will likely have to be flexible and adapt. So, while your proposal provides the road map, your journey will probably not unfold exactly according to plan and this is OK! Your proposal is really the starting gate but where you end up may look rather different.

730

16 How Should I Shape and Defend My Proposal?

16.1.3 The Process for Developing a Proposal In the Prelude to this book, we visualised the research journey in Fig. P.1. Figure 16.1 re-presents this journey and maps on to it the thinking pathways you will need to take in order to develop and present a convincing proposal. A proposal is technically one important outcome from your research program, and it needs to be convincing in its own right. Your proposal reflects anticipatory thinking about the plan for your research. You are trying to reduce uncertainty about what you will do in your research by making concrete choices and committing them to a form that can be judged by others. So, what Fig. 16.1 shows is that, once you have adequately prepared for your research journey (phases 1 and 2, including your research journal), a large part of developing your proposal involves building a platform from which your research can proceed. You cannot build such a platform without making decisions about guiding assumptions, research frames, contextualising (including identifying key stakeholders) and positioning your research or without connecting your proposal with relevant prior research (phase 3). Furthermore, you cannot build such a platform without figuring out how to configure your research activities (phase 4). Here is where anticipatory systems thinking comes into play; you need to make choices about how you will navigate the ‘Data Triangle’, including identifying and describing your intended data sources and how you will sample them, describing what data gathering strategies you will implement and what analytical approaches you plan to undertake (phases 6 to 8). All of this thinking needs to be done in recognition of four important points: (1) your end goal is to be able to create convincing research outcomes at the end of your journey (a thesis, dissertation or portfolio being the primary outcome), so your proposal should reflect awareness of this (phase 9); (2) your proposal should reflect critical thinking about your choices and their consequences (e.g., what limitations that they impose or what opportunities they create) as well as about likely obstacles you might need to overcome; (3) the choices you reflect in your proposal

1. Establish/maintain a place for research in your life 2. Prepare for your research journey 3. Contextualise, frame & position your research 4. Configure your research activities

Preparing yourself for the journey Building the overall platform for your research

5. Present a convincing proposal 6. Access/connect with your data sources 7. Implement your data gathering strategies 8. Build meaning from your data

Navigating the ‘Data Triangle’

9. Create convincing outcome(s) Fig. 16.1 The research journey with thinking/development pathways superimposed

16.1

Preparing a Research Proposal

731

should be made in light of relevant paradigm-specific research quality criteria as well as the more general meta-criteria; and (4) that what you propose is not set in stone but will very likely evolve and change as your journey progresses, perhaps dramatically so. To produce a convincing proposal, you must condense all your thinking about this anticipated journey into a relatively few short pages in order to convince relevant significant others that the journey will be worth taking and will yield something original and of value.

16.2

Structure of a Research Proposal

16.2.1 What Is the Normal Structure for a Research Proposal? What is required in a proposal can differ (but usually only marginally) between universities. Once again, your program regulations may have very specific guidelines as to the likely key headings. However, most research proposals include the following discussions and components but these may vary (in sequence and/or in content) depending on the type of study you are undertaking (adapted from Polonsky & Waller, 2015; Punch, 2016). It is perhaps more useful to consider this list as a mapping of substantive content to be addressed rather than as a specific set of headings for the proposal. Check to see if your institution offers a more specific template for the structure of a research proposal (for example, the Australian National University offers a template at https://services.anu.edu.au/research-support/tools-resources/phdresearch-proposal-template; the University College of London offers one at https:// www.ucl.ac.uk/prospective-students/graduate/applying-graduate-study/what-youneed-complete-application/research-proposal-phd; Deakin University offers one at https://www.deakin.edu.au/__data/assets/pdf_file/0009/533826/Template-forThesis-Proposal-for-Confirmation.pdf). 1. Descriptive title—depicting the working title of the proposed research project. 2. Abstract—providing a brief summary of the proposal; it is not an introduction to the proposal or study and, as with all abstracts, it is written last. 3. Key words—you may be requested to provide up to five key words to describe your research. As many journals now request this, it is good practice to develop a key word list for papers that you write. The key words may be used for assigning an appropriate reader to your proposal and/or for classifying your research for institutional purposes and records. You will also use these key words for day-to-day literature searches. However, in the context of a research proposal, it is more likely that the key words are also being sought in order to identify an appropriate supervisor, if one has not already been assigned. 4. Introduction—discussing an outline of the project and a description of what you are intending to do. In this discussion, you want to demonstrate the objective(s) of the research, where the study fits within the current body of literature and/or,

732

5.

6.

7.

8.

16 How Should I Shape and Defend My Proposal?

where relevant, a community of practice, what broad problem you are attempting to address, why it is important, that is, the rationale for the research and the positioning of the study. This is where some of your contextualisation and positioning is done. Here, you are trying to invite the reader to look more closely at the research you are proposing. Literature review [refer also to Chap. 13]—embodies a concise overview of the relevant domains of literature, significant prior research and identification of any gaps in the literature. In a proposal, an extensive coverage of the literature is typically not required; however, you should provide a clear indication of the key theoretical and, where relevant, methodological and/or profession/ practice-oriented domains relevant to your study along with examples of literature in each of those domains. It should also clearly signal how the literature informs the shaping of your research problem and objectives or has stimulated your thinking with respect to an investigative issue, unanswered questions or areas worthy of further exploration. Readers would generally expect this section to close with a clear statement of line of enquiry and why this is relevant. Conceptual/theoretical framework [refer also to Chap. 12]—if relevant to your study, this provides a description of the principal theoretical or conceptual framework you will be employing as well as key concepts and/or constructs and how you see them relating to each other. Such a framework should connect to and logically follow from your literature review and, as such, may form a capstone to your literature review. Research contextualisation, framing, positioning and questions [refer also to Chaps. 9, 10 and 11]—Here you would summarise your primary pattern of guiding assumptions as well as the overall positioning of and frame for your study. Both the broad and specific questions which you plan to address could be detailed in this discussion. Where your research is an iterative process, the starting point and what questions will be asked in the process, could be identified. For more interpretivist/constructivist research, this section may feature initial research questions and domains of interest. If specific hypotheses are being tested in a more positivist study, they should be mentioned here. Some of this discussion could form part of your introductory section and other, more methodologically-relevant aspects could form an early part of the general methods discussion to be discussed next. Methods [refer also to Chaps. 12, 14, 17, 18, 19, 20 and 21]—this discussion identifies your overall research configuration (recall Chap. 12 and the proposed method(s) of inquiry, that is, the type(s) of data that you plan to gather and the possible strategies for gathering them. You should also explain why you selected those approaches, i.e., justify your proposed approach and the reasons for excluding other approaches. You also need to indicate from where, or whom, you intend to draw your data, and the sampling strategy you intend to use; criteria for inclusion and exclusion of data sources should be outlined. How you intend to gather and record the data, that is, measurement instruments, if any, and procedures to be utilised, consistent with the pattern of guiding assumptions you have adopted, and a brief description of how you intend to

16.2

9.

10.

11.

12.

13.

14.

15.

Structure of a Research Proposal

733

analyse the data and display the results should be included. Any intended experimental or quasi-experimental manipulations or naturalistic events that are the focus of your research should be described. Significance—this discussion is sometimes viewed as setting out the intended contribution of the current study to the body of knowledge and/or addressing the question about the benefit or value of your proposed research and for whom (doing this requires that you understand whom your important stakeholders will be). This section should be closely aligned with the positioning of your research established earlier. This discussion can be incorporated into your introductory discussion. Limitations—discussing what you perceive as some of the weaknesses associated with your proposed approach at the onset. This is not to say that other limitations might not emerge as the research progresses. Readers will be looking at: the feasibility and convincingness of your approach, factors that might prevent you from achieving the goals you aspire to in the research and the possibilities for extending your findings to other areas. In this light, you may wish to target a section of your proposal toward anticipating and addressing these potential concerns. This section could take the form of a risk assessment and table which identifies the risks to the research and what procedures have been or will be undertaken to mitigate those risks. This discussion could be incorporated into your general methods discussion. Ethical implications—discussing what, if any, ethical issues do you anticipate? Has ethics approval been given, or will it be sought following approval? If relevant to your study, how do you propose to handle issues of access as well as consent and protection for research participants and the data they provide? This may require the completion of an ethics application for subsequent approval to be included as an annex to the research proposal. [Refer also to Chap. 15— identify the process for ethical approval]. This discussion can be incorporated into your general methods discussion. Project plan—presenting a timeline or Gantt chart outlining the schedule of activities for the period of your research. This is, essentially, the timetable for your research which flags critical milestones and specific completion dates. Anticipated additional research outcomes—discussing possible anticipated research outputs such as journal papers, conference papers or book chapters. In today’s environment, these are considered highly valuable for research-oriented departments and is one way that postgraduate students can contribute to the overall research climate. Budget and resources required—outline any resourcing implications relevant to the project, such as equipment, software or laboratory costs, mailing costs, photocopying, telephone costs, travel, materials, internet survey costs etc. This could also encompass travel and attendance at conferences or training workshops. This discussion could be integrated with discussion of your project plan. Bibliography/references—listing all the references cited in the proposal.

734

16 How Should I Shape and Defend My Proposal?

16. Glossary of terms—where appropriate, this discussion would list definitions of key words, acronyms or relevant terminology used in your proposal. 17. Table of contents—this may form a provisional table of contents for your thesis, dissertation or portfolio. 18. Appendices—could contain: ethics approval, consent forms, drafts of questionnaires or other measurement instruments, topical landscape for semi-structured interviews and, possibly even outcomes of a pilot or trialling study. If you want a broadly applicable set of possible headings for your proposal, consider the following, which would be workable for most postgraduate proposals: • • • • • • • • •

Title page Abstract Table of contents (if you have a glossary, it could follow the table of contents) Introduction (and contextualisation) Literature Review (and conceptual/theoretical framework, if appropriate) Proposed Methods (or Methodology) Project Plan, Budget and Resources References Appendices

If you are doing a professional doctorate, you may need to vary these headings somewhat and perhaps have a heading for a section that discusses anticipated stakeholders, one for a section that describes the research context(s) in more detail and one that describes the intervention, practice change and/or innovation that will form the focus of your research. Once, again, it is important that you check to see if your institution offers more specific guidelines for proposals within specific postgraduate programs. You might have noticed that several of the above headings for a proposal mirror likely chapter headings for your final thesis or dissertation; for a doctoral portfolio, there will likely be more variation, perhaps more specificity, in the chapter headings required. Essentially, you could think of your proposal is a mini-mini thesis and, as Peters (1997) has remarked, “Because the proposal mimics the final structure of your thesis, with some changes, much of the writing you do for the proposal will form the foundation of your thesis, particularly the introduction and literature review and methodology chapter. So, the more you do now, the better”.

16.2.2 Digging Deeper into the Research Proposal? While your final research project may vary from the initial proposal, it is at the proposal stage that you are starting to firm up the key dimensions of your study and your intention is to convince an approving reader or committee that your proposal is good enough to proceed to the next stage. In many institutions, presentation of your

16.2

Structure of a Research Proposal

735

proposal to peers, academic staff and supervisors for further comment and feedback constitutes a critical step in the candidature confirmation process. Although written from the perspective of a business winning a new client, many of the strategies suggested by Hamper and Baugh (2010) are those that a doctoral student might consider in relation to the preparation of their research proposal. The characteristics of a winning proposal have been suggested as being: • • • • • •

evidence that you clearly understand the problem; a strategy plan or design to further investigate and possibly solve the problem; clear documentation of your capabilities of carrying out the research program; evidence of your reliability and dependability; a convincing reason why they should choose or support/confirm you; and your proposal looks like a winner (it has addressed all the correct components and is professionally presented).

So, the purpose of a research proposal is for you to not only do the thinking ahead of time in key areas, but also to get a sense of the timelines involved. Just as you will be looking at postgraduate research outcomes in your area, you may find it valuable to look at other proposals that have been submitted and successfully vetted. When looking at other successful proposals, be mindful that the specific orientation of the qualification should be reflected in your proposal. For example, many professional doctorates (e.g., Doctor of Business Administration; Doctor of Education, Doctor of Psychology, Doctor of Nursing Science, and newer forms such as UNE’s PhD.I) differ from the Doctor of Philosophy (PhD) because of their increased focus on the contextualised creation and application of knowledge, contributions to social change/innovation/professional practice/community development/policy development and, in many cases, the possibility or requirement to produce a portfolio rather than a thesis or dissertation. In contrast, a PhD typically has a focus on making a significant contribution to theory (Lockhart & Stablein, 2002), so keep an eye on the primary orientation of your qualification when writing your own proposal. Having presented a broad overview of the key components of a postgraduate research proposal, it is now appropriate to look at some of the substantive areas in more detail. 16.2.2.1

Thesis/Dissertation/Portfolio Title

Your research topic will be reflected in your research title. It may have already been determined through your application process or by preliminary discussion with supervisor(s). When determining your research topic and title, avoid jargon and go for concise, clear statements that cover the critical dimensions of your research. Try to keep it broad enough to give you some wriggle room if you need it later as your research evolves. Minimalists advocate an eight-word thesis title, with anything longer deemed too wordy. However, most titles are longer than this in order to present a better descriptive snapshot of the research. One approach is to state the

736

16 How Should I Shape and Defend My Proposal?

main discipline area up front, followed by a colon, and then the domain of study, for example, Business Ethics: A cross-cultural comparison in the Asia Pacific Region. Another could be a statement of the research context followed by a colon, then a statement of the research approach. Once again, look at other thesis, dissertation and portfolio titles to get a feel for the phraseology which could be used. Below are some illustrative titles from theses and portfolios supervised by Ray in different disciplines. Note that some are relatively short and sweet single phrases, some are much more elaborate single phrases and some use the general:specific or specific:general pattern of construction. • General:Specific pattern – The Role of Vice-Chancellor in Australian Higher Education: A Role Theory Analysis – Organic Foods and Consumer Choice in Context: An Exploration of Switching Behaviour – Characteristics of Financial Reporting Quality: A Holistic Assessment of Financial Outcomes and Communication Practices – Gross National Happiness Education in Bhutanese Schools: Understanding the Experiences and Efficacy Beliefs of Principals and Teachers – Greening the Wharfies: Organisational Learning for Sustainability at Sydney Theatre Company – Path Dependence: A Prism for Framing Constraints on Adaptation in Australian Dairy Farms – Navigating Pathways through Complex Systems of Interacting Problems: Strategic Management of Native Vegetation Policy • Specific:General pattern – The Influence of Switching Barriers on Service Recovery Evaluation in the Retail Banking Industry in Chile: Construct Development and Testing – Exploring Organisational Citizenship Behaviour in the Federal Hospitals in the United Arab Emirates: A Cross-Cultural Research Study [Doctor of Health Services Management professional doctorate thesis] – The Use and Disclosure of Intuition(s) by Leaders in Australian Organisations: A Grounded Theory – Harnessing Sources of Innovation, Useful Knowledge and Leadership within a Complex Public Sector Agency Network: A Reflective Practice Perspective [PhD.I professional doctorate portfolio] – Importing Western HR Systems and Practices into a Saudi Company: A Case Study Analysis – The Emergence of Group Dynamics from Contextualised Social Processes: A Complexity-Oriented Grounded-Theory Approach • Single Phrase – A Comparison of the Self-Estimates of Vocational Interests of Australian High School Pupils to Measured Interests

16.2

Structure of a Research Proposal

737

– An Investigation into the Feasibility of Using Computerised Adaptive Testing in the Classroom Context to Facilitate Assessment and Reporting of Student Achievement in High School Mathematics – The Adoption of Agricultural Innovations – Research into Identification and Classification of Patterns of NonCompliance in Data Using a Doctor-Shopper Sample – Evaluation of a Theoretical Framework for Explaining the Interrelationships between Country-of-Origin Effects and Consumer-Based Brand Equity – An Investigation of Factors Influencing the Continued and Frequent Use of Internet Banking by Australian Consumers – Understanding Public Perceptions of the Police in the ACT through Observations of Police-Public Turf Interactions and Surveys of the Public – Efficiency and Effectiveness in the Australian Technical and Further Education System [EdD professional doctorate portfolio] – Transformational Leader Behaviours, Social Processes of Leadership and Substitutes for Leadership and their Relationships with Employee Commitment, Organisational Efficacy and Citizenship Behaviours – Recruitment and Selection Practices for Female Administrative Officers in Saudi Public Sector Universities

16.2.2.2

Introduction

Your introductory discussion sets the stage for what you are proposing to research. Here, you will identify your topic area and set out your rationale for choosing it. This rationale could focus on a perceived gap in the literature, in which case, you will refer to the literature to anchor and contextualise your arguments. Alternatively, you may be focusing directly on a specific problem that you feel demands research, say, ageing of the academic work force. In this situation, you would need to contextualise why this is a problem, supporting your discussion by reference to relevant documentation and, where relevant, statistics and other relevant data. For PhD proposals, your introduction should also encompass the significance of your planned research (why it is important to carry and who might benefit), identification of your research context, discussion of your positioning as researcher, positioning of your research with prior literature, setting out a conceptual framework, where relevant, and, of course, identification of your intended research questions. Along the way, it would be useful to identify key stakeholders in your research and how they connect to your project. For professional doctorate proposals, your introduction may also contain a description of the contextualised innovation, program or intervention you plan to develop and investigate and why (this latter will directly address the significance of your development as well as your research). Here, your discussion of the research context and relevant stakeholders needs to be rather deeper than what you would

738

16 How Should I Shape and Defend My Proposal?

typically need for most PhD research. Your own positioning as research/developer as well as perhaps an inhabitant of the research context itself (e.g., your workplace, a community in which you reside, a profession to which you belong, an organisation you work for) also needs to be discussed as does the positioning of your planned participants. One subheading you could find in some proposal formats is “Aims and Objectives”. Here, your goal is to set out your overall learning intentions for your research, that is, what you want to achieve. An alternative subheading might be is “Statement of Purpose”. You could place either of these subheadings immediately after your discussion of your overall topic area and rationale as well as of your intended research context (in the context of prior research for a PhD proposal or in the context of both prior research as well as information from various contextual stakeholders for a professional doctorate). You could then follow with a subheading like “Contextualisation & Positioning of the Research” (recall Chap. 10). For example, your discussion in this section could be, “Within a qualitative action research frame, I intend to improve the process of …”, or, “Using a pluralist approach combining questionnaires and in-depth interviews, I intend to explore new relationships and extend existing theory for…”. In this way, you are flagging the type of study which is to follow in the proposal as well as what you would like to learn from your research. You could also spell out your own positioning as researcher vis-à-vis the research problem as well as other relevant roles in your workplace (the latter being especially relevant for a professional doctorate).

16.2.2.3

Literature Review

Commonly, many institutions do not require a detailed proposal until some months after you commence your postgraduate program. One of the advantages of such a delay is that you would have had time to become immersed in the relevant literature. Because of word limit restrictions, the literature review section of your proposal will clearly be nowhere near the length or depth of the literature review that will be contained in your final research outcome. However, you will need to convey to the reader that you have a firm understanding and appreciation of what has been written relevant to your topic. To this end, for your proposal you should only incorporate discussion of the subset of literature most germaine to the development of your conceptual or theoretical framework, if you develop one, and to setting out your research questions/hypotheses. To ensure that this section is taken seriously, some supervisors may advise you to actually write a draft of a literature review chapter and then summarise it for your proposal. The rationale behind this request is that the student is then able to relax about the world limit, think broadly about the literature relevant to their topic, capture the seminal and recent literature and gain a fuller perspective of the domains of the relevant literature. As the chapter on your literature will be continually modified throughout the duration of your journey, what you would be creating at

16.2

Structure of a Research Proposal

739

this stage is, in fact, just a first draft. However, a better summary for the proposal will be created. In this section of your proposal, the reader will be looking for: • knowledge of the theoretical underpinnings to your study; • the main groupings or domains of literature (feel free to provide your own categorisation and even a diagram); • identification of examples of specific literature within those domains (including both seminal and current material and, where appropriate, relevant grey literature); • a brief critique of that literature; • where appropriate, identification of gaps or deficits in the theory and/or literature that your study will be addressing; and • how your research will address the gaps, challenge or extend the literature. The literature review contained in your proposal is just a snapshot in time that has been taken at the start of your research. That landscape could change over the years as your study progresses and this will be reflected in the final literature review chapter in your research outcome.

16.2.2.4

Conceptual or Theoretical Framework

A conceptual or theoretical framework (recall Sect. 12.5) is more commonly found in your final research outcome but can be useful, given their diagrammatic form, for both your proposal and for future discussions with your supervisor(s). It is important to realise that whether you produce/include a conceptual or theoretical framework in your proposal, the shape it will take will depend upon the pattern of guiding assumptions you decide to adopt (recall Chap. 9) as well as the literature you review. Under positivist assumptions, the conceptual framework may double as your theoretical framework. Under interpretivist/constructivist assumptions, your conceptual framework may be very loose, if you have one at all to start with, and simply depict a landscape of possible issues to be touched on rather than specific preconceptions of variables and relationships.

16.2.2.5

Research Questions

Your research proposal will generally require a statement of your research questions. By clearly developing and stating your research questions, you provide the reader with the focus that anchors your ensuing discussion of proposed methodology (recall the discussions and illustrations in Sects. 11.4 and 11.5). Your research questions and/or hypotheses should flow logically from your conceptual or theoretical framework, if you have one, or from logical argument and critique of the literature you review.

740

16.2.2.6

16 How Should I Shape and Defend My Proposal?

Methodology

The methodology section is usually the largest section in your research proposal and often the one with which postgraduates have the most difficulty. The proposal will require you to document your approach to your intended research right up to the point of actually carrying it out. An “oh, I’ll sort that out later” attitude will not be convincing; all aspects of your intended data-related activities must be considered and documented. In short, you must lay your anticipatory thinking out on the table for all to see. For example, from whom will you gather your data, on how many occasions and using what data gathering strategies? While you may not have reached the point of gaining access to data sources, you should have a full understanding of who will be involved, what their contribution is likely to be and how you will seek and record data from them. What is being evaluated in this section of the proposal is the appropriateness of your planned approach to deliver answers to your research questions/hypotheses and the feasibility, or, if you like, the ‘do-ability’ of your project. The methodology section of your proposal could have a number of sub-sections containing the following stories: • You should briefly discuss the philosophical approach you are taking in your study, which is embodied in the pattern(s) of guiding assumptions you adopt. Note that in many cases, these guiding assumptions will need to be identified and argued for before the research questions and conceptual framework are set out. So, these aspects of the methodology might have to be discussed at the end of the literature review section, just prior to discussion of the research questions or as part of your researcher positioning discussion. This is especially important for non-positivist research. Your adopted pattern of guiding assumptions should influence your choices of data gathering strategies and any reader of your proposal will look for consistency in your choices. If you adopt a pluralist research approach, then your arguments may have to occur on several fronts, particularly if you explicitly choose different data gathering strategies to reflect different patterns of guiding assumptions in different aspects of your research configuration. • You should discuss any ethical issues that you will need to address and of the processes you will need to go through for gaining ethical approval for your research where it is required. While your entire proposal must demonstrate academic integrity, it is here that you focus specifically on the ethical safeguards for participants you will put in place. • You should discuss the research approaches you will use. When choosing the data gathering strategies to use for either primary or secondary research, there are numerous choices from an armoury of methods and associated analytical tools (recall Chap. 14). Once again, the pattern of guiding assumptions that you have chosen will significantly influence the type of data gathering strategies that are appropriate. Of critical importance when constructing the research approach to be used in your study is which data gathering approaches comprise the best

16.2

Structure of a Research Proposal

741

and the most feasible way to deal with the research questions/hypotheses that you have posed. The reviewers of your proposal will be interested in not just what your overall research approach is, but also in why you chose particular data gathering strategies as the most appropriate for your study. Some discussion of alternative choices, and why yours is the preferred choice, would be expected within your proposal. You would also want to give some attention to how you intend to ensure the quality of your research data as one aspect of building convincingness. In positivist research, you should give attention to matters relating to internal and external validity, necessary control procedures, and the actual assessment of validity and reliability. For interpretivist/constructivist research, you should give attention to handling issues of transparency, authenticity, sufficiency and, where relevant, transportability. Over and above these paradigm-specific quality criteria, you should be guided by the meta-criteria as well. If you plan a pilot study or trial of your methods (a strategy we highly recommend), then this should be flagged as well and a sketch about how this would be conducted should be provided. • You should also discuss any measurement instruments or other data gathering techniques and accompanying procedures you propose to employ. Here you want to provide an overview of your planned data gathering strategies. If you are doing research under positivist pattern of guiding assumptions, then it would be appropriate to append at least a draft version of your intended instrument(s) and/ or questionnaires. If you are doing research under interpretivist/constructivist guiding or other non-positivist patterns of assumptions, then it would be appropriate, for example, to append at least a sketch of the landscape of topics you wish to explore with participants if you are planning semi-structured interviews or a sketch of your observational plans, venues, events and foci if you are doing participant observation. How you would record your interviews/ observations and transcribe your interviews should also be discussed. • You should discuss your intended data sources and how you will choose/sample them (see Chap. 19). From where, or from whom, will you be gathering your data? It is at the proposal stage that you will be giving some thought to these questions and to the feasibility of your data gathering activities. If there are any difficulties with obtaining access to your data sources, this could significantly undermine your proposal. Consider who or where the data sources are, and whether those data sources will be able to provide the data you will require to address the research questions you pose. Part of this process will be identifying key gatekeepers you may have to negotiate with to gain access to your desired data sources. • You should briefly describe how you propose to analyse and present the data (see Chap. 21); you cannot leave this until after data gathering and simply hope you will figure things out. You should clearly indicate whether you will employ particular software packages (e.g., SPSS, eViews, Stata, MAXQDA, NVivo, dedoose) to support your data analysis activities. Part of this discussion should also include some comment on data preparation strategies (see Chap. 20). Reviewers of your proposal will want to see that you have given analysis strategies some

742

16 How Should I Shape and Defend My Proposal?

serious thought. It goes without saying that your proposed analytic strategies should consistent with the pattern of guiding assumptions you have adopted and should be appropriate for the type of data you propose to gather and for the research configuration you propose to implement. Reviewers should be able to see how your analysis strategies connect to your research questions/hypotheses. The onus is on you to argue for what you do choose to do. It is possible to be reasonably precise and specific when discussing analysis of quantitative data under the positivist pattern of guiding assumptions and you should flag that you will be conducting data screening (e.g., checking for outliers and other data anomalies, non-normal data distributions, patterns in missing data) prior to commencing analyses. Even under a non-positivist pattern of guiding assumptions, you can give broad indications of how you will approach analysis and data display. There is a fine balancing act to be achieved in writing the proposed analysis section for a research proposal that adopts a non-positivist perspective. This is because the perspective itself requires that your preconceptions are kept to a minimum—you do not set out looking for specific outcomes, which makes planning for data analysis a much less prescriptive, more open-ended affair. Nevertheless, you can make and defend certain choices, based perhaps on what has been used in the previous literature or recommended by particular authors. If you are a visual person, you may know going in that you will prefer to construct visual representations for what you learn, which may impose different software requirements, such as Inspiration, Decision Explorer or even simply PowerPoint. 16.2.2.7

Limitations

In this section of your proposal you are being asked to identify what you perceive to be the potential limitations of your study. Acknowledging limitations is a hallmark of critical reflection on your own research plan. Limitations should be gauged against paradigm-appropriate quality criteria as well as assessed against the meta-criteria offered in Chap. 9. The limitations you identify in your discussion and, down the track, in your final research outcome, should not be perceived as failures in your research skills, or as an opportunity to identify every fault or error in the research (Polonsky & Waller, 2015) but rather as identification of potential hurdles to your study and constraints on your findings. In articulating these limitations, you are indicating that you are going into the study with your eyes wide open and are realistic about the processes and the outcomes. Recognise early that there is no such thing as the perfect study—every research investigation has flaws. The trick is to minimise the most damaging flaws and your proposal should clearly show how you will do this. You will not be able to negate all potential flaws, because in the end your study must be doable, and this will necessarily mean that you must make trade-offs between what you would ideally like to do and what is realistic for you to do—it is from these trade-offs that limitations emerge. IF you can do this critically and transparently, your proposal will be more convincing.

16.2

Structure of a Research Proposal

743

The approach to take is to stand back, be quite dispassionate about your research and ask yourself these questions, “What could get in the way of my drawing clear and unambiguous conclusions from my study?”; “What methodological impediments remain that I cannot feasibly handle?”; Where could surprises emerge that could limit my findings (this is not unlike the thinking about Plan B referred to earlier)?”; “What could limit the generalisability/transportability/applicability/ usefulness of my findings?” (When writing up the limitations in your final research outcome, you could take a similar approach by asking questions such as, “If I were to do this research again, what would I do differently, and why? What went right and what went wrong? What could have improved the process or the results?”) Once you have identified the potential limitations of your study, discuss them and suggest solutions as to how they could be resolved, or mitigated. Treat this as a learning exercise and demonstrate that you have not only been able to objectively gain insight into some of the weaker points of your research, but have reflected upon them, and have suggestions as to how they could be addressed or acknowledged. We can all get a bit tunnel-visioned in relation to our research, which is why peer review and feedback can also be very useful in pointing out potential limitations in our research project.

16.3

Writing Up Your Proposal

It is advisable to check with your supervisor regarding word limit and structural expectations for your proposal. If you have a writing block and can’t get going on the proposal, Peters (1997) has a good suggestion. Before writing the actual proposal, write a short pre-proposal. This is a few pages in length and is a much shorter treatment of the topic and, as a consequence, is more manageable, less scary and one that you could quickly receive advice on. If you have more than one topic in mind, then you may wish to write two or three short pre-proposals for feedback. If you are a visual person, mind mapping your proposal may help to dissolve the blockage. Once you have settled on your topic, write your proposal from the reader’s point of view (Jay, 2000). As a consequence of your prior reading and the preparation of this proposal, you have now become fully acquainted with the theories, concepts and constructs associated with the topic. Be aware that the reader is coming to it fresh, that is, they don’t have the benefit of all the background. So be very conscious of the fact that you cannot take great inferential leaps with their understanding. You should avoid jargon and not assume that the reader of your proposal will understand all the terminology you are using. Often proposals are read by a fairly broad range of academics who may not be fully acquainted with your field of study. If you are doing a professional doctorate, your audience may be more diversified to include professionals, community members, workers and so on, depending upon the nature of your project; your proposal will need to speak to such a diversified audience. If necessary, compile a glossary of terms and put it at the front or back of the proposal, as appropriate. It is probably better to spell out each

744

16 How Should I Shape and Defend My Proposal?

concept in full at the first opportunity. The same is true for any acronyms you might employ in the proposal—nothing is more annoying for a reader than to encounter an impenetrable cloud of acronyms. Give each section a heading. This may already be provided to you by the format suggested by your supervisor or institution but, in the absence of any guidelines to this effect, utilise the headings to your advantage as they break up the page and improve the layout and flow of the document (Jay, 2000). Sub-headings are also helpful for the reader if they need to look back through the document to find a section. Having read the entire document, it is not uncommon for a reviewer of a proposal to go back to the methodology section to check that the methodology proposed will, in fact, deliver on the questions posed. Unfortunately, many proposals fail to present a cohesive, integrated flow of material from start to finish. Writing a proposal is much like baking a cake. Each ingredient in and of itself would be rather tasteless and uninteresting but, when the ingredients are combined in a precise manner and allowed to interact and unite into one entity, the whole becomes attractive, palatable and highly desirable (Stewart & Stewart, 1993). In relation to tone, your proposal should reflect an underlying confidence but not arrogance. If you are numbering sections, be careful about getting the sub-sections too small. The subdivision of numbers in a small document further than 1.1, 1.2, down to 1.1.1.2 can get messy. Tammemagi (2010) suggests a number of useful techniques when writing proposals. Adapting his list to the postgraduate student researcher, he has recommended the following points: • avoid jargon or complex wording that might not be understood by the reader; • use lists when presenting a number of items or points rather than stringing the items together in long sentences as lists tend to be easier to comprehend; • use paragraphs to separate main sections of your proposal; • use summary tables wherever possible; • ensure that you match your information to the headings provided; • don’t repeat lengthy sections; • try not to ‘over-do’ the proposal; it is still a proposal; • ensure the proposal does not have typographical, spelling or grammatical errors; and • beware of gaps or leaps in your logic. Finally, the reader of your proposal is not just interested in your research study and how you will approach it; your proposal will also reflect your ability to write clearly and concisely. As a consequence, the same rules apply as for any other written material that will be reviewed by others. Ensure that it is absolutely free of any typographical, grammatical or spelling errors, as these can be extremely distracting for the reader. It is regrettable, but true, that readers are more likely to dismiss your project on the basis of sloppily written presentation; you task is to negate such perceptions by making your proposal ‘tight and right’. Have your proposal read over by someone else. Preferably, have them read it twice, first for

16.3

Writing Up Your Proposal

745

clarity of expression and understanding and, secondly, to pick up any typographical errors. If you are a postgraduate student where English is not your first language, try to ask a native English speaker to read your proposal for clarity.

16.4

Self-evaluation/Reflection

Before submitting your proposal to be evaluated by your supervisor(s), you may wish to first undertake a self-evaluation. This is an opportunity for you to be brutally and critically honest with yourself regarding what you are considering for your research and what you have written. Basically, you want to see if you consider your own proposal convincing. Here is a list of questions that you could ask yourself about your proposal: 1. Does your proposal show imagination and intellectual craftsmanship? 2. Is your research problem clearly stated and have you appropriately contextualised it? 3. If relevant, are your hypotheses clear, unambiguous and testable? If you have research questions instead of hypotheses, are your questions clearly stated? Can they be feasibly addressed? Do they follow logically from your review of literature and/or from your reflections on the problem context? 4. Is your research problem appropriate in scope in light of the degree you are enrolled in? 5. Are your research frame and configuration clearly set out? 6. Have you positioned both yourself and your participants with respect to the proposed research? 7. Is your methodological approach feasible within the timeframe you are constrained by and with the resources you have access to? 8. Can you gather high quality data and have you appropriately anticipated the necessary ethical processes/protections? 9. Does your proposed approach(es) to data management and analysis make sense vis-à-vis your research questions/hypotheses? 10. Is your intended sample to be drawn likely to be receptive to your research? 11. What are the consequences if: Your research processes fail? You have a high participant or institutional refusal or dropout rate? You cannot access the data sources you need in order to gather all of the data you require? 12. Are your major research activities listed and key milestones identified? 13. Is a time estimate attached to each major activity and milestone? 14. Is your research trying to do too much? If yes to this question, what would make the project more manageable? That is, does your research require more scoping and shaping? Another very useful strategy for your self-evaluation is to reflect upon how convincing your proposal is in terms of meeting the expectations of the research quality meta-criteria (recall Chap. 9, Sect. 9.7, as well as relevant paradigm-specific

746

16 How Should I Shape and Defend My Proposal?

quality criteria, Sect. 9.6). For the purposes of self-evaluation/reflection, the Contextualisation and Realisation meta-criteria domains will be most relevant, but you will also be casting your mind forward to consider issues relevant to certain Explication meta-criteria domain (recall Sect. 9.7.3.1 and Fig. 9.9). Table 16.1 lists the 12 meta-criteria for convincingness along with keys questions you could ask yourself as you conduct your self-evaluation/reflection. One important thing to note is that, if you evaluate your own proposal against the meta-criteria, this may make writing certain portions of your final research outcome somewhat easier by providing you with handy information that can help you make the overall outcome much more convincing. The outcome of your self-evaluation/reflection process may be that you will make changes. However, if you decide to go ahead and submit your proposal despite some lingering doubts, at least you will be mentally prepared for any of the feedback that you might get. If feedback and critique from others confirms your own hesitations about an element of the research, you will be more prepared to accept it and make the necessary changes.

16.5

Proposal Review and Approval

16.5.1 How Does One Get a Proposal Approved? In some universities, the approval process is one of review and comment by a committee, with the possibility of further revision being undertaken prior to being given the notification of full acceptance of the proposal. The Research Proposal Committee (or equivalent) is usually comprised of the program director, a selected group of academic staff and, invariably, if there is a visiting academic around, they frequently get roped in. In other universities, the proposal itself is not approved, but candidature in the degree may be confirmed with the proposal and its presentation to staff and peers playing key parts in the confirmation process. This basically gives the student institutional (not ethical) permission to proceed with the research phase of their study. Look at your program regulations or ask around in order to find out who approves proposals and what the process and timeframe is. Is it the supervisor (s)? Is it the head of the department or graduate school? Is it the program director, or is it a committee? Do you need to defend your proposal at a seminar? Is this part of the approval process? When is the proposal due? Is the proposal part of a larger confirmation process and, if so, how does this work? In a formalised proposal approval process, your proposal may be assigned to one or two reviewers, besides your supervisor, who will read and comment on it. In a confirmation process, the entire confirmation panel may review your proposal. As you approach this approval stage, you need to have the right frame of mind. Try not to look at it as an assessment exercise where you are being judged, because whoever is reviewing your material is, in fact, doing you a huge service. Not only are they checking for potential problems which could trip you up later, but they are

16.5

Proposal Review and Approval

747

Table 16.1 Meta-criteria as aids to your self-evaluation/reflection process (adapted from Cooksey, 2008) Meta-criteria domain

Meta-criterion

Self-evaluation questions

Contextualisation

Juxtapositioning with other research

Does your proposed research logically connect with relevant previous research and, where appropriate, with a specific problem context or issue? Have you remained open to prior research and perspectives that have shown findings contrary to what you expected or adopted guiding assumptions that may differ from yours? Are you reflecting critically on where and how you are situated with your research, keeping in mind your current skill set, your anticipated developmental needs, your other life and work roles and, where appropriate, your roles within the context in which you wish to do research? Have you argued clearly for the pattern(s) of guiding assumptions you have/will adopt in your research and is the mode of knowledge you are pursuing clear? Have you made clear arguments for the research frame that provides the over-arching logic for your proposed research? Have you appropriately acknowledged the positioning of potential research participants in the research context, being mindful of their roles, expectations and fears as well as of your ethical and cultural obligations? Have you considered key relevant stakeholders and their perspectives as they relate to your proposed research? Have you made appropriate choices for the types of data sources you wish to connect with? If you plan to connect with non-human data sources (e.g., media stories, organisational reports, policies, publications, and so on), are you aware of the contexts in which and purposes for which these sources were created? Will your proposed research activities appropriately acknowledge and honour the contexts in which you will be obtaining data, including cultural traditions and historical events and trends? Does it appear that you will be able to appropriately interpret your analyses, results and conclusions in light of the context(s) in which the data were produced?

Researcher positioning

Positioning of participants & other data sources

Contextual sensitivity

(continued)

748

16 How Should I Shape and Defend My Proposal?

Table 16.1 (continued) Meta-criteria domain

Meta-criterion

Self-evaluation questions

Realisation

Internal coherence

Given your adopted pattern(s) of guiding assumptions, can your proposed research meet the expectations of relevant paradigm-specific quality criteria? Have you clearly set out your planned research configuration and the data gathering strategies that will be implemented within that configuration? Given your adopted pattern(s) of guiding assumptions, can your proposed research meet the expectations of relevant paradigm-specific quality criteria? Have you clearly set out your intentions regarding generalisation and/or transportability? Have you clearly indicated how you will sample data sources from which to gather your data, including identifying key gatekeepers you may have to negotiate access with? Will your analyses have a good chance of producing/ revealing appropriate meanings from the data you gather? Have you made the significance and likely contribution your proposed research would make clearly evident; in short, have you addressed the question of why your research is important to carry out? This meta-criterion is much less relevant at the proposal stage but becomes critical when you are writing up research outcomes This meta-criterion is much less relevant at the proposal stage, but becomes critical when you are carrying out analyses, developing interpretations and conclusions and when writing up research outcomes Have you clearly signalled awareness of the limitations that your choices of research frame, contextualisations, positionings and configuration impose upon your proposed study? Have you shown how you might overcome or compensate for certain limitations (signalling a Plan B for your research could be useful here)? Is your proposal produced in accordance with any institutional guidelines and professional standards that are relevant? Have you proofread your proposal to remove all typos, spelling and grammatical errors and have you read it through from beginning to end to ensure logical consistency and coherence from start to finish?

Extensional reasoning

Analytical integrity

Explication

Value for learning

Fertilisation of ideas

Handling of unexpected outcomes

Acknowledgement of limitations

Presentational character

16.5

Proposal Review and Approval

749

also acting as an early warning system as to the ‘do-ability’ of the study. Frequently, they may also point to issues or opportunities that you may not have thought of. If the study is too large then, given their experience, it is here that they will flag the size of the project as being an issue and recommend that you make appropriate scoping and shaping changes. They may be able to save you both time and money, so try not to be defensive about any feedback that you get or changes that are requested from the committee. In hindsight, you may find that they have saved you a considerable amount of time and effort and that taking their feedback on board will enhance your chances of success in the program.

16.5.2 Presenting Your Proposal It has been said that one useful litmus test to gauge the degree of lucidity of proposed research is to try to explain the project to a peer in one sentence—a feat that few academics can master (Dadich & Fitzgerald, 2006). One sentence may be difficult; however, as part of the approval process, you may be required to explain your project in a presentation and in support of your written proposal. The intention behind this activity is that you develop not only your verbal presentation skills but that you also gain experience in answering questions as well as receiving and handling verbal critique. If you have a chance to present your proposal, grab it, as a great opportunity to not only share your hard developmental work with others but also to receive input into improvements that can be made to the various components of your proposal. If you are to orally defend your doctoral or master’s proposal, speak to students who went through the process last semester or last year, and ask them what questions they were asked, and how they handled the process. Many of the questions asked will have been specifically related to the research project being presented, but you may get a feel for what the common types of questions are and what some of the hot buttons or key issues are for some academics. More often, the oral presentation of your proposal will be done in a collective session where other stakeholders (even non-academics—more likely for a professional doctoral proposal presentation) may be present and where other postgraduates are also presenting or are at least part of the audience. With any luck, you will not be the first. Rather than sit there in a daze, sweating it out in anticipation of your presentation, listen to how the members of the audience critique the presentations before you. When you are going to present your proposal, follow the etiquette associated with such presentations; that is, should it be a straight verbal presentation, three overhead slides or a full PowerPoint presentation? Find out what is expected. Avoid the most common mistake of putting too much into your presentation. You do not have the time to cover everything in great depth, but it is important that you get the breadth of coverage. The positioning of your research (contextualisation, your epistemology and research paradigms, frame) will be important at the start of the presentation in order for the audience to get an understanding of the type of

750

16 How Should I Shape and Defend My Proposal?

research you are doing. The positioning also provides a frame of reference from which the audience can then move forward and assimilate the more technical information that follows in regard to your chosen methodological approach. Be very careful with timing your presentation and make sure that time is allotted for each section of the proposal. Stay aligned with the content of your written proposal. Ensure you have the right allocation of time and are not rushed at the end. When you realise that you only have two minutes to speak on the methodology for your proposed research, that certainly focusses what you intend to say! Practise your presentation. In our experience, the biggest mistake made by students in their proposal presentations is taking too long to get through the background and literature aspects of their project, leaving too little time to set out the research questions and methodology. Remember, the better your attention to your research questions and methodology, the less likely it will be that you will get tossed a curly question. As a rule of thumb, if you have 20 min to present your proposal (with 10 additional minutes for questions form the audience), reserve at least 10–12 min of the 20 to focus on your research questions and methodology; this will mean you have 8–10 min for the background context, literature and theory. Where possible, use a software support system such as PowerPoint (https:// products.office.com/en-au/powerpoint), Prezi (https://prezi.com/product/) or Slide Dog (https://slidedog.com/) to prepare and organise your slides (even Inspiration has presentation preparation and support capability, in addition to its mind mapping and diagramming capabilities). Your presentation is not only an opportunity to receive feedback on your approach, but also builds skills you will find necessary for future presentations, both within the department and at conferences. Predictably, you will be somewhat nervous about presenting your ideas to fellow academics, however, it is hoped the material will be particularly familiar to you and you can comfort yourself with the knowledge that, given the volume of reading you have undertaken when preparing the proposal, you probably know more about this specific aspect of the subject than the others present. When it comes to question time, once again, don’t be defensive. It is an excellent chance to get some insight into what the limitations of your research might be and an opportunity to correct them before you begin. Engage in the discussion and don’t hesitate to admit what you don’t know. Remember, this is a learning experience. If a question is way off-topic, then a simple response could be “that’s an interesting question but not what this study is about.” and then move on to another question.

16.5.3 What Are Stakeholders Looking for in a Proposal and How Would They Evaluate It? The following things are typically looked for in a proposal: • a clear and reasonably well-defined area of interest that has relevance; • evidence that you are familiar with the literature in your field of interest; and

16.5

Proposal Review and Approval

751

• a specific focus, problem, related questions or a proposition which you will argue within the field which you intend to pursue. The key dimensions for a research proposal are not dissimilar to those that are used for evaluating a thesis, dissertation or portfolio. Specifically, in relation to a proposal, the reviewer will be looking for: • Is the research problem and purpose clearly set out along with logically developed research questions and/or hypotheses? • Is there confidence that the literature has been reviewed and the conceptual/ theoretical foundations and, where relevant, practice-based foundations, identified? • Are the research methods described in sufficient detail for the student to obtain the answers they are seeking, in accordance with the epistemology and pattern (s) of guiding assumptions they have chosen? • What are the means by which the data will be gathered, managed, analysed and presented and are these means congruent with the framing, contextualisation, positioning and configuration of the research? • Is the researcher aware of any potential risks or flaws, or limitations in their research? • Will the research make a contribution to the field of study and/or context of application? In discussing the primary qualities of a doctoral dissertation proposal, Kilbourn (2006) has commented that proposals are working documents on the way to the production of a final postgraduate research outcome, and the qualities of a proposal very much guide the qualities of that outcome. Kilbourn (2006) suggests that the qualities of a proposal are: 1. How informed is the Introduction? Is it easily understood? 2. How long before you understand what the proposal is about? 3. Is there a clear articulation of the problem that the study will address? How far do you have to read before you have a clear sense of it? 4. Is a plausible argument made for doing the study? Will the study likely make a significant contribution to practice and/or theory? 5. Are the questions that the study will address clear? Do they seem reasonable given the nature of the problem? 6. Is the literature review adequate, and is it conceptually integrated with the problem and the questions posed? 7. Is there a convincing argument for any theoretical perspective taken in relation to the problem? Is the perspective (or theoretical framework) from which the data will be collected and analysed clear and reasonable? 8. Is there a convincing argument for the methodological approach taken? 9. Are the methods of the study spelled out in detail? Will they have a good chance of answering the questions the study poses?

752

16 How Should I Shape and Defend My Proposal?

10. Is your researcher role in the inquiry clear and acceptable, given the nature of the problem? 11. Is the configuration of the study apparent and does it have integrity? Is there a coherent train of thought that runs through your proposal from the beginning to the end? 12. Is the structure of a proposal apparent and lucid? Do the parts fit together? 13. Are the transitions from one part of the proposal to another clear and helpful? 14. Are the ethical considerations clear and acceptable? 15. Is the proposal well-written? Do you guide the reader through the work?

16.5.4 What Are the Most Common Problems That Crop Up in Research Proposals? • Not proof-reading the proposal and submitting it with significant errors which only serve to annoy and alienate the readers. • Not getting sign-off from the supervisor(s). • Not giving supervisors enough time to read the proposal. • Submitting the wrong version of the proposal (yes, this happens!). • Starting your project without approval from your supervisor(s), confirmation panel or committee as appropriate for your institution, or without ethics approval. • Utilising the wrong format; not following the recommended format your supervisor, department or school has suggested. • Not sticking to the word limit. • The literature review is not integrated or related to the research purpose and research questions. • Not adequately covering the rationale or purpose of the study. • Uncertainties with regard to your research plan: whether the project will actually work, whether your data gathering and analysis strategies will provide what is needed to answer the research question(s), and whether your project is feasible and doable in the context of your program of study. What to avoid in your proposal: • Stylistic, formatting and presentation errors. • Lack of a concise statement of the purpose of the project. • Unclear, insufficiently developed or inappropriately stated research question(s)/ hypotheses. • Inadequate literature review or a literature review with no evidence of critical consideration. • An ill-considered research configuration. • No rationale for the data gathering strategies. • Sample nature and/or size not specified, or no rationale given for the selection of participants. • Ignoring any consideration of analysis approaches.

16.5

Proposal Review and Approval

753

16.5.5 Proposal Approval Outcomes Usually, there are four categories of approval (these may be formal or informal, depending upon your institution) and approval may be attached to your proposal or to your candidature in general (as part of a confirmation process): (a) Approval and acceptance of the proposal or confirmation of candidature, in which case, you are OK to proceed. (b) Approval granted pending the meeting of certain minor conditions which does not require re-writing and resubmission of the proposal or re-review of candidature (e.g., approval may be granted pending completion of the ethics review process or some developmental training in a specific area). (c) Approval withheld, pending the meeting of major conditions, normally requiring revision and re-submission of the proposal for further consideration (typically signals a flawed and unconvincing proposal, but where the research problem is still deemed acceptable). (d) Approval denied, where you may be required to develop an entirely new proposal (research problem deemed not acceptable or doable) or where your candidature may be discontinued (really only happens if those making the decision find that you do not have the requisite capabilities to successfully complete the research). Obviously, outcome (d) is the least desirable and is one to be avoided if at all possible. With reasonable effort, motivation and appropriate contact and interaction with supervisor(s), almost all postgraduates can avoid this outcome. It is probably fair to say that the most likely outcome is either (b) or (c). In regard to suggested revision and resubmission of your proposal, this may require the proposal to be rewritten and returned to the committee or, alternatively, they may delegate that approval to your supervisor. It is, however, more common for the committee to retain ownership of the approval process until the proposal is deemed to be acceptable. Once approved, the project is then passed on to the supervisor to assist with the action of the research.

16.5.6 Making Changes to Your Proposal It is not uncommon for requests for further clarification or for a revision of a research proposal to be made. While this can be somewhat gruelling, take it on the chin and treat it as part of the standard review and critique process that accompanies most academic work. Where amendments to a proposal have been requested, do not be discouraged. It is an excellent learning experience and is not dissimilar to the way you will approach feedback from reviewers of submitted journal manuscripts. Here is one approach you could take with respect to revision:

754

16 How Should I Shape and Defend My Proposal?

• First, read the feedback then put it aside and reflect on it, resisting the temptation to become discouraged or defensive. • Meet with your supervisor(s) to discuss the feedback. Come armed with some recommendations you have regarding changes that should be made, and seek supervisory input on both the comments and your recommendations. • Solicit further suggestions for changes from your supervisor(s). • Go back to the proposal when you are fresh and ready, having had a period of reflection on what modifications should be made to your research. Remember, the changes you make here may significantly impact on data gathering and the possible length of your research journey, so be careful when incorporating new elements. • Having made the changes, give the proposal to your supervisor(s) to look at, prior to re-submission. • Go back to your project plan and make modifications based on any additional tasks you believe may be required, and change deadlines, given the delay that the revision of the proposal has caused (adapted from Stewart & Stewart, 1993).

16.6

Conclusion

The implicit expectation of a postgraduate degree is that the candidate is capable of independently conducting original research of a standard that is expected of professional researchers in their particular discipline (Finn, 2005). Therefore, a research proposal must give confidence that when the project has been implemented, it will result in a study worthy of being written up as a doctoral research outcome. As a consequence, a proposal will need to demonstrate that the research can: • Be undertaken by an independent researcher. • Contribute to knowledge—does the nature of the research question provide you with the ability to contribute to the foundation of knowledge in the area? • Address a clear problem that has not been addressed before. • Achieve intended outcomes with the proposed methodology. • Provide evidence of critical evaluation—does it contain an appreciation and appropriate evaluation of the conceptual and theoretical dimensions? • Be original—is the research going to provide original contribution suitable for publication? Will you, in future, be able to publish from this material? (Finn, 2005; Kilbourn, 2006). While there are specific elements a reviewer will look for in a proposal, there is an over-riding parameter against which your proposal will be evaluated. This parameter is one of feasibility. Unlike a professor who has research assistants, a postgraduate student has fewer options for delegation and typically rather fewer resources to call upon. It is, therefore, essential that the project can be undertaken by an individual and successfully completed. In the absence of any substantive methodological flaws, the lingering question will be whether the research can be undertaken within the

16.6

Conclusion

755

obvious constraints that surround postgraduate research at this university. These constraints relate to time, accessibility, resources and ability of the researcher, and so on. A very good piece of advice is that your research does not have to address ‘the meaning of life’. However, it is far better to find a gap in the knowledge or relevance to practice (Panda & Gupta, 2014) that can be explored in the appropriate time that you have and under the constraints with which you have to live. The research proposal should convince the reader that you have a good understanding of how you are going to tackle your research question, and that you will be able to complete the project. However, don’t be surprised if your final research project varies considerably from your research proposal. That is to be expected as you are exposed to new concepts, theory and processes. Your research journey will not always go according to plan. However, your proposal is a working template to give an indication of how you intend it to proceed; it signals your intentions. Regrettably, students are often just ticking the boxes when it comes to writing a research proposal, considering it a necessary but somewhat cumbersome and distracting step on the way to the real business of doing the research. This attitude or approach is unfortunate as you can miss out on the value and significant benefits that can be gained from doing this exercise well and the opportunity to iron out many of the issues and circumvent at least some of the obstacles you may encounter before you actually start. Anticipating the various aspects of your research through the proposal process is an indispensable step toward a less bumpy research journey. As well, you will find that writing a coherent and convincing proposal gives you a great opportunity to develop the writing and logical argument skills you will need to write a convincing final thesis, dissertation or portfolio outcome. As a final check, the Institute of Education at the University of London suggests some key questions for you to answer in connection with the research proposal: 1. Have you given yourself enough time to plan and write an accomplished proposal? 2. Have you read all the right institutional forms and guidelines thoroughly? 3. Do you have a clear idea of all the rules you have to follow in submitting your proposal application? 4. Have you spoken to people who can help? 5. Have you given time and thought to your proposed research project? 6. Do you have a clear objective in mind? 7. Have you thought about what makes your proposal convincing and unique? 8. What criticism do you anticipate? 9. Have you given thought about the best way to structure your proposal? 10. How does your proposal relate to your career interests? 11. Have you considered how your confidence and experience apply? 12. Have you written a clear first draft? 13. Have you met the word limit? 14. Have you carried out any revisions? 15. Have you taken the draft to an academic for advice?

756

16 How Should I Shape and Defend My Proposal?

16. Have you acted on this advice? 17. Have you clearly noted the deadline for submission of your proposal? For a further checklist, see Appendix: Research Proposal Check List. For further reading on how to write a proposal, you could consult one of the following texts: Krathwohl and Smith (2005), Locke, Spirduso, and Silverman (2014), O’Leary (2017), Punch (2016), Sternberg (2014), or Stokes and Wall (2014).

16.7

Key Recommendations

• The skills that you develop in writing up a good proposal will not be lost if you intend to stay in academia, as a proposal is very similar to an application for internal or external research funds. You will also use these same skills in writing up your thesis/dissertation/portfolio and in writing journal articles, professional reports and conference papers. • Research proposals are future-oriented in that they set out what you intend to. While the proposal attempts to plan out in detail what you intend to do, modifications are to be expected as your project progresses. • A simple way of looking at your research proposal is that it addresses the ‘what, why, how, when and where’, with ‘what’ being what you intend to study, the ‘why’ is the justification/significance of the research, the ‘how’ is how you intend to undertake the study, ‘when’ is the timeframe, and ‘where’ is the location of your research. • There is a definite link between the sections of your proposal. Your positioning as a researcher will influence both the epistemology of your study and its emerging research questions. Consequently, your choice of pattern of guiding assumptions, research frame, contextualisations and positioning of participants and other data sources will suggest likely data gathering strategies. Your analytical methods will be predicated on the data gathering strategy or strategies you choose which, in turn, will influence how you will undertake data display and presentation. • Of equal importance is not only what strategies you use for your data gathering, but also why those strategies were chosen over other possibilities. These arguments may be connected to overcoming certain limitations or obstacles in your research. • Like all documents that go to an external audience, it is a good idea to have the proposal proof-read by someone else to double check it for typographical, grammatical or spelling errors. • Remember your audience as you write your proposal. If you are undertaking a PhD, your audience will primarily comprise other academics and you will need to write with appropriate academic tone, disciplinary awareness, language and style of argumentation congruent with Mode 1 knowledge production. If you are undertaking a professional doctorate, your audience will likely comprise a mix

16.7

•

•

• •

Key Recommendations

757

of academics and non-academics and possibly other stakeholders as well. This means that you must achieve a balance in your writing in order to connect with such a diverse audience. If you write too ‘academically’, you will not engage non-academics and vice versa. Typically, what this implies is that you will need to moderate the academic tone and language in your proposal, but not so far as to alienate your institutional and academic stakeholders (since they hold ultimate sway over whether you are awarded the degree). Your arguments need to connect not only with academic expectations about theorising and learning but also with non-academic expectations about change, costs and benefits, utility, consequences and unintended side-effects. Before submitting your proposal for approval, have your supervisor(s) check it over and to ensure further clarity have a colleague outside of your field read it as well. If your native language is not English, arrange for a native English speaker to review your proposal. If you are doing a professional doctorate, have a non-academic colleague review your proposal as well. If you are orally presenting your proposal, the same rules apply as for conference and other academic presentations. You need to be organised, have your structure clear in your mind, design, balance and pace your presentation appropriately and, of course, practise your presentation. Your audience may appreciate receiving a hard copy of your presentation slides to follow along as you present and so they can record more targeted notations and observations to circle back to with their questions. Do not proceed with your research until you have had your proposal approved and/or your candidature confirmed, and you are also in receipt of institutional ethics approval, where required. While your research proposal is going through the approval process, keep yourself busy by continuing with your literature review activities and other work relevant to identification of potential data sources.

Appendix: Research Proposal Check List To recap, the intention of your research proposal is to cover a number of key questions, including: • • • • • • • • • •

What are the purpose and aims of the research? What is your broad research problem? What are your specific research questions/hypotheses? What prior research is relevant to the area of study? What areas of theory inform those research questions? What is the underlying epistemology and guiding assumptions? How are you positioning your research? What is the theoretical or conceptual framework (where appropriate to have one)? What data gathering strategies are you are going to use (and do you have a Plan B)? How will you pilot test or trial your research approach?

758

16 How Should I Shape and Defend My Proposal?

• How will you go about addressing each research question? • How will you approach the sampling of data sources, including how you will negotiate access via relevant gatekeepers (and do you have a Plan B)? • How will you address any ethical concerns and expectations? • What form of analysis and data displays are you considering? • What do you think are the intended outcomes? • What are the potential strengths and weaknesses of the data sources and what does this mean for your research plan? • What might be the benefit or value of this research and for whom? • What do you anticipate might be the limitations of the research plan?

References Cooksey, R. W. (2008). Paradigm-independent meta-criteria for social & behavioural research. Proceedings of the 2nd Annual Postgraduate Research Conference, University of New England, Armidale, NSW, pp. 4–17. Dadich, A., & Fitzgerald, A. (2006). Qualitative research in the making: A practical guide to project design. Paper presented at the ACSPRI Social Science Methodology Conference, Sydney, Australia. Finn, J. A. (2005). Getting a PhD: An action plan to help manage your research, your supervisor and your project. London: Routledge. Hamper, R. J., & Baugh, L. (2010). Handbook for writing proposals (2nd ed.). New York: McGraw-Hill Education. Jay, R. (2000). How to write proposals and reports that get results: Master the skills of business writing (Rev. ed.). London: Prentice Hall. Kilbourn, B. (2006). The qualitative doctoral dissertation proposal. Teachers College Record, 108 (4), 529–576. Krathwohl, D. R., & Smith, N. L. (2005). How to prepare a dissertation proposal. Syracuse, NY: Syracuse University Press. Locke, L., Spirduso, W., & Silverman, S. (2014). Proposals that work: A guide for planning dissertations and grant proposals (6th ed.). Thousand Oaks, CA: Sage Publications. Lockhart, J. C., & Stablein, R. E. (2002). Spanning the academy-practice divide with doctoral education in business. Higher Education Research & Development, 21(2), 191–202. O’Leary, Z. (2017). Doing your research project (3rd ed.). Thousand Oaks, CA: Sage Publications. Panda, A., & Gupta, R. K. (2014). Making academic research more relevant: A few suggestions. IIMB Management Review, 26(3), 156–169. Peters, R. (1997). Getting what you came for: The smart student’s guide to earning a Master’s or a Ph.D. (Rev. ed.). New York: Noonday Press. Polonsky, J. M., & Waller, S. D. (2015). Designing and managing a research project: A business student’s guide (3rd ed.). Los Angeles: Sage Publications. Punch, K. F. (2016). Developing effective research proposals (3rd ed.). Los Angeles: Sage Publications. Sternberg, R. (2014). Writing successful grant proposals from the top down and the bottom up. Thousand Oaks, CA: Sage Publications. Stewart, A. L., & Stewart, R. D. (1993). Proposal preparation (2nd ed.). New Jersey: Wiley. Stokes, P., & Wall, T. (2014). Research methods. London: Palgrave.

References

759

Tammemagi, H. (2010). Winning proposals: How to write them and get better results (3rd ed.). North Vancouver, CA: Self-Counsel Press Inc. Walker, R. (2014). Writing a research proposal. Retrieved October 27, 2018, from http:// transnationalteachingteams.org/documents/Toolbox%207/Writing_A_Research_Proposal.pdf.

Chapter 17

How Can I Gain Access to Data Sources?

17.1

Gaining Access to Data Sources

It is interesting that many textbooks on research methods discuss data collection methodologies and review the advantages and disadvantages of various data collection methods. However, they are relatively circumspect when it comes to the discussion of actually negotiating access to data sources. Most texts assume that the researcher has already neatly identified their likely participants and will have no difficulty engaging their participation in their study. The reality, however, is far from this rosy picture as postgraduates may experience repeated rejection, or a commitment that fizzles out at a later date owing to a change of management, change of attitude or the departure of a senior executive. Difficulty in obtaining access, restrictions placed on access, or withdrawal of support can be devastating to a postgraduate researcher and can cause not only considerable emotional upheaval, but such a setback can also have ramifications on project quality as well as completion. The fact that many of the problems associated with lack of access to data sources involve the researcher’s often unanticipated loss of control over what happens in their research and this magnifies the emotional upheaval and project risk. Data gathering can involve the greatest costs in terms of not only time and money, but also the mistakes that occur at the data gathering stage can have a huge impact long-term. Mistakes at the outset are rarely reversible and can restrict the type and quality of data gathered. Given the importance of accessing data for research, we are also surprised at how little attention is given to this in the support provided to postgraduate students. Hayes (2005) reports on a researcher’s experience of gaining access to three statutory social work agencies in order to conduct a study examining how social workers respond to family support cases. One of his conclusions is that researchers need to give greater priority to access considerations. Training programs provided by universities and preliminary discussions undertaken with supervisors often put the focus, initially, on scoping the topic and research questions. However, where the © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_17

761

762

17

How Can I Gain Access to Data Sources?

information will be generated and who will answer the questions is sometimes not given sufficient attention. In reality, postgraduate researchers often experience considerable difficulty securing sufficient data from appropriate sources to achieve a robust study. Data access could involve digital and/or personal interaction with participants as well as access to documents and other artefacts and access to these may be controlled by one or more gatekeepers (Broadhead & Rist, 1976). For many postgraduates, this will be their first experience in trying to negotiate with gatekeepers to secure access to data sources unknown to them. Prior research does provide some interesting insights into a variety of data access circumstances such as accessing digital repositories (Feijen, Horstmann, Mangho, Robinson, & Russell, 2007), crowdsourcing (Sheehan, 2018), dirty or hidden data (Larsen & Walby, 2012), hard to reach populations and social media (King, O’Rourke, & DeLongis, 2014), vulnerable and Indigenous populations (Anderson & Hatton, 2000; Smith, 2012), schools (Wanat, 2008), children (Mauthner, 2006), young adolescents (Riesch, Tosi, & Thurston, 2007), Hispanics (Evans, Coon, & Crogan, 2007) and virtual communities (Norcera, 2002). For postgraduates who are required to make direct approaches to organisations or communities in order to gather their data, this can be most challenging given the skills required of cold calling, sales techniques and negotiation, all of which could be necessary for securing a commitment from another party who basically has no inherent interest or desire to support you. Accordingly, this process will be a focus of this chapter.

17.1.1 Why Is Gaining Physical Access Difficult? The more common reasons why gaining physical access to data sources is difficult are: • Because of the time and resources involved, organisations or individuals may not be prepared to engage in what is, essentially, a voluntary activity. • The request for access to the organisation may fail to interest the person who receives it. • The organisation may find itself in a difficult situation owing to internal or external events totally unrelated to any perceptions about the nature of the request or the person making the request, so they have no choice but to refuse access (Tang, 2008).

17.1.2 What Is Actually Involved in Gaining Access? Put simply, there are three steps in the process of accessing information: • specifying the kind of information required to answer the research question(s); • identifying the sources of this information and methods of gathering it; and • devising the techniques and instruments for collecting the information (Thomas & Brubaker, 2007).

17.1

Gaining Access to Data Sources

763

This description is quite clinical and doesn’t allude to the complexities that can lie between the points. For example, sending questionnaires from a database with a covering letter is relatively informal, but access is more problematic when you have to approach an organisation or community and actually gather data from its members. You need to overcome any issues or problems that the organisation or community may have regarding your research, as well as with interact with organisational or community members. Some of the issues you may need to consider are: • establishing your credibility with intended participants; • developing your access on an incremental basis; • identifying possible benefits, to the organisation, of their participation in your study; • using suitable language in the data collection process; • facilitating ease of reply when requesting access; and • allowing sufficient time (Tang, 2008). Based on the work of Buchanan et al. (1988), and Matthiesen and Richter (2007), we have proposed a number of more detailed steps for negotiating access in social and behavioural research. Negotiating access is, essentially, an ongoing process of determining what is required, getting into an organisation or community and gaining their permission, obtaining access to physical materials/artefacts and/or participant involvement and then possibly doing this all over again with another organisation or community. For our discussion, we have suggested a ten-step process for successfully gaining access to data sources.

17.2

Ten Steps for Gaining Access to Data Sources

17.2.1 Preparation Before figuratively knocking on doors, you need to undertake some preliminary thinking as part of the preparation and planning process. This includes choosing an appropriate research approach, deciding on the degree of researcher involvement, identifying key figures and gatekeepers, anticipating obstacles and defining mutual benefits (Rossman & Rallis, 1998, pp. 91–112). You need to go further than you have been before in considering, for example, who your participants are, what level of access you require, what type of data you need, how much time is involved, is your approach reasonable and feasible, and what your time plan is (Denscombe, 2017)? Given privacy issues and other ethical constraints, consideration also needs to be given as to how permission can be granted and how participants’ rights will be protected. For example, if you are using the customer or staff database of an organisation, what permissions will those on the database need to give you in order to allow you to research them. You will need to get specific permission not just

764

17

How Can I Gain Access to Data Sources?

from the organisation but from individuals themselves and, as part of that process, you need to indicate to them what their rights are as participants and how you will protect their information in your research. It is far better to develop a viable research project at the outset than to commence with a research project that proves, after expenditure of a great deal of energy, to be unrealistic. As part of your preparation, and as an extension of your initial proposal and the development of your research approach, you will need to consider a number of issues. To assist with thinking through data sources access issues, we present a number of preparation issues, which you may wish to turn into questions to ask yourself in relation to your own research. In responding to these questions, try to think as laterally as possible, be open to being a bit creative and, remember, this is just the start of the process. There will be a lot of twists and turns, you may hit a brick wall or new opportunities may open up to you. The issues we present are as follows and you may wish to record your initial responses in your research journal as this could potentially be useful in discussions of methodology in your research outcome. • Prior research—what has been done before in similar studies? What were the sources of data? How many? What methods did they use? What were the participation or response rates for studies which have used a similar configuration or approach? What have they suggested should be done differently? • Research questions—what is your research problem and what are your research questions? Where or with whom might the answers to these questions be? Where might the holders of this information be? What type of data will you be generating? • Unit of analysis—consider what your unit of analysis is. Is it an individual (e.g., an historical figure, consumers, teachers), a group of individuals (e.g., IT entrepreneurs), an existing or historical organisation (e.g., Enron), a group of organisations (e.g., a chain of hotels), a cultural subset (e.g., Chinese Malaysians), or is it a general cultural group (e.g., all Samoans)? • Key associations—with whom does your unit of analysis interact? Who is important to them? What associations do they belong to? This is critical to consider because one or more of these associations may actually be gatekeepers who control preliminary access to the sources of data you want to tap into. For example, to access teachers and/or students in any government school, this may first require an approach to the Ministry or State Department of Education. Only after their permission has been received can a researcher then approach individual schools to negotiate access. One organisation may be able to facilitate the other organisation you wish to approach. Here is an example from our own research. In order to study primary schools that have introduced Values Education into their curriculum, the means through which these schools were accessed was through their membership of the New Zealand Character Education Foundation, thereby making data collection access relatively simple once you have engaged with the Foundation. • Sampling—what defines your population and how large is it? Alternatively, what defines the set of individuals you want to connect with? What is your proposed sampling plan? Who are the target participants? Where can you obtain a sample frame? From the sample frame, how do you propose to generate your

17.2

•

•

•

•

•

Ten Steps for Gaining Access to Data Sources

765

sample? What numbers do you require (keep in mind that a proportion will decline to participate)? If, for example, you are using an internet questionnaire, they commonly have very low response rates. Are there any criteria for the selection of the sample and the rationale for the size of the sample? What are the inclusion or exclusion criteria for sample/participant selection? If your data sources are documents or other artefacts, who controls access to them and how will you select which ones to obtain. The number of participants—can I get access to enough data to warrant the use of this type of data source? For quantitative questionnaires, that depends upon your intended inferential or multivariate statistical analyses, and as a very rough estimate, you would need 5–10 participants per question on your questionnaire. Data gathering strategies—what data gathering strategies are most appropriate for the type of participant? The strategies used are often dictated by the sources of data. For example, busy CEOs are unlikely to fill out a questionnaire but may be more willing to be interviewed or allow you to observe a meeting. Permissions—how accessible are the data sources? What permissions are required and from whom? This could vary. For example, no permission is required where the information is of a secondary nature and has already been published and is publicly available through library, government or industry sources. This would be the case, for example, if you were conducting a meta-analysis of published literature or using publicly available annual reports of companies or databases. However, for primary data sources, you may need to navigate a hierarchy of permission layers, from an initial gatekeeper (e.g., a government body) to secondary gatekeepers (e.g., organisations, schools) to tertiary gatekeepers (e.g., the managers or supervisors of the people from whom you want to seek data) to the individual themselves (to obtain their informed consent to participate). Planning is essential in order to ensure you have all of the required permissions covered. Additionally, each layer may require a slightly different approach or process to seeking their permission so you shouldn’t assume you can get away with just a single type of approach. Your supervisor(s) can play a key role in the planning process here. Possible hurdles—what issues might there be around use of databases that may provide lists and contact details for potential participants? For example, a professional association is unlikely to give you access to their database of names because of privacy issues. Similarly, an organisation is unlikely to give you a list of employees to sample from. Thought, therefore, needs to be given as to how to get around the restrictions. For example, try asking for your questionnaire to be enclosed in the organisation’s newsletter and offering to pay for that month’s mail out. Or, alternatively, co-opt an organisational gatekeeper who will agree to act as your intermediary where, for example, you would provide your invitation to participate to them and they then distribute to those you want to target. Potential problems—what are the potential problems associated with the data source access point? For example, are you sure that the HR manager will actually fill out the questionnaire, or will they pass it on to somebody else? Will the organisation let you gather data on their premises, or will you have to

766

•

•

•

•

17

How Can I Gain Access to Data Sources?

arrange for an off-site data gathering location (which means you have to plan and negotiate for participant’s travel to and from the site, along with permission for them to miss that period of work, etc.)? Know what you don’t want—what compromises will you be willing to make (here we see another reason to have a Plan B ready for your research sampling plan)? Will these compromises be in relation to the type of individuals or organisations involved in your research or the number of participants or the amount of time you will be allowed to have access to them, etc.? Often it is difficult to know what restrictions could be placed on your research. However, it is a good idea to try to anticipate what could happen and what the impact might be. Information quality—what potential biases do you risk in the way you intend to select participants? What impact might the data gathering methodology have on the quality of your study? If, for example, you are using the ‘snowball’ sampling scheme, where you are obtaining referrals from one individual to another, what effect might that have on the quality of information you are gathering? What biases may subtly be injected? What errors or problems might occur? For more detail on snowball sampling, see Atkinson and Flint (2001). With random sampling, errors occur when the sample members selected for your study are not representative of your intended research population and, response errors occur when participants give inaccurate answers. Data recording—Problems might also occur, not just with the answering of your research questions but with your recording of the data generated. In research guided by positivist assumptions, these issues relate to the reliability and validity of your measurements as well as to the internal and external validity of your research configuration. For interpretivist/constructivist research, these issues relate to matters of transparency of data gathering as well as to authenticity and sufficiency. Time frame—what are the time frames? When will ethics approval be secured and what is the potential time line associated with each stage of securing your data sources? For example, how long will it take to construct and send out an inquiry letter, undertake a follow-up phone call, set up a meeting with the organisation, do a presentation, address any requirements, attend another meeting with the company, have the arrangements for access approved and get permissions? This will often be required as part of the ethics approval process so needs to be undertaken early before you can initiate contact within the potential participants, and gather your data etc.

What type of data do I need? The type of data you need will depend entirely on your research and could be either primary or secondary data. For example, you could be looking at: • Historical accounts—these could include descriptive chronicles which trace events over a period of years in a family, organisation, ethnic group or movement etc., or interpretive histories which trace incidents over a period of years, such as biographies or autobiographies (Thomas & Brubaker, 2007).

17.2

Ten Steps for Gaining Access to Data Sources

767

• Open access and Databases—there are extensive opportunities for both qualitative and quantitative data regard to the provision of open access material databases. For open access material, look into OpenDOAR (www.opendoar) which contains over 900 worldwide predominately institutional repositories. In the UK, Economic and Social Data Services (ESDS) are jointly funded by JISC and the Economic and Social Research Council (ESRC). ESDS provides resource, discovery, access and support for a wide range of quantitative and qualitative economic and social data. The analysis of databases is assisted by automatic summarisation systems or text mining tools, which are used to search out relevant papers, cluster them into related topics and produce short summaries of each paper. The UK data archives (http://data-archive.ac.uk/) house data collections in the Social Sciences and Humanities. They are a lead partner in the Economic and Social Data Services (https://www.ukdataservice.ac.uk/) which include four specialist data services, government, ESDS, international ESDS, Longitudinal and ESDS Qualidata. NaCTeM (2016, see www.nactem.ac. uk) is the national centre for text mining but provides text mining services for the academic community. OSSWATCH (2014, see www.oss-watch.ac.uk) is an open source advisory service. • Case studies and ethnographies—case studies reveal individual attributes of a particular person or institution. Ethnographies identify beliefs and customs shared by members of a social system. (Case studies are singular in nature; ethnographies are more group focused.) For postgraduates undertaking a case study or an ethnographic study, the level of involvement will depend on the research approach. A more positivist approach will rely significantly on sample composition and data gathering accuracy, where an interpretivist post-modern approach will have the student more immersed in the data from an observational or possibly participant perspective. Text books on ethnographic research with chapters on access include Hammersley and Atkinson (2007) and Burgess (1984). • Direct-data methods—these involve soliciting data from individuals, groups or institutions by means of interviews, questionnaires, or observations. Given the growth of available data bases, there are a number of support texts which could prove useful to you specifically in relation to institutional repositories (Buehler, 2013), big data and the public good (Lane, Stodden, Bender, & Nissenbaum, 2014), research algorithms (Scholarly Editions, 2013), data mining (Sumathi & Sivanandam, 2006), managing and sharing research data (Corti, Van den Eynden, Bishop, & Woollard, 2014; Fienberg, Martin, & Straf, 1985), data linkage methodology (Harron, Goldstein, & Dibben, 2015), digital libraries and information access (Chowdry & Foo, 2012), data protection and data access (Mochmann, de Guchteneire, & Guchteneire, 1990), data access and research transparency (Cambridge University Press, 2015) and more general digital research methods (Halfpenny & Proctor, 2015; Fielding, Lee, & Blank, 2008; Best and Krueger, 2004).

768

17

How Can I Gain Access to Data Sources?

How will my sampling affect access issues? The sampling scheme you use will relate significantly to the problems you will encounter with accessing data sources. If, for example, you are undertaking probability sampling using a random selection (where each member of the population has an equal chance of being selected for the sample) of people from a telephone, email or other kind of public directory or listing, no preliminary contact is required. It is merely a matter of hoping that the person who answers the phone or receives an email will be responsive to your invitation to participate. The challenge in these cases is either ensuring that the directory or listing itself does not have in-built biases or limitations or is compensating for them in some way. Non-probability schemes such as convenience sampling involves targeting who you can get to or encounter easily whereas purposive, theoretical or quota sampling relies on the personal judgement of the researcher to decide who to approach for inclusion in the sample (Polonsky & Waller, 2015). Snowball sampling relies on the judgments of others to suggest who to access. These non-probability approaches have inherent difficulties, particularly in relation to potential choice biases, which you need to transparently act to overcome. Having considered some of the preliminary issues, you now need to look at the practicalities of obtaining access to data sources. How might approval be obtained from key members of relevant organisations and individuals within the organisation from whom you will be soliciting information?

17.2.2 Identify Key Contacts Research in business, health, community, group or educational environments may require you to contact one or several sources. Getting to these individuals or organisations is usually by way of a key contact or through gatekeepers. Although the term ‘gatekeeper’ generally has a negative connotation as an individual who restricts access, it is an important role that researchers must acknowledge and plan their approach to. They come in a variety of forms and could be a senior or even a junior person in the organisation. The most common gatekeepers you will encounter are personal assistants to important individuals or chief executives of organisations (e.g., school principals, head nurses, CEOs). Bear in mind that a gatekeeper can act in a positive manner by providing you with access through such actions as a personal introduction, approval and/or endorsement (approval implies no necessary endorsement of your research, it simply grants access; endorsement implies overt support for the purposes of your research and encouragement to participate). The challenge is to find the relevant gatekeeper(s) and to engage them to achieve such positive access outcomes. A distinction can been made between ‘warm contacts’ (those you have generated through your own personal, your supervisor’s or other professional networks, such

17.2

Ten Steps for Gaining Access to Data Sources

769

as when you attend conferences), and ‘cold contacts’ who have no idea about you or your research. Cold access key contacts or gatekeepers are usually identified through databases, listings, internet searches, supervisor contacts, and so on (Matthiesen & Richter, 2007). Your elevator pitch may work well for dealing with warm contacts but will probably be too short on detail and implications for a cold contact. Working with a cold contact may require multiple meetings, with the first likely being a meeting for introductions and building of trust, without pursuing access permission. Some basic points to keep in mind when considering gatekeepers: • Find out who the gatekeepers are for your research context(s)—ask around to determine who will makes decisions regarding researcher access to data sources and who works closely with those people. See if anyone you already know has connections to those people, e.g., the gatekeeper’s spouse was in the same cricket team as your supervisor’s cousin … use the network (Delamont, Atkinson, & Parry, 2000). You only find out these associations by asking around, e.g., “Does anyone have any contacts in the insurance industry?” • If there are no prior personal connections you can draw upon, do your homework—this may entail contacting the organisation in question and saying to the receptionist, “I am about to send a letter/email to your CEO, may I please have the spelling of his/her name and the correct title?” Engage with their personal assistant (i.e., gatekeepers may have gatekeepers!). • Keep accurate and tidy records—enter the contact details into your own database and keep journal records of your interchanges with each gatekeeper. • Develop a brief précis of what your research is about—this has been called the 45-second sound bite strategy, and is used to prepare for meetings with gatekeepers (Tolich & Davidson, 1999). It has been suggested that students prepare by imagining they are in a railway station where you are to meet your cousin who is there to catch another train. The cousin asks you what the ‘destination’ of your research is just as the train becomes visible. In the time available, you have to explain your research project in a summarised, clear and succinct form. This is very similar to what has been called the ‘elevator pitch’ but with an extra twist that you must make sure you indicate why it is you want access to the data sources they are gatekeeper for. Try to get your explanation to take fewer than 5 min, and then reduce it to 45 sec as that may be all the time you have if you are making the contact via telephone call. • Approach gatekeepers confidently but sensitively—be conscious of how well you look and present yourself. As we know, a lot of communication is through non-verbal means so think about how you can create a good impression. Be prepared to answer the questions ‘why should I agree to give you access to my organisation/group/department …?’ and ‘What’s in this for us?’ • Communicate your needs and intentions regarding your research while ensuring you understand what the gatekeeper’s expectations are—make it clear what will be expected of participants and indicate the value of their contribution. You also need to think carefully about what you commit to providing back to

770

17

How Can I Gain Access to Data Sources?

the organisation with respect to what you learn from your research. Some organisations might expect to see some kind of summary report on what you have learned as a condition of allowing you access. If you promise to provide such a summary report, you are ethically bound to ensure that you deliver. Some organisations may ask to see the responses of individual participants as a condition of granting access. Ethically you cannot agree to this unless each and every implicated participant has given their informed consent permission for this to happen (very unlikely!). Here is where knowing what your limits are comes into play. If you received such a request from an organisation, you would probably need to delete them from your sampling frame and live with whatever limitations on your results that decision might lead to.

17.2.3 Determine What Is of Value to the Key Contact and Their Organisation When gaining access to potential data sources, you need to think like a salesperson. In doing so, place yourself firmly in the shoes of the client. What are their needs, what are the difficulties they are experiencing and how can you help? When the gatekeeper/ client/potential data source is listening to you, they will be thinking from a personal point of view, which is, essentially, “What will this do for me?” In contrast, postgraduates are approaching it from their perspective of, “How can they help me?” What should I promise in order to gain access? Only promise outcomes you can deliver on. It is unlikely that you will be giving everyone a copy of your research outcome. However, a one-page summary of your research conclusions would be appropriate. Do consider the time frame—the data collected a few years ago will have little relevance for participants who have long since forgotten their participation and your promise of feedback. Are you willing to exchange services? For example, you could promise to a set number of consultancy hours or offer a workshop to the organisation. What information would either the participants or the key contacts find beneficial? What are they hoping to learn or improve? What information do they want? This could mean that you insert an additional set of questions into your data gathering efforts in order to satisfy their informational needs, or you provide advisory support for what they might be undertaking.

17.2.4 Make Contact There are a number of questions you should now be finding answers to as your data gathering phase comes into full swing. Questions should now be resolved such as: Who are the organisations that will be most suitable for the project? How many of

17.2

Ten Steps for Gaining Access to Data Sources

771

them are eligible? Who could be a sponsor for you in this research? Who would be most interested in the findings you are generating? What authorisation is required? Are there any additional stakeholders that need to be brought on board? Who are the target research participants? Where are they, how many do you need (Buchanan, Boddy, & McCalman, 1988)? You should now be fairly clear on who the main decision-makers are (it could be a CEO, school principal, department head or a board) in relation to your study. Essentially, who can say ‘yes’ or ‘no’ to your data gathering efforts being conducted within that organisation? The gatekeeper and the decision-maker may be the same person or, alternatively, the gatekeeper may be the stepping-stone needed to reach the decision-maker. You will now need to make contact either with the preliminary gatekeeper or with the actual decision-maker. Commonly, a combination of methods is used. Initially, contact is usually made by email or letter that heralds a telephone call, followed by a one-on-one or face-to-face meeting, possibly followed by meetings that may have increasingly more individuals present. With community groups and Indigenous research, personal consultation is used more often. Let’s walk through these approaches. How do I make the first contact? Draft an introductory letter/email explaining who you are and what your research is about, who will benefit from the research and how much time and resources will be involved. State why their organisation has been chosen and is important to your study. Gather your contact list of names and addresses. While most people are used to sending emails, for some managers this is still considered to be an informal mode of communication and a more formal letter, that is professionally laid out, on university letterhead if at all possible, and appropriately signed, could be more appropriate in some cultures. Send the introductory letter either directly or attached to your email. Indicate at the end of the letter/email that you will be following up with a telephone call to arrange for an informal meeting if required to discuss any questions they may have. Send the letter/email and, as promised, make the call within a week. Any tips for that all-important first contact? First contact via telephone call is a personal and direct communication as a follow-up from your correspondence. While the telephone is a personal medium, for many postgraduates it is also stressful. You need to exhibit high levels of confidence, practising what you intend to say, having notes in front of you. Scripting the conversation could be considered although you do need to come across as confident and relaxed, rather than stilted and rehearsed. Remember, you were probably not on their business plan for this year so somehow you need to sell them the idea of the benefits to them of your research. Here are some suggestions when making contact with potential gatekeepers/data sources: • Create a list of the key points that you want to cover. You could script this, but it does need to flow easily and not sound like you are reading it. • Practice what you are going to say out loud.

772

17

How Can I Gain Access to Data Sources?

• Make sure that you have the right name with the number (yes, mistakes can happen when you are phoning a few people on a list). • Rehearse the person’s name a few times before you pick up the phone. • Introduce yourself and where you are from slowly. It may be all too familiar to you, but this information will need extra processing time on the other end of the phone. • Don’t ask them is this is a convenient time to speak as you could start to sound smarmy and like someone from a call centre. • Be confident, positive and upbeat (without sounding manic). • Breathe regularly and remain relaxed. • Don’t ramble on and on. Possibly ask them a question to get them talking. • Listen to the person you are talking to and give them the opportunity to pause and digest what you are saying. Don’t speak over them. • Be sensitive to their time. If they sound harassed suggest that you can call back at a more convenient time. Make sure you arrange a time for this call-back. Make your contacts at appropriate times. If you are phoning internationally, you need to be cognisant of time differences. Most research is, however, undertaken domestically, and the best time to phone is probably first thing in the morning. Always ask “are you available to speak now” and don’t be hesitant to arrange another more suitable time to telephone if something else has come up for the person. Start by identifying yourself and establishing the purpose of your contact and, if appropriate, how you were made aware of this individual, i.e., through a personal contact or a professional association. Where appropriate, refer to your prior correspondence and very briefly recap on the information (they already have all the information but may have just skimmed it). You may find this hard to believe, but apparently, you’ll come across much better on the phone if you keep a smile on your face and project that smile on to the other person while you are talking. It is also suggested that telephoning while standing literally heightens your sense of authority while sharpening your mind. You may be fortunate and access for your research is granted over the phone. You should then ask for the name of the person you should liaise with. The person granting approval is usually the manager and they will then pass this on to an assistant or someone below them. You need to get this name. If access is not initially forthcoming, your main aim is to get in front of this person (it is harder to turn someone down in person) so be fairly intent on getting an informal meeting arranged as soon as practicable but without sounding too pushy. How do I ensure I make a good impression at the first meeting? This first meeting is critical as it is where the potential participant organisations identify you and make judgments about you. With the first contact, professionalism is the key in the way that you approach the company, provide information to them, present yourself and verbally present your research. The individual or individuals you are about to see are taking valuable time out of their day and are coming to your study completely cold. They have no vested interest in your study. On the other

17.2

Ten Steps for Gaining Access to Data Sources

773

hand, you have been actively involved in this research for many months, and you have a high level of knowledge and emotional involvement as the project is important to you. Looking at it from both perspectives, you will realise that there are two ends of the continuum, theirs being more passive and indifferent, and yours enthusiastic, but anxious and nervous. Somehow you need to bring them into the centre, first by relaxing and, secondly, by exciting the individuals you are meeting about your research and getting them on board. Suggestions here are: • Always confirm your appointment once it has been set—this is particularly important when you need to travel to an appointment. Once an appointment has been made, confirm the appointment by email or letter so that there is a written record of the appointment. • Re-confirm—a day prior to the proposed meeting, re-confirm the meeting. It is not unusual for an executive to suddenly realise that they have a clash with your proposed meeting time. In anticipation of this, it is always good to have another time in mind when you would be able to meet. If a change is requested, be courteous. Remember, they are doing you a favour by seeing you. Immediately get another appointment. Changes in meeting times can be frustrating, but it is all part of the inevitability of undertaking research, and you should just take this in your stride. • Do your homework on the organisation—this may involve several avenues of pursuit, i.e., the internet, their annual reports, etc. However, being able to casually mention an aspect of their financial performance, recent achievements or a recent product launch will show that you are not blindly entering the discussion. • Review—on the day of the meeting, review your background notes and research project material, so that the meeting will have a relaxed feel to it. Ideally, you would like to get to the point of being able to participate in the meeting without referring to notes, which can be off-putting to some. • Arrive in good time—when you arrive at the reception desk, clearly announce your name, and indicate the name of the person you have come to see, then take a seat. Don’t waste precious time, pull out a paper that you need to review or a draft chapter that needs editing. The person may be delayed, and in 15 min you can achieve quite a bit. When the person comes to greet you, you also look productive. • Have all your material prepared—keep the material you are making available to a minimum—they don’t need to see your proposal, introduction and literature review chapters, they just need a one- or two-page summary of the research. Have enough copies for everyone at the meeting, plus a few more for individuals who may be dragged in at the last minute. Come armed with your credentials (e.g., a CV, postgraduate student card, introductory letter from your supervisor) in case they ask about your background and/or ask for verification that you are indeed pursuing research for degree purposes. • Place a time limit on your meeting—you would be lucky if you got an hour.

774

17

How Can I Gain Access to Data Sources?

• Know the names of the individuals who will be at the meeting—if additional individuals turn up, simply jot down their name on a piece of paper so you can refer to it. It helps if they give you their business card. The Chinese business practice of leaving the cards on the table in front of you face up is a very sensible practice. Incorporate people’s names in your discussion and make regular eye contact with them. • Have an agenda—after pleasantries and introductions, state your agenda, something like—“Thank you for taking time out of your busy day to meet with me. In this meeting I would like to cover …” Outline the research and what you hope to achieve, what you are looking for in the way of access, and what you are able to offer (if you think you may need some points for negotiation purposes, you may want to hold off on this latter point at this time). • At the meeting your aim is to engage them to gain their interest—you want to pique their interest, if not excitement, in regard to your research. Pitch your presentation to the level of the audience. Avoid using jargon or words they may not understand. Provide some informative background (people always like to learn something new), but don’t lecture. • Your presentation should take approximately the first twenty minutes then open it up for discussion—don’t do all the talking; encourage engagement as this is where you will be able to pick up on any issues they are not clear about. Unless you have really honed your presentation, there is usually a little misunderstanding as you have attempted to convey the complexities of your research in layman’s language. • Their interest will be less on the specifics of your research approach than on the outcomes and potential disruptions to normal work flows—focus on the benefits, what you are looking to find and what problem you are wishing to address, rather than the mechanics of the study. Emphasise that you have thought carefully about the extent of the imposition and disruption to the work day that your research could create with individual participants and that you have worked to ensure minimise these as far as possible. • Discuss the needs that the organisation may have—it is not all about you. Ask questions about the organisation and listen carefully to the responses. People love talking about their organisations and the issues they are experiencing and sharing them with someone outside the organisation is strangely cathartic. Listening attentively and providing the occasional insightful comments will endear you to those at the meeting. • Clarify questions—indicate that you are quite passionate about your research and ask whether there are any ambiguities you would be happy to clear up. • Indicate what you need—once you have clarified any questions they may have, move on to the next stage which is, essentially, your wish-list of when you would like to start and how many people you would like to have involved and/or what documents you would like access to. This list would have been thought of in your preparation phase where you consider everything you may need, right down to the possibility of a spare office and telephone (Don’t hold out for that,

17.2

•

•

•

•

•

•

Ten Steps for Gaining Access to Data Sources

775

but sometimes large organisations will be able to provide somewhere you can camp while you are gathering your data). Have a draft agreement pre-prepared—have a draft access agreement on hand for when the meeting looks to be going well. It can simply be in bullet points as to what they will provide and what you will provide. It doesn’t have to be in legal terminology, just plain, simple language. This will be needed for ethics approval from your own institution. However, you will have to gauge carefully whether to put this document on the table at this meeting. Be sensitive to cues that may signal they are not quite ready for that step and proceed accordingly. Do not leave the room until you have an outcome—the outcome could be that you have either secured an immediate response, a time frame for receiving a response, a request for a more formal proposal or a request for another meeting (which could include a presentation). If the outcome is negative, try to ensure that you understand why they have declined your request, but remain professional at all times. After all, it is their right to decline and they may have very good reasons for doing so. Be available—if they get excited, they may wish to show you something within the organisation so don’t have another meeting scheduled straight afterwards. It is not uncommon for an organisation to want to show you their new building, a new classroom or facility, their new manufacturing plant, or a piece of equipment. Asian managers are particularly gracious in their provision of hospitality, possibly even lunch. If asked, this is a good sign and you should make yourself available. However, realise you are still being evaluated. Follow up—they may ask for additional information that you were not able to provide at the time. If this has been requested, follow up within 24 hours. In doing so, you demonstrate that you can be relied upon and that you are taking everything seriously. Set up the next meeting—if an additional meeting or meetings are required, be proactive by asking with whom the meeting should be arranged. For example, if you are to meet the CEO, you would not necessarily contact the CEO directly but would contact their personal assistant. You may need to act as a quasi-personal assistant in order to arrange a suitable time for everyone, including yourself. Bring in the big guns—it is probably not necessary but, if it is a particularly large organisation and is critical to your studies, you may wish to ask your supervisor to be present at the follow-up meeting.

17.2.5 Undertake Follow-up Meetings For a variety of reasons, the first meeting may not secure access and you will need to go back for another meeting. It could be because they are reticent about your research and require convincing. There may be more people they wish to have

776

17

How Can I Gain Access to Data Sources?

briefed, or you may have targeted the wrong level and authorisation further up the organisation’s hierarchy may be required. There may be cultural barriers, for example, you have not yet established a relationship and further meetings are required before a trusting relationship can be established. For example, as a PhD student, gathering data in Malaysia with professional management organisations, it required at least three or four meetings of relatively informal chatting and drinking of tea before, on the final meeting, unexpectedly, the doors all opened, and it was agreed that I would be able to submit my questionnaire with their next newsletter. I was initially puzzled but, on reflection, it seemed quite natural that time was needed to get to know and trust me before they could progress to the next stage. • Work out who is the decision-maker—find out who can approve the access for your research. If your research involves individuals within a specific group, e.g., teachers in specific subject areas, employees who work in the manufacturing unit or the sales unit, the likely authorisation will not be the CEO but the head teacher, production manager or the marketing manager. If, however, the access you are requesting is organisation-wide and involves a large number of people across the entire organisation, authorisation by the CEO, principal, head nurse or other senior executive may be required. If your research involves significant interventions, e.g., the implementation or modification of strategy, it is possible that the research will require approval from the Board. Never forget that these gatekeeper permissions only give you permission to approach individuals to participate—it does not remove your ethical obligation to obtain informed consent from each individual. No gatekeeper can give permission on behalf of another individual; their permission is merely a signal that you have their approval to take the next step in securing access. • Get support—where approval is required higher up in the organisation, it is always best to secure the support of someone lower down. Invariably, the CEO will talk to one or more lower level managers off-line and ask them if they think that granting access is a good idea. A positive response from the lower level managers will likely see you sail through. If the subordinate manager exhibits some hesitation, you could be in for a rough ride. You, therefore, need to spend time to ensure that the subordinate manager is well and truly on board with your research. • Be realistic about the time required for gaining access to data sources— working with organisations usually requires a number of levels of approval and, consequently, the set-up time can be quite considerable. • Treat each subsequent meeting with the same enthusiasm as you had at the first meeting—consider that, at the next meeting, there may be individuals who were at the first meeting and some of the individuals in the room will have heard your presentation before. This may not make any significant difference to the material you have provided; however, you may wish to incorporate your observations and understandings from the previous meeting. The more knowledge you can project about the organisation, the better.

17.2

Ten Steps for Gaining Access to Data Sources

777

• Be careful that you don’t over-hype your research—be honest about your expectations. • Be very clear on what you expect from the organisation—particularly in regard to the time commitment, resources required (if any, such as a private room for interviewing, etc.) and the number of people involved. Provide a sample of the kinds of questions you will ask in the interview or a copy of the questionnaire. If you are doing participant observation, you will need to make clear what events and areas of the organisation you would like to have access to as well as the time frame in which you intend to do data gathering (remember that the participant observation data gathering strategy typically involves you will being in the organisation, likely on a daily basis, for a lengthy period of time). • There may be a need to negotiate—negotiation involves satisfying the gatekeepers/decision makers on a number of issues. There may be discussion about data confidentiality, clearance for publication, and the extent to which the organisation is likely to benefit from the findings of your study. Negotiation means you need to give something and let some things go in order to obtain a win/win situation. So, remain flexible and open to discussion. Be clear as to what you will and will not negotiate on, and what you would be prepared to give away or provide. For example, in the future, you may wish to publish your research based on data that people in their organisation provide, so you need to secure the right to publish. From experience, the organisation is usually concerned with its reputation, so do not be surprised if there is a requirement for someone in the organisation to review your paper before submitting it for publication. Remember, organisations have the right to set conditions on your access and you have the right to decide whether to accept those conditions (a stay-or-go decision). • Gaining entry to an organisation may require you to provide a brief written proposal—this will vary substantially from the type of proposal you have written for university project approval or funding. It should be written most definitely in layman’s language and after you have had a number of informal discussions with key individuals in the organisation, so that their interests can be incorporated and their concerns addressed (Devers & Frankel, 2000). • Secure an outcome—you are looking for acceptance and some level of proactivity on the part of the organisation. That is, they may wish to assign a person to assist you with the practicalities of setting up appointments etc. Try to avoid vague indications of approval. • Write up as you go—document the access negotiations as you go along, while they are fresh in your mind. Record the process in your journal and keep copies of all documents. This will be important for your methodology chapter. What happens if they turn me down? As one researcher observed, physical access may take weeks or even months to arrange and, in many cases, the time invested will not result in access being granted (Tang, 2008). It is unfortunate but true, a lot of the hard work and emotional investment of contacting an organisation and doing a presentation could amount to

778

17

How Can I Gain Access to Data Sources?

nothing more than a polite rejection with one sentence, “We are sorry we cannot be involved at this time”. While you will be disappointed, think about sales people who do this every day for a living. If they let a rejection seep into their psyche, it would undermine their confidence. They have, however, learnt to push on and be optimistic about the next call. Approach each negotiation quietly confident but maintain realistic expectations. If you are repeatedly receiving rejections you may, however, wish to discuss them with your supervisor(s) as perhaps either your approach or your research needs to be re-scoped. What are the issues when accessing Indigenous people? We mentioned earlier that one method of negotiating access is through a consultative process rather than the more formal corporate letter/email, telephone call and follow-up meeting approach covered above. A consultative approach is more appropriate if seeking access to groups, especially within the Cross-Cultural or Indigenous research frames. When undertaking research with cultural groups, it has been suggested that you should find the appropriate person (or group of people) in those cultural groups to act as an advisor for your research (Tolich & Davidson, 1999). It is also useful to observe the approach taken by others. For example, Maginn (2007) provides information on the process of securing access by describing his experience when trying to negotiate access into three ethnically diverse neighbourhoods in London. He concludes that negotiating access can be a lengthy and complex process as it involves developing relationships and earning the trust of a wide array of informants. Some of the difficulties associated with consultation with Indigenous people are: • • • •

knowing whom to consult; actually gaining access; respecting cultural traditions and knowledge; and the length of time it takes.

For example, in relation to New Zealand’s Indigenous people, Māori behaviour is underpinned by Māori beliefs embodied in Kaupapa Māori which is their philosophy or world view which also entails a code of ethics for maintaining and developing Māori knowledge. Access can be more problematic in these circumstances as it might involve a more rigorous negotiation and entry into the research setting to ensure “a culturally safe environment for all” (e.g., see Cram, 2001; Smith, 2012). In Māori communities, research ethics extends far beyond issues of individual consent and confidentiality and, as a consequence, arranging consent may require group consultation and participation. Consent to participate will usually involve approaching the community and holding a Hui (meeting) to talk about the research. The building of relationships and trust is important and needs to be included in the process. As a result of the interaction, the required resources might change or be modified. It is not unusual for the community to expect a Koha or some form of payment. In this context, it is also about the behaviour of the researcher after access has been given. If there is a feeling that the researcher is there just to collect data, with no ongoing

17.2

Ten Steps for Gaining Access to Data Sources

779

benefit to the community and may not be seen again, the researcher may experience difficulties in gaining further access. Many of the same considerations arise when considering research in other Indigenous communities, for example, in Canada (e.g., see Ball & Janyst, 2008; https://www.ryerson.ca/content/dam/research/documents/ ethics/guidelines-for-research-involving-indigenous-peoples-in-canada.pdf) and in Australia (e.g., see Rigney, 2006; https://aiatsis.gov.au/research/ethical-research/ guidelines-ethical-research-australian-indigenous-studies/negotiation-consultationagreement-and-mutual-understanding). One important thing to realise about research with Indigenous people is that Indigenous views on knowledge and its ownership and on relationships between people, life, land, environment, spirituality and the cosmos are distinctly non-western and must be respected if access is to have a chance of being successfully negotiated. Suggested guidelines for consultation and engagement with Indigenous peoples are: • access appropriate documents, policies, procedures or protocols; • seek advice on how to proceed—consider what form of consultation will best meet your objectives; • communicate about your research—how you communicate and with whom and how will you report back to the participants and those you originally consulted with are all critical considerations; and • secure agreement in a form that can be evidenced to an ethics committee (Unitec Academic Policies and Procedures, A Māori Dimension in Unitec Programmes—an adaptation of the Consultation and Engagement Guidelines with Māori from the New Zealand Ministry of Education, 2007). Indigenous communities are very conscious of what they perceive as ‘deficit research’, that is, research looking at areas where they are deficient or what they are doing wrong by western standards. This is one manifestation of the adverse colonisation impacts that Indigenous communities have experienced over centuries. As a result, these communities are now looking for a more empowering and decolonised model. They want the researcher to provide information that the community wants to know and that will be of benefit to them. They want to know their worldviews and traditions are and will be respected in the research. These have emerged as especially important considerations with Australian Aboriginal communities. Here, researchers must build a relationship of trust, usually with community elders first. Aboriginal communities are increasingly expecting that any research agenda involving their people, their land or artefacts of their culture is jointly negotiated with appropriate respect and acknowledgement of the Aboriginal contribution to the research as well as some kind of return of benefit to the community (Dunbar & Scrimgeour, 2006; Smith, 1997). As a researcher, you may find that you will have to negotiate on matters related to intellectual property (i.e., ownership of the Indigenous knowledge you may gain access to). These are not easy matters to resolve and will certainly involve, at a minimum, your supervisor and perhaps a key Aboriginal connection who can work with you as you negotiate access.

780

17

How Can I Gain Access to Data Sources?

17.2.6 Engaging with Participants There is a difference between physical access and cognitive access. Physical access is where you have been granted access to the organisation—the doors have been opened. However, you still need participants to walk through those doors. Cognitive access is actually getting your participants to put their minds around your research and be willing to take part (Tang, 2008). Once a researcher has passed the relevant gatekeeper(s) and is in the research context, that is not the end of the negotiation ritual. Negotiation is ongoing as you interact with potential participants in your study. Once you have secured agreements, you will need to make contact with active members of the organisation. Again, these individuals usually have supervisors who need to be brought on board. Don’t assume that because the senior management has given approval, there will be open acceptance of your research at lower levels. In fact, there might be resentment of you and your presence being foisted on them by senior management. In addition, you will still need to convince individuals to become participants in your research. You will need to convince them about why they should participate given that they have the option of refusing to join in or to pull out part-way. Remember the bottom line is that each and every individual you approach is effectively their own gatekeeper and has a right to give their informed consent to participate! It may mean you need to go through the presentation or discussion of your research once again, but possibly in a more informal manner in order to get them on your side. Understand any difficulties that participants or their superiors may have and work to resolve their issues. Be flexible, be open and keep in regular communication. What points should I be communicating to participants? • Explain what your research is about and is trying to achieve—this is through your information letter. • Sell the potential value and importance of the project and, where appropriate, point out what the participant might gain (either in the short term or longer term) by participating. • Encourage participation by reinforcing that the individual will be a much-valued participant. • Reassure participants about their rights to confidentiality and anonymity with respect to their identity and any information they may provide. • Remind participants that they have the opportunity to remove themselves from the study for any reason, without explanation.

17.2.7 Implement Your Data Gathering Strategies When gaining access to specific participants where personal contact is being used, it is particularly important to establish trust and rapport. While you may have already

17.2

Ten Steps for Gaining Access to Data Sources

781

secured this with the gatekeepers and the decision-makers, it needs to be undertaken again with the actual participants in the study. You are going to be asking people if you can take up valuable time in their day to interact with you to be interviewed, observed, participate in an experiment or complete a questionnaire. This may be sometimes take up to an hour or more of their time and depending upon your research configuration, may require multiple occasions of participation. If they are asked or wish to look over transcriptions following interviews, that can take a further half an hour to an hour. If you need to interview them more than once or require their participation in a questionnaire or experiment on more than one occasion, be sure to factor this commitment into your thinking. This can be a considerable burden for some individuals and, while you are anxious to obtain their involvement, you also need to be realistic about the commitment they are undertaking. Points to consider here are: • Write a good cover or information letter to invite participation and give appropriate assurances. • In keeping with the ethics requirements associated with social and behavioural research, you will also need to obtain their informed consent, by the signing of an ‘Informed Consent’ form. • Consider the means by which your data gathering is going to be undertaken and information recorded and ensure that participants are fully comfortable with that process. • Remember that you have obligations and responsibilities with respect to the academic integrity of your research, which also extend to the means by which you hold, retain and dispose of the information you have gathered.

17.2.8 Fulfil Your Obligations During the access negotiation phase, you may have promised specific actions as part of the deal to gain access. You must deliver on those promises, which could entail undertaking training sessions for the organisation or providing a specified number of consultancy hours. More commonly, one post-data gathering/analysis obligation is the provision of feedback to gatekeepers, decision-makers and participants. The managers of the organisation will also probably appreciate some preliminary feedback. Even before you have entered or analysed the data, you will be getting some feel for the information. Do not provide feedback until all your data are in, because of the possible risk of biasing future data gathering. However, you may wish to provide a de-brief on your initial observations. More formal feedback can be provided once your data have been analysed. If you have promised to provide them with a summary of your research findings, even though it may be many months or even years after the initial interviews, you are obligated to fulfil this promise. A one-page summary would generally be sufficient. In a recent study that one of the authors participated in, the researchers provided a copy of the first

782

17

How Can I Gain Access to Data Sources?

page of their publication, which included their abstract. This provided information on where to source the material. The abstract was sufficient to fulfil the earlier promise of feedback on the information provided. Always maintain good communication with the organisation as you may find during the analysis stage that you have missed something and need to go back. Going back can be difficult if you have burnt your bridges or annoyed some people. You may also want to remain in academia, and this could be a useful organisation for you to study in the future with another research problem or issue, so considerable care should be taken to keep the door open. You also don’t want to bias the organisation against participating in others’ research, so you want to keep the relationship positive by honouring all of your obligations. Consider also giving key participants a thank you gift. Students are not known for being flush with money but choose something that indicates your appreciation. Give it some thought as to a low-cost but meaningful gift (don’t underestimate a batch of home-made biscuits or a cake) that can be provided to key participants in the organisation who have assisted you in gaining access and providing data.

17.2.9 Reporting Your Research Although you are primarily collecting the information for your postgraduate research, you also need to consider the possibility that you may be writing up your research for publication. As mentioned, this will need to be covered in the agreement you have with the company. They may require that they vet the paper or article before you submit the manuscript and give their approval prior to publication. In recent years, when reporting research information, there has been a call for greater reflexivity in published research as well as a call to “avoid the tendency to present rationalised or sanitised accounts, thereby enabling more rigorous analysis of the impact of the researcher’s role in the field” (Johnson et al., 1999, p. 1234). However, participants are usually most concerned with confidentiality and how their information will be used. Although you may feel that you are repeating yourself at every turn, you need to allay any fears that your participants may have. The same principles apply as discussed in the chapter on academic integrity. When accessing information from participants, you need to ensure that there is prevention of harm and you need to obtain specific authorisation before you can use verbatim quotations noted in the course of your data gathering activities. Participant identities can be safeguarded through the use of pseudonyms as long as you maintain control over who knows the connections between pseudonyms and actual identities; you must flag who has this knowledge to the participant as part of the process for gaining their informed consent. Be aware that some of your participants may also be operating in contextual arenas where you operate. This is highlighted by the concern expressed by an anthropology academic who recalled the slightly alarming experience of having people whose situations she was discussing turning up at an international

17.2

Ten Steps for Gaining Access to Data Sources

783

conference. She questioned why her informants don’t just ‘stay in the field’, but recognised that in a globalised world, the separation between industry and academia is becoming fuzzier and one is no longer left protected by distance or the bastions of academia. This problem becomes even more prominent in professional doctorate research, where you may need to maintain longer-term connections with people, maybe even colleagues, who also participated in your research.

17.3

Staying in Contact

If your school or faculty hosts corporate lunches, breakfasts or similar events, make sure that the key contacts in your study are on the database for those invitations or for any newsletters emanating from the faculty or school. They should now be considered a ‘friend’ and need to be cultivated. They could provide further benefits to your own university in the form of student intern placements that have nothing to do with your research, so be generous with your contacts. The more contacts an organisation has with a university, the stronger the relationship and the easier it will be for the next researcher to negotiate access.

17.4

Conclusion

It has been observed that the existing methodological literature tends to over-simplify or downplay the data source access process. Okumus et al. (2007) recommended, particularly for qualitative studies, that researchers should be better trained in dealing with the complexities of facilitating and maintaining access to large organisations. Matthiesen and Richter (2007) concur, observing that gaining and maintaining access to such participants can be challenging, especially for postgraduates and early career researchers, and yet, they are provided with little guidance. They argue that researchers undertaking qualitative or exploratory projects require an even greater skill set when it comes to negotiating access. While the unit of analysis could be individuals or groups, they are usually housed within organisations, whether they be commercial organisations, government organisations, educational and other public and private service organisations, religious organisations, not for profits or professional associations. As a consequence, the issues in relation to negotiating access more commonly relate to approaching an organisation and obtaining approval to further access individual members of that organisation. If, before you approach an organisation, you have given thought during your planning stage as to where the information, either primary or secondary, may be located, this can pay off with some creative solutions emerging. For research done outside of organisations, such as community-based

784

17

How Can I Gain Access to Data Sources?

research (e.g., shoppers, users of recreational areas and other public facilities), there may be no higher-level gatekeeper beyond the individual. For research with Indigenous people, the gatekeepers may be the community itself or perhaps tribal elders, in consultation with the community. Discuss your data source requirements with others who may be able to suggest suitable organisations or contacts. A key person to discuss potential sources of data with is your supervisor, who may have conducted prior research in the field and, as a consequence, has developed contacts which you could benefit from. Don’t wait for an organisation or individuals to come to you, it won’t happen; you need to take the initiative to identify, approach and secure the organisations and individuals in which you wish to research. Remain flexible. While you may have some general opinions and thoughts on potential data sources, it is important that you remain open and flexible for a variety of reasons. The first is that a specific data source may, in fact, be inappropriate for your research purposes. Second, a data source may decline involvement in your project and, third, a data source may be considered inappropriate for ethical reasons. On a positive note, sometimes you will find an opportunity that opens up access to a more productive data source than you had initially anticipated. Use your contacts and pursue contacts from others. In addition to being flexible, you will also need to develop resilience as new demands emerge. Being resilient and persistent will come in handy when a key contact is eluding you, a door shuts on you, or access is denied mid-way through your data gathering. Yes, these all happen to postgraduates (and academic researchers as well) and, if you are to resurrect your project from such a situation and end up with meaningful data, you need to have a strong problem-solving orientation. Positive self-talk and visualisation are two very effective ways of achieving positive thinking (Cartwright, Collins, Green, & Candy, 1998, p. 53). Don’t obsess on what might go wrong (however, some anticipation never hurts) or did go wrong, concentrate on looking forward and working smoothly through the process,. Read some accounts on successful access. There is a good one in Linda Valli’s Becoming clerical workers (1986) and another in G. A. Fine’s Shared Fantasy (2002). For other supporting texts, you might find the following to be of use. For a general discussion of data gathering issues, check out Sapsford and Jupp (2006). For a discussion about gaining access for qualitative and field researchers, see Feldman, Bell and Berger (2003), Flick (2014, Chap. 12) and Yin (2011, Chap. 5). For a text related to access for case studies, see Marschan-Piekkari and Welch (2011). In closing, keep a constant eye on the important dimensions of your study related to your plan for accessing a representative/suitable sample, having high levels of research integrity, minimising response rate errors and response bias errors. If it seems as if one or more of these dimensions are being substantively compromised, you may want to reassess whether you should continue to be involved with that particular organisation.

17.5

17.5

Key Recommendations

785

Key Recommendations

• Data source access considerations can involve both primary and secondary data and human as well as non-human sources. • The data gathering strategies you use and the data sources that will provide the information you seek are intricately connected with the research questions you are asking. Ask yourself who may have the answers to your questions or views on your problem and how difficult are they to get access to? Weigh up the costs versus the benefits of obtaining this information. • In some circumstances, you may be using a pluralist approach with multiple data gathering strategies and multiple types of data sources. For instance, you may be searching a company database as well as conducting focus groups, in-depth interviews and sending out questionnaires. How feasible is this, are you clear on which types of data sources will be best suited to provide specific kinds of data, and will the additional data access demands actually assist in addressing your research question(s)? • You need to consider issues of data integrity and quality, such as are you measuring what you intend to measure, will the data provide consistent results if the study were to be replicated by another researcher, will you achieve a biased or selective set of perspectives, will you get access to genuine or authentic perspectives, have you secured appropriate permissions? In short, can you secure high-quality information from the data sources you gain access to? • Find out who the gatekeepers are in the organisations you intend to enlist. HR managers are quite useful as they tend to have knowledge of the entire organisation and the relevant individuals to contact within each department. For research with Indigenous people, gaining access will largely involve a more intensive consultative process and there are a range of additional issues, including reciprocity and benefit, safeguarding of knowledge, respect for cultural traditions, that you must address before access might be granted. • In your introductory letter/email, introduce yourself and provide your credentials. Include a background sketch of the research focusing on the problem or issue you are investigating and indicate what you are looking for. Mention that you will follow up to arrange the possibility of a future meeting. The letter/email should only be one page in length with no dense paragraphs. Have somebody else look over the correspondence to ensure it doesn’t contain any errors and that it can be readily understood by someone who is not knowledgeable about your research. • Realize that there might be a continuum of receptivity with respect to data sources. For example, an organisation may be happy for you to troll through published material such as their financial statements but be less enthusiastic about you having access to their internal documentation and email communications, and even less receptive to you interviewing their staff.

786

17

How Can I Gain Access to Data Sources?

• Identify benefits that might accrue to the other party. Carefully consider what you promise and ensure that you deliver. • All participants must be provided with information on the study and sign an informed consent form. Intriguingly, most people prefer to be told about a study rather than having to read about it, so be prepared to discuss your project with potential participants before they sign the form. • There is a current trend for taking a mixed method or, more appropriately as we have argued elsewhere in this book, a pluralist approach to postgraduate research, but it typically involves a considerable amount of additional effort and you will likely have to access more diverse types of data sources. In such cases, you need to plan extra time to work through all of the access processes. • If you have serious doubts about the suitability of an organisation or the individuals you have accessed, pull out before you invest any further time or energy. Make sure you record your processes and your reasoning in your research journal as well as any observations on how this decision might impact on your data integrity and quality.

References Anderson, D. G., & Hatton, D. C. (2000). Accessing vulnerable populations for research. Western Journal of Nursing, 22(2), 244–251. Atkinson, R., & Flint, J. (2001). Accessing hidden and hard to reach populations: Snowball research strategies. Social Research Update, Summer 33. Department of Sociology, University of Surrey, Guildford UK. Retrieved January 19, 2019, from http://citizenresearchnetwork. pbworks.com/f/accessing%2Bhard%2Bto%2Breach%2Bpopulations%2Bfor%2Bresearch.doc. Ball, J., & Janyst, P. (2008). Enacting research ethics in partnerships with indigenous communities in Canada: “Do it in a good way”. Journal of Empirical Research on Human Research Ethics, 3(2), 33–51. Best, S. J., & Krueger, B. S. (2004). Internet data collection. Thousand Oaks, CA: Sage Publications. Broadhead, R. S., & Rist, R. C. (1976). Gatekeepers and the social control of social research. Social Problems, 23(3), 325–336. Buchanan, D., Boddy, D., & McCalman, J. (1988). Getting in, getting on, getting out and getting back. In A. Bryman (Ed.), Doing research in organisations (pp. 63–77). London: Routledge. Buehler, M. (2013). Demystifying the institutional repository for success. Amsterdam: Chandos Publishing. Burgess, R. (1984). In the field. London: Allen & Unwin. Cambridge University Press. (2015). Data access and research transparency (DA-RT): A joint statement by political science journal editors. Political Science Research and Methods, 3(3), 421. Cartwright, R., Collins, M., Green, G., & Candy, A. (1998). Managing yourself: A competence approach to supervisory management (2nd ed.). London: Wiley-Blackwell. Chowdhury, G. G., & Foo, S. (2012). Digital libraries and information access: Research perspectives. London: Facet Publishing. Corti, L., Van den Eynden, V., Bishop, L., & Woollard, M. (2014). Managing and sharing research data: A guide to good practice. Los Angeles: Sage Publications.

References

787

Cram, F. (2001). Rangahau Māori : Tona tika, tona pono—The validity and integrity of Māori research. In M. Tolich (Ed.), Research ethics in Aotearoa New Zealand: Concepts, practice, critique (pp. 5–52). Auckland, NZ: Longman. Delamont, S., Atkinson, P., & Parry, O. (2000). The doctoral experience success and failure in graduate school. London: Falmer Press. Denscombe, M. (2017). The good research guide: For small-scale social research projects (6th ed.). London: Open University Press. Devers, K. J., & Frankel, R. M. (2000). Study design in qualitative research—(part 2): Sampling and data collection strategies. Education for Health, 13(2), 263–271. Dunbar, T., & Scrimgeour, M. (2006). Ethics in indigenous research—Connecting with community. Journal of Bioethical Inquriy, 3(3), 179–185. Evans, B. C., Coon, D. W., & Crogan, N. L. (2007). Personalismo and breaking barriers: Accessing Hispanics populations for clinical services and research. Geriatric Nursing, 28(5), 289–296. Feijen, M., Horstmann, W., Mangho, P., Robinson, M., & Russell, R. (2007). Driver: Building the network for accessing digital repositories across Europe. Ariadne, 53. Retrieved January 19, 2019, from http://www.ariadne.ac.uk/issue/53/feijen-et-al/. Feldman, M. S., Bell, J., & Berger, M. Y. (2003). Gaining access: A practical and theoretical guide for qualitative researchers. Walnut Creek, CA: Altamira Press. Fielding, N. G., Lee, R. M., & Blank, G. (2008). The Sage handbook of online research methods. Los Angeles: Sage Publications. Fienberg, S. E., Martin, M. E., & Straf, M. L. (1985). Sharing research data. Washington, DC: National Academies. Fine, G. A. (2002). Shared fantasy: Role playing games as social worlds (2nd ed.). Chicago: Chicago University Press. Flick, U. (2014). An introduction to qualitative research (5th ed.). Los Angeles: Sage Publications. Halfpenny, P., & Procter, R. (2015). Innovations in digital research methods. Los Angeles: Sage Publications. Hammersley, M., & Atkinson, P. (2007). Ethnography: Principles and practice (3rd ed.). London: Routledge. Harron, K., Goldstein, H., & Dibben, C. (Eds.). (2015). Methodological developments in data linkage. Chichester, UK: Wiley. Hayes, D. (2005). Gaining access to data sources in statutory social work agencies: The long and winding road. British Journal of Social Work, 35(7), 1193–1202. Johnson, P., Duberley, J., Close, P., & Cassell, C. (1999). Negotiating field roles in manufacturing management research: The need for reflexivity. International Journal of Operations & Production Management, 19(12), 1234–1253. King, D. B., O’Rourke, N., & DeLongis, A. (2014). Social media recruitment and online data collection: A beginner’s guide and best practices for accessing low-prevalence and hard-to-reach populations. Canadian Psychology/Psychologie Canadienne, 55(4), 240–249. Lane, J., Stodden, V., Bender, S., & Nissenbaum, H. (Eds.). (2014). Privacy, big data, and the public good: Frameworks for engagement. New York: Cambridge University Press. Larsen, M., & Walby, K. (2012). Brokering access: Power, politics and freedom of information process in Canada. Vancouver, BC, Canada: University of British Columbia Press. Maginn, P. J. (2007). Negotiating and securing access: Reflections from a study into urban regeneration and community participation in ethnically diverse neighborhoods in London, England. Field Methods, 19(4), 425–440. Marschan-Piekkari, R., & Welch, C. (2011). Rethinking the case study in international business and management research. Cheltenham, UK: Edward Elgar Publishing. Matthiesen, J. K., & Richter, A. W. (2007). Negotiating access: Foot in the door or door in the face? Psychologist, 20(3), 144–147.

788

17

How Can I Gain Access to Data Sources?

Mauthner, M. (2006). Methodological aspects of collecting data from children: Lessons from three research projects. Children and Society, 11(1), 16–28. Mochmann, E., de Guchteneire, P., & Guchteneire, P. F. A. (1990). Data protection and data access: Reports from ten countries on data protection and data access in social research, with an annotated international bibliography. International Federation of Data Organizations for the Social Sciences, Sociaal-Wetenschappelijk Informatie- en Documentatiecentrum (Koninklijke Nederlandse Akademie van Wetenschappen. Amsterdam: North-Holland [for] SWIDOC/ IFDO. NaCTeM. (2016). The National Centre for Text Mining. University of Manchester. Retrieved November 9, 2018, from http://www.nactem.ac.uk/. Norcera, J. L. A. (2002). Ethnography and hermeneutics in cyber cultural research accessing virtual communities. Journal of Computer-Mediated Communication, 7(2) JCNC721. Retrieved January 19, 2019, from https://doi.org/10.1111/j.1083-6101.2002.tb00146.x. Okumus, F., Altinay, L., & Roper, A. (2007). Gaining access for research: Reflections from experience. Annals of Tourism Research, 34(1), 7–26. OSSWATCH. (2014). Open Source Options for Education. Retrieved November 9, 2018, from http://oss-watch.ac.uk/. Polonsky, J. M., & Waller, S. D. (2015). Designing and managing a research project: A business student’s guide (3rd ed.). Los Angeles: Sage Publications. Riesch, S. K., Tosi, C. B., & Thurston, C. (2007). Accessing young adolescents and their families. Journal of Nursing Scholarship, 31(4), 323–326. Rigney, L. I. (2006). Indigenist research and aboriginal Australia. In J. Kunnie & N. I. Goduka (Eds.), Indigenous peoples’ wisdom and power: Affirming our knowledge through narratives (pp. 61–77). Aldershot, UK: Ashgate Publishing. Rossman, G. B., & Rallis, S. F. (1998). Entering the field. In G. B. Rossman & S. F. Rallis (Eds.), Learning in the field: An introduction to qualitative research. London: Sage Publications. Sapsford, R., & Jupp, V. (2006). Data collection and analysis. Los Angeles: Sage Publications. Scholarly Editions. (2013). Algorithms—Advances in research and application (2013th ed.). Atlanta: Scholarly Editions. Sheehan, K. B. (2018). Crowdsourcing research: Data collection with Amazon’s Mechanical Turk. Communication Monographs, 85(1), 140–156. Smith, A. (1997). Indigenous research ethics: Policy, protocol and practice. The Australian Journal of Indigenous Education, 25(1), 23–29. Smith, L. T. (2012). Decolonising methodologies, research and indigenous peoples. Dunedin, NZ: University of Otago Press. Sumathi, S., & Sivanandam, S. N. (2006). Introduction to data mining and its applications. Berlin: Springer Science & Business Media. Tang, W. (2008). Chapter 1: Research methods for business students. Retrieved November 7, 2018, from http://site.iugaza.edu.ps/walhabil/files/2010/02/Chapter_1.pdf. Thomas, M. R., & Brubaker, D. L. (2007). Theses and dissertations: A guide to planning, research, and writing (2nd ed.). Thousand Oaks, CA: Corwin Press. Tolich, M., & Davidson, C. (1999). Starting fieldwork: Introduction to qualitative research in New Zealand. Melbourne, Australia: Oxford University Press. UK Data Archive. (2018). Retrieved November 9, 2018, from http://data-archive.ac.uk/. UK Data Service. (2018). ESRC Economic & Social Research Council. Retrieved November 9, 2018, from https://www.ukdataservice.ac.uk/. Valli, L. (1986). Becoming clerical workers (critical social thought). New York: Routledge. Wanat, C. L. (2008). Getting past the gatekeepers: Differences between access and cooperation in public school research. Field Methods, 20(2), 191–208. Yin, R. K. (2011). Qualitative research from start to finish. New York: The Guilford Press.

Chapter 18

When and How Should I Deal with Measurements?

Measurement is a pivotal stage in the design and execution of any research investigation that purports to collect quantitative data, especially, but not exclusively, in the Survey, Evaluation and Explanatory research frames. Measurement, as a process, is primarily associated with research guided by the positivist pattern of guiding assumptions, although research conducted under other patterns of guiding assumptions (e.g., critical realist) may also employ measurement processes to help round out their stories. Virtually any of the experience-focused and data-shaping strategies, as well as structured interviews, systematic observation and archival/ secondary strategies, will require you to consider measurement issues. If your research involves the gathering (or generation) and analysis of any sort of quantitative data, then you need to be concerned with measurement issues and processes. Part of the story you will have to tell relates to the nature and quality of the measures you employ. Good measures will facilitate analyses and their meaningful interpretation; poor measures will have an adverse effect on analyses, leading to indefensible interpretations. It is important to realise that no amount of sophisticated data analysis will save an investigation that uses poor measurement processes to obtain the quantitative data being analysed. Measurement concerns the processes for assigning numbers to reflect some phenomenon of interest. There are four central considerations that are important with respect to measurement: scale, validity, reliability and sensitivity. Measurement scale concerns the meaning of the actual numbers you use to quantify some phenomenon of interest. Measurement validity concerns what can properly be inferred from the numbers used to quantify some phenomenon of interest. Measurement reliability concerns how consistently the assigned numbers reflect the phenomenon of interest and is closely related to the amount of error associated with the assignment of numbers to reflect that phenomenon. Measurement sensitivity refers to the ability of your measurement scale to reflect the magnitude of changes or differentiation you are looking for. If you are gathering quantitative data under an interpretivist/constructivist guiding assumptions, then measurement scale becomes © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_18

789

790

18 When and How Should I Deal with Measurements?

a primary concern. Here, the role of quantitative data will typically be to characterise relevant attributes of participants and/or analytical codes for comparative purposes.

18.1

The Process of Measurement

Measurement, quite simply, is the systematic process of using a number to represent some observable phenomenon of interest with respect to a specific characteristic or property (Athanasou, 1997). In the physical world of tangible objects and events, measurement processes are easy to conceptualise (this is a major focus of physical/ physiological/economic measurements within the Measurement data-shaping strategy). For example, physicists use one system of units (divided into grams, kilograms etc.) to measure the mass of objects, another system of units (divided into meters, kilometres, etc.) to measure the linear dimensions of objects and distances between objects and yet another system of units to (divided into degrees) to measure temperature. Such physical measurements will always have some error associated with them linked to the quality and fidelity of the measuring instrument and the conditions under which the measurement is obtained. However, these measurement errors will usually be relatively small and random, in large part because a physical comparative reference standard can be defined. For example, the standard for a distance measurement of exactly 1 m is currently defined as “the length of the path travelled by light in vacuum during the time interval of 1/ 299,792,458 of a second” (National Institute of Standards and Technology (NIST), 2001, see https://www.nist.gov/pml/weights-and-measures/si-units-length). Some characteristics of people do have physical manifestations (e.g., age, gender, hair or eye colour, weight, height, colouration, how long it takes to complete a task) and it is relatively simple to measure these characteristics. The errors associated with such measurements would typically be small, but they would not be zero, since they involve some degree of human judgment and fuzziness in defining category membership, depending upon the measurement method employed (e.g., self-rating, rating by an observer). Physiological measurements are usually associated with some sort of instrumentation and their accuracy and reliability depends heavily upon how well-calibrated the instrument is. Economic measurements attach dollar values to objects and events in the world and their accuracy and reliability generally depend upon consensus agreement within some defined economic entity (e.g., a nation, a market, an industry). Many types of measurements in the social and behavioural sciences vary in quality and degree of error depending upon the individual generating the measurements. For mail and internet questionnaires, for example, measures are completed by participants themselves (a major focus of self-report questionnaires within the Measurement data-shaping strategy). Such self-report measures are susceptible to a variety of subjective influences, which contribute to measurement errors:

18.1

The Process of Measurement

791

• Social desirability—where participants make responses that will present them in what they think is a ‘good light’ with respect to how society might perceive them. This bias is more likely to occur with questions of a sensitive nature or questions that might require the participant to reveal aspects of themselves or their behaviour that might be considered illegal, immoral, unsafe or objectionable to the society they live in. Some people have a greater tendency to fall prey to this bias than others. This bias can be handled by designing more indirect questions or by explicitly measuring social desirability (using another kind of self-report instrument) and then statistically adjusting responses for the tendency. • Acquiescence bias—where participants tend to agree with every statement put to them, even when they don’t actually agree. Here, the participant is trying to appear cooperative and give you what (they perceive) you want. This bias can be handled by varying the directionality of the wording in items to require participants to use the entire range of the scale. • Response pattern biases—where participants respond according to some pattern facilitated by the design of the instrument. Such patterns may include extreme responses (only ticking the end points of any scale because they may tend to think of the world in ‘black and white’ terms) and ‘middle-of-the-road’ responses (only ticking ‘neutral’ for attitude questions, because they may not wish to reveal their true position). These biases can be handled through clearer instructions to participants and through better scale design to discourage improper use of the ‘neutral’ category. For face-to-face or telephone interviews or observation studies (using the structured interviews or systematic observation data gathering strategies), measures are completed by an interviewer/observer with respect to some focal object or person(s) of interest. Such measures can avoid many of the pitfalls of self-report measures, but at the cost of potentially reflecting interviewer or observer judgment biases, which again add to measurement error: • ‘Halo’ or ‘Rusty Halo’ effects—where the interviewer/observer forms a generally favourable or unfavourable impression of the participant and their ratings are then subtly influenced by this general impression. This bias can be handled by better wording of questions to focus on concrete and observable behaviours rather than feelings or judgments about those behaviours. • Questioning biases—where, in interviews, the interviewer asks questions inconsistently by varying wording or intonation to better target their perceived impression of the participant. This is handled through carefully scripting and training of interviewers. • Data recording biases—where the interviewer/observer uses a measurement scale incorrectly or where too much latitude in interviewer/observer judgment is allowed in determining measurements. This bias can be handled by better measurement design coupled with more thorough interviewer/observer training. What complicates things for researchers is that many of the characteristics of people that we seek to measure are not directly observable; we must infer their

792

18 When and How Should I Deal with Measurements?

existence or strength on the basis of indirect measurement processes. For example, we might be interested in measuring the organisational commitment of an employee. We cannot directly observe or make a instrument reading of their level of commitment, we may only infer it indirectly based on other observable behaviours such as whether the person is applying for other jobs, how often they take sick days and so on, that we think reflect commitment. When we are talking about intellectual or cognitive capabilities, attitudes, values, beliefs, emotions or perceptions, the same problem arises: how do we defensibly measure something, especially something that we cannot directly observe? The logical follow-on question then becomes: how do we know we have constructed a good measurement process for a construct? This relates directly to the central problem associated with measurement in the human, social and physical worlds: what are we justified in inferring based on the measurements we obtain? This is the question of validity. For example, it is one thing to objectively measure how long someone takes to complete a task (using a time scale in seconds or minutes) and another rather more complicated thing entirely to infer whether the person is proficient in the task or requires additional training, based on that resulting time measurement. This highlights the central problem in any measurement process linked to human behaviour: we must take care to make defensible inferences from our measurements to the characteristics or phenomenon we are interested in—this links measurement directly to processes of human judgment (in our case, you, as the researcher).

18.2

Operational Definition of a Construct

The process of building a defensible measurement system, especially for something that we cannot directly observe, commences with a clear identification of the construct to be measured. A construct is basically a concept or property theorised to describe specific differences between people. A theory places a construct (sometimes referred to as a hypothetical construct) in a larger context by showing how it is hypothesised to relate to other constructs of interest. Testing a theory therefore requires, as a first step, the design of a measurement system for every construct embedded within it. The process of translating a construct into a measurement system is called operational definition. Technically, an operational definition is defined as “spelling out, very clearly, the steps the researcher will go through to obtain concrete measures of that construct” (Cooksey, 2014, p. 18). Operational definition describes how observations of manifestations of the construct should be translated into numerical values. Figure 18.1 shows the operational definition process in action. Here, we have the theoretical construct of organisational commitment informed by three theoretical sub-constructs (Affective, Normative and Continuance) where each sub-construct is operationally defined by writing three items for each on a self-report inventory. Allen and Meyer (1990) went through this process when they created their well-known three-component measure of organisational commitment, only they generated more than three items

18.2

Operational Definition of a Construct

793 All items measured using a Likert-type interval scale: 1 = strongly disagree 7 = strongly agree Item NC2 Item NC1

Normative Commitment (NC) Affective Commitment (AC)

Item NC…

Item AC2 Item AC1

Continuance Commitment (CC)

Organisational Commitment (OC)

Theoretical construct & sub-constructs

Operational Definition

Define domains and write items to target sub-constructs of OC construct

Item CC2

Item AC…

NC

Item CC1

AC

Item CC…

CC

OC Inventory

Measurement domains

Fig. 18.1 Illustrating the logical process of operational definition flowing from general specification of the construct to the writing of specific items in the Allen and Meyer (1990) development of an organisational commitment self-report instrument

per sub-domain. Examples of items include: “I would be very happy to spend the rest of my career with this organization” (Affective item), “I do not feel any obligation to remain with my current employer” (Normative item) and “I feel that I have too few options to consider leaving this organization” (Continuance item). Note that each item tries to tap an aspect of a specific organisational commitment sub-construct by querying a specific behaviour thought to reflect that sub-construct. The actual measurement is the number ticked or circled on a response scale, ranging from 1 (strongly disagree) to 7 (strongly agree), linked to each item.

18.3

Measurement Scales

There are four scales of measurement that can be used to quantify phenomena in the social and behavioural sciences. The scales of measurement are ordered such that each scale subsumes and extends the properties of the scales that precede it. The scales are described below in order of increasing complexity and amount of information represented.

18.3.1 The Nominal Scale The nominal scale assigns numbers to serve simply as names for clearly defined, mutually exclusive and exhaustive categories of a phenomenon (for this reason, it is

794

18 When and How Should I Deal with Measurements?

sometimes considered to be effectively qualitative in nature, serving simply to name categories). For example: • demographic characteristics such as sex (e.g., 1 = female; 2 = male; 3 = non-binary, not included in the traditional binary male–female distinction); • geographical area where one lives (e.g., 1 = metropolitan; 2 = regional; 3 = rural/remote); • marital status (e.g., 1 = currently single; 2 = currently married/partnered; 3 = previously married/partnered, now single; 4 = other); • religious affiliation (e.g., 1 = Christian; 2 = Islamic; 3 = Jewish; 4 = Other; 5 = none); • ethnic background (e.g., 1 = Australian; 2 = Australian Aboriginal; 3 = Australian South Sea Islander; 4 = Torres Strait Islander; 5 = other). The numbers assigned reflect nothing more that the category itself; the magnitude and ordering of the numbers or the differences between successive numbers have no meaning. If you are gathering quantitative data under an interpretivist/constructivist pattern of guiding assumptions, then you will likely make use of the nominal measurement scale for counting occurrences of codes and themes and for classifying certain characteristics and attributes of participants (such as sex, ethnic or religious background, occupation).

18.3.2 The Ordinal Scale The ordinal scale assigns numbers to serve not only as names for mutually exclusive (and possibly exhaustive) categories, but also to reflect a rank ordering amongst the categories. Objects, people and events may be ranked using an ordinal scale. The ordering of the numbers assigned reflects the extent to which a phenomenon or an object or a person possesses some characteristic or property of interest. Examples include: • highest educational level attained (e.g., 1 = completed primary school; 2 = completed secondary school: 3 = completed tertiary diploma; 4 = completed bachelor’s degree; 5 = completed postgraduate qualification); • age (1 = under 25; 2 = 26–35; 3 = 36–55; 4 = over 55); • income (e.g., 1
$25,000/year; 2 = $26,000–50,000/year; 3 = $51,000–100,000; and 4  $100,000/year). While ordinal scales reflect rankings, the differences between successive numbers have no meaning. If you are gathering quantitative data under an interpretivist/ constructivist pattern of guiding assumptions, then you will likely make some use of ordinal scales for classifying certain characteristics and attributes of participants (such as age, job experience, position in an organisational or institutional hierarchy or highest education level attained). In such cases, you may prefer to use linguistic qualifiers to convey the ordinal positioning, rather than explicitly assigning a

18.3

Measurement Scales

795

number (for example, you may describe a person’s job role as ‘significantly expanded’ relative to what it was before they were promoted, and you would have to then describe how and why you made this judgment).

18.3.3 The Interval Scale The interval scale assigns numbers to serve not only as names for mutually exclusive (and possibly exhaustive) categories, but also to reflect ordinal positioning amongst those categories as well as incremental amounts of difference between adjacent numbers (that is, the spacing between numbers is significant and assumed to be at least approximately equal). However, ratio comparisons between different measurements along the scale cannot be sensibly made (e.g., 20 °C is not twice as hot as 10 °C, but the difference between 20° and 21° is the same amount of temperature change as between 10° and 11°). The measurement of cognitive constructs such as intelligence, job performance, learning, ability and achievement are typically considered to be interval scales (e.g., scores on a classroom test might be scaled from 0 to 100%). The measurement of other psychological constructs such as attitudes, values, sentiments, perceived amounts, importance and preferences are often treated (debatably) as quasi-interval rather than ordinal in nature (e.g., Rate how satisfied you are with your current job: 1 = strongly disagree, 2 = agree, 3 = neither agree nor disagree, 4 = agree; 5 = strongly agree). It is important to realise that this latter type of scale (called a Likert-type scale), while very commonly used in social and behavioural research, is often only assumed to have interval properties. This means that we must be willing to accept the assumption that the difference in job satisfaction between 1 and 2 is approximately the same amount of job satisfaction difference as between 4 and 5). Some researchers dispute this assumption, based on good empirical evidence, and argue that such scales should be treated as ordinal scales only. However, you will find that the norm in the literature seems to be that Likert-type attitude items are analysed as if they were interval scales. Interval scales do not have a rational or conceptually defensible zero-point. For example, there is no way to say, using the job satisfaction scale defined above, that a person has zero amount of job satisfaction (a ‘neutral’ rating might mean this, but it might mean other things as well, such as total indifference, unwillingness to reveal a position or don’t understand the question). Thus, relative comparisons between different points along an interval scale will make no conceptual sense. For example, a person who rated an item as 4 (‘agree’ on the above scale) could not be said to be twice as satisfied with their job as a person who rated the item as 2 (‘disagree’). A student who scored 90% on a classroom test could not be said to know twice as much as a student who scored 45%. If you are gathering quantitative data under an interpretivist/constructivist pattern of guiding assumptions, then you may make some use of interval scales for summarising certain characteristics and attributes of participants, perhaps as a

796

18 When and How Should I Deal with Measurements?

consequence of administering a short inventory (such as characterising a participant’s values or political orientation) or for making certain types of ratings based on what is being reflected in a document or transcript.

18.3.4 The Ratio Scale The ratio scale is the highest and most information-rich scale of measurement possible. It assigns numbers to not only reflect ordinal positioning but also incremental amounts of difference between the phenomenon, objects or people being measured relative to a definable zero-point. This means that ratio comparisons between different measurements along the scale can be sensibly made (e.g., one object has twice the mass of another object or is half as long as another object). The ratio scale includes counts (e.g., the number of times a phenomenon has been observed) and elapsed time measurements as well as measurements of physical properties of objects (such as length, strength, force and mass). For example, time taken to complete a task can be conceptualised as a ratio-scale measure. Time could be measured in elapsed seconds from the task starting point, for example. We live in a universe where it is possible to conceive of zero seconds as meaning no elapsed time. Time thus has a rational and defensible zero-point (as does a measure of mass in the absence of a gravity field). The existence of a rational zero-point means that relative comparisons between difference measurements along the scale make sense. For example, it would be perfectly sensible to infer that a person taking 50 s to complete a task took twice as long as a person who took only 25 s. For economic measurements, ratio scales are often represented in dollar-equivalent amounts. Examples include: Gross Domestic Product (GDP) per capita index measuring economic productivity of a nation, crime rate, pulse rate, cholesterol level, age expressed in actual years, income expressed as actual dollars earned per year, percentage of votes received by a political party in an election. Examples of ratio scales are relatively rare in the social and behavioural sciences, mostly because of the indirect nature of many of the inferences we make about constructs. It is possible to construct quasi-ratio scales through careful design of questionnaire items (Fig. 18.2 shows such an example in item 9). If you are gathering quantitative data under an interpretivist/constructivist pattern of guiding assumptions, then you may make some occasional use of ratio scales for summarising certain characteristics and attributes of participants (such as actual years of experience in a job or position, the size of the department a participant works in or the size of the participant’s family). Sometimes the choice of measurement scale to employ is relatively easy because the nature of the characteristic is such that the scale of measurement is directly invoked. For example, gender is generally considered to be an inherently nominal scale measure, possessing two categories: ‘male’ and ‘female’ (note, however, that in today’s world, gender is a more fluid and complex construct and may no longer be susceptible to such a simplistic binary categorisation, depending upon whether

18.3

Measurement Scales

797

1.

Nominal: Categorical scale:

What is your gender?

2.

Ordinal: Ranking scale:

Rank these outcomes from life from most important (1) to least important (5) to you:

Male

Having a good job Having a good family life Having good friends Having material possessions Having control over my life

Female

______ ______ ______ ______ ______

3.

Ordinal: Graphic rating scale:

I like my supervisor:

4.

Interval: Bipolar Likert-type scale:

I want to participate in making decisions for my company.

[No neutral point offered]

5.

Interval: Bipolar Likert-type scale: [Neutral point offered]

Strongly Disagree 1

7.

Interval: Unipolar Likert-type scale:

Disagree -1

Interval: Graphic rating scale:

9.

Ratio: Points distribution scale:

Moderately Agree 5

Strongly Agree 6

Unable to Rate 9

Neither Agree Nor Disagree Agree 0 1

Strongly Agree 2

Unable to Rate 9

How important is it for you to have colleagues that you work well with? Not at all Important 0

Slightly Important 1

Warm Fresh Comfortable Friendly Dark

___:___:___:___:___:___ ___:___:___:___:___:___ ___:___:___:___:___:___ ___:___:___:___:___:___ ___:___:___:___:___:___

Interval: Semantic differential:

8.

Slightly Slightly Disagree Agree 3 4

Most of my colleagues in the department feel uncomfortable about making suggestions regarding work improvements to our department manager. Strongly Disagree -2

6.

Moderately Disagree 2

Moderately Important 2

Very Important 3

Extremely Important 4

My Work Place

How do you feel about your supervisor?

Cold Musty Uncomfortable Unfriendly Light

Dislike |------------------------------------| Like

Assume you have 100 points and distribute them among the five alternatives below in a way that reflects how important each is to you when you consider buying a new car. Use numbers in such a way that it would make sense to say one thing is x times more important than another. For example, something you give 40 points would be twice as important to you as something you give 20 points. Remember your total must equal 100: Fuel Economy _____

Colour _____

Style _____

Size _____ Environmental Impact _____

Fig. 18.2 Illustration of some different ways to implement various types of measurement scales

you focus on physical manifestations of gender or gender identity). Other measures typically considered to be inherently categorical include ethnic background, country of birth, occupation and religious affiliation. For other types of characteristics, the choice of measurement scale needs to be thought through very carefully (as would the construct of ‘gender’ given what we said above). For example, measuring someone’s age using a self-report questionnaire item could be measured using a ratio scale, where the participant would write their exact age as a response to the question, or an ordinal scale, where the participant would choose a category representing a range of ages (e.g., 20–34 years old, 35–50 years old, more than 50 years old). Which scale you would choose depends less upon absolute measurement requirements and more upon how you think participants might react to the question; many people would not like having to write their exact age down, they would prefer instead to indicate a categorical band. Asking about someone’s annual income would likely invoke the same considerations. Figure 18.2 illustrates some typical measurement scales and item styles employed in social and behavioural research. If you scan through the list of examples, you will see both numerical (items 2, 4–6 and 9) and graphical (items 1, 3, 7 and 8) scales illustrated. Graphical scales are generally simpler for participants

798

18 When and How Should I Deal with Measurements?

to cognitively deal with (ticking a box or line as for items 1 and 7; circling a face as for item 3 or placing a mark on a line as for item 8) and may be more suitable for certain types of participants. Numerical rating scales range from relatively simple as for items 4–6, where a number just needs to be circled, to more demanding, where participants must generate numbers according to some rule or principle (as for items 2 and 9). It might seem like we are stating the obvious here, but it draws attention to the need to carefully consider not only your measurement requirements, in terms of types of scales, but also the cognitive demands your measurement processes will impose on participants. Numerical scales are more cognitively demanding than graphical scales because participants will vary in the degree to which they are comfortable using numbers and attaching consistent meaning to them. An item such as item 9 in Fig. 18.2 imposes a high cognitive demand on participants, not only because of the more complex response required but because of the complexity of the instructions needed to ensure the participant completes the item correctly. Ill-considered response demands may dramatically reduce validity and reliability, not because the item content is wrong but because participants may not be able to figure out how to make an appropriate or meaningful response. Part of designing a good measurement instrument thus requires constant sensitivity to the types of participants who will be completing the measure. Robinson and Leonard (2019) offer extensive coverage of how to craft questionnaire items that are of high quality and appropriate for intended participants. It is also important to note that, under normal circumstances, it is possible to transform items at a specific scale of measurement down to a lower/simpler scale of measurement, but not vice versa. For example, an interval or ratio-scale measure can be transformed into an ordinal scale measure (which is precisely what nonparametric statistical procedures do) but an ordinal or interval-level measure ordinarily cannot be transformed into a ratio-scale measure, unless more sophisticated measurement methodologies are implemented (e.g., best–worst scaling, see Lee, Soutar, & Louviere, 2008). For more detailed discussions of measurement scales and their implications, see Cooksey (2014, pp. 19–23), de Vaus (2002) or Bryman and Cramer (2004, pp. 17–19). We should also note there is an alternative approach for producing measurement scales, often used in educational research, called Rasch modelling (one form of the more general item response theory). This approach “simultaneously estimates item difficulties and person abilities along the same interval-level measurement scale” (Cooksey, 2014, Procedure 9.1, p. 464; also see Andrich (2005), Bond and Cox (2001), Lamprianou (2008) and Ostini and Nering (2006) for more detailed discussion and illustrations).

18.4

What Counts as a ‘Good’ Measure?

Operational definition is only first necessary step in the design of a measurement system. We flagged earlier that an important follow-on question focused on how we know we are measuring well when we construct a measurement system. This

18.4

What Counts as a ‘Good’ Measure?

799

question concentrates our attention on the issue of measurement quality. With respect to measurement quality, we can ask three questions: (1) is the measurement instrument accurately reflecting the phenomenon or characteristic it was designed to (the issue of validity, which also goes to the issue of what inferences are appropriate to make using the measure); (2) is the measurement instrument measuring the phenomenon or characteristic consistently (the issue of reliability); and (3) is the measurement instrument sensitive enough to measure changes or differences in the phenomenon or characteristic (the issue of sensitivity). We will deal with each question in turn.

18.4.1 Validity There are four important types of measurement validity: • Construct validity addresses the question of whether the measure clearly and defensibly reflects the theorised construct of interest. • Content validity addresses the question of whether the measure adequately samples some learning, task or skill domain of interest. • Criterion-related validity addresses the question of whether the measure effectively predicts some criterion of interest. This type of validity is tied to the strength of the connection between two different measures, the predictor measure (the measure of interest from a criterion-related validity perspective) and a criterion measure (the measure of interest from a decision-making perspective). • Face validity addresses the question of whether an instrument ‘looks/feels’ like it measures its intended construct, generally from the viewpoint of participants providing the measurements. Bryman and Cramer (2004), Fogarty (2008) and Zumbo and Rupp (2004) provide more in-depth exploration of issues associated with measurement validity. Construct Validity Construct validity (Fig. 18.3) is arguably the most important kind of validity for measurements to possess in research guided by the positivist pattern of assumptions, especially if you employ the self-report questionnaires Measurement data-shaping strategy. However, you should know that construct validity is also a relevant measurement consideration in the physical/physiological/economic measurements and objective tests and assessments Measurement data-shaping strategies as well. Constructs emerge from theoretical thinking. The idea is that hypothesised constructs will be major causes of the patterns of responses observed on specific items designed to reflect that construct (hence the directed arrows from the construct to the items in the diagram). In Fig. 18.3, items a1 to a4 operationally define hypothetical construct A and b1 to b3 operationally define hypothetical construct B. Additionally, constructs A and B are be hypothesised to be correlated with each other. For positivist research to be convincing, construct validity must be either

800 Fig. 18.3 Diagrammatic depiction of construct validity

18 When and How Should I Deal with Measurements?

Construct Validity

Measurement items designed to reflect construct A

a1 a2 Hypothetical Construct A

a3 a4

Constructs A and B may be correlated, yet should be discriminable from each other

Measurement items designed to reflect construct B

b1 b2

Hypothetical Construct B

b3

demonstrated or argued for every construct involved in your research. Demonstration of construct validity will typically extend your research timeline and the complexity of your research configuration. Construct validity can be statistically demonstrated using multivariate statistical procedures such as factor analysis or principal components analysis. Here you explore and/or confirm the internal structure and dimensionality (i.e., the internal correlational structure amongst your items) of your construct and you could use a single MU configuration if instrument development and validation was your only research goal. However, in many cases, instrument development and validation may represent only the first stage in your research, and you will require a distinct developmental phase in your research configuration (such as the programmatic/ accumulative sequential MU configuration) and, most likely, a separate sample on which to run your validation analyses. In analysing measurement items to demonstrate construct validity, you are trying to show that your items group together in meaningful (and perhaps, pre-designed) ways to measure specific constructs or sub-constructs. In the earlier example about organisational commitment, Fig. 18.1 showed a three-component organisation amongst the measurement items. Using techniques such as exploratory or confirmatory factor analysis (see Cooksey, 2014, Procedures 6.5 and 8.6; see also Bryman & Cramer, 2004, pp. 26– 33; de Vaus, 2002, pp. 134–146), you could verify if items AC1, AC2 and AC… grouped together to form a ‘factor’, NC1, NC2 and NC… grouped together to form a second ‘factor’, separate from AC1, AC2 and AC… and so on. Generally, any

18.4

What Counts as a ‘Good’ Measure?

801

study that employs factor analysis, or some variant of it, is using a statistical approach to make the case for construct validity. Three items are considered the minimum necessary to statistically validate a single ‘factor’ but, in practice, researchers may start with many more items to allow for scale revision through deletion of poorly performing items (e.g., Allen & Meyer, 1990, started with 51 items for their organisational commitment inventory and eventually pared this number down to three sets of 8 items to measure the three types of commitment). Construct validity can also be empirically demonstrated, in a single MU configuration, using a process called the ‘known groups’ method, where pre-existing groups are hypothesised to be ordered in specific ways on the construct and empirical measurements of the construct are gathered to confirm if this is the case (see Hattie & Cooksey, 1984). With respect to a construct like organisational commitment, this could be done by comparing three groups of participants, group A comprising employees with a company over 20 years, group B comprising employees who just joined the company and group C comprising employees who just left their employment with the company. If you argue, before you gather your data, that comparisons should show that group A has higher average organisational commitment than groups B and C and that group B has higher average organisational commitment than group C, and your results then reflect that pattern, you would have evidence for the known groups validity of the measure. Construct validity can be logically argued if you can substantiate, via evidence from the literature or based on logical argument, that the structure of your construct measure maps well onto the theorised or previously validated structure of the construct. This approach can be used when you decide to adopt a measure that has been validated by another researcher. Based on their evidence and comparisons of your sample context and measurement conditions to that of the validating research study, you might argue that you can assume construct validity would generalise to your study. This argument would not be convincing unless you could show sufficient comparability between sample contexts and measurement conditions and such arguments would be especially problematic in the Cross-Cultural or Indigenous research frames. Generally, we recommend that logic not be the only process driving your arguments for construct validity; either the statistical or empirical approach should be used as well. Finally, construct validity can be demonstrated if you can show that your measure of a relevant construct such as organisational commitment, correlates in expected ways with measures of other constructs, such as organisational citizenship behaviour or job satisfaction. This process is really an extension of the known groups logic, where you argue that your measures should be related in specific ways, gather data on all of the relevant measures and then test whether or not the expected relationships are confirmed. Where multiple constructs are being measured, two sub-types of validity should be evident: (1) discriminant validity, where different constructs should be clearly and statistically distinguishable from each other (as with constructs A and B in Fig. 18.3) and (2) convergent validity, where measures intended to target a specific construct should be observed to statistically converge on (i.e., correlate highly with) that

802

18 When and How Should I Deal with Measurements?

construct. One process for evaluating convergent and discriminant validity uses what is called a Multi-trait Multi-method (MTMM) Matrix (see, for example, the classic paper by Campbell & Fiske, 1959 and Brewer & Hunter, 2006, pp. 115–118). We noted above that a minimum of three items per construct is required if exploratory methods such as factor analysis or principal components analysis are used as the construct validation tool. It is also possible to confirm a previously theorised construct structure using a process called confirmatory factor analysis, which is a form of structural equation modelling. There is, however, potential for using single item measures of constructs, especially in the realm of physical measurements (e.g., pulse rate or O2 saturation reading, blood pressure reading, time taken to complete a task). Skill- or learning-based, psychological or sociological constructs (e.g., intelligence, personality, group identity, stress, service quality, job satisfaction, organisational commitment, leadership style, quality of life) that cannot be directly observed typically require multiple items (sometimes referred to as indicators) per construct. However, some research (see, for example, Bergkvist & Rossiter, 2007; Gardner, Cummings, Dunham, & Pierce, 1998) demonstrated the relative merits, properties and uses of single-item construct measures compared to multi-item measures. Bergkvist and Rossiter (2007) indicated that, in terms of predictive validity, single-item measures would be preferred if the object or attribute being measured is concrete. Content Validity Content validity (Fig. 18.4) is the most useful type of validity for objective tests and assessments to possess where constructs deal with skills, aptitudes, learning, capabilities and potentials (i.e., when using the objective tests and assessments Measurement data-shaping strategy). For example, a final examination in a university unit of study needs to have content validity to ensure that all relevant domains of learning are being evaluated at the correct level and with the correct emphasis. Without content validity, inferences about achievement in the unit will be much weaker. A test for selecting people for managerial positions should have content validity by ensuring that all relevant aspects of the job of ‘manager’ are reflected and measured in the instrument, giving rise to the concept of a ‘job-sample’ or ‘work-sample’ test. Content validity is not generally assessed using statistical indices (although such indices do exist); rather it is built into the test or assessment using a test blueprint, that you, as the researcher, or a professional test/assessment organisation develop. A test blueprint transparently maps the intended relationships between item domains and the complexity and depth of learning (e.g., from Bloom’s revised learning taxonomy (Krathwohl, 2002): remembering, understanding, applying, analysing, evaluating, creating) that each item is intended to tap. The numbers of items in each cell of the blueprint (see the table in Fig. 18.4) can reflect the degree of emphasis on that domain and learning level in the curriculum, emphasis in the teacher’s or assessor’s marking or assessment scheme or relevance/importance to accomplishing specific tasks (such as work or managerial tasks).

18.4

What Counts as a ‘Good’ Measure?

Fig. 18.4 Diagrammatic depiction of content validity

803

Content Validity

e.g., 5 remembering/understanding-type measurement items sampled from sub-domain A

Researcher creates: TEST BLUEPRINT Sub-domain A

Remembering/ Understanding

Applying/ Analysing

Evaluating

5 items

3 items

2 items

Sub-domain B

5 items

3 items

2 items

Sub-domain C

5 items

3 items

2 items

a1 a2 a3

Task Domain

a4 b1

b2

b3

e.g., 3 applying/analysing-type measurement items sampled from sub-domain B

a5

A work or job sample test is one form of objective test where content validity is essential. With such a test, specific skills relevant to the accomplishment of work task or job are assessed. For tests that measure aptitude/ability-type constructs such as intelligence or cognitive ability, both construct validity (with respect to the theorised construct, such as general ability, verbal comprehension, spatial ability, numerical reasoning, and its dimensions, if any) and content validity (sampling skill/task domains relevant to each construct dimension) are essential. Criterion-Related Validity Criterion-related validity (Fig. 18.5) is an important type of validity for measurements used to inform decision making (could be relevant in the Evaluation or Action research frame, for example). Criterion-related validity focuses on the usefulness of one construct (the predictor) for predicting/making statements about another construct (the criterion). For example, the Myers-Briggs Type Indicator (MTBI) is sometimes used by employers to select employees having management potential. In such cases, the MTBI needs to demonstrate criterion-related validity because its intended use is as a selection tool to choose employees with Measurement items designed to reflect construct A

a1 a2 a3 a4

Criterion-related Validity

Measurement items designed to reflect construct B

[scores on construct A predict scores on construct B]

b1

Criterion

Predictor Hypothetical Construct A

Hypothetical Construct B

Concurrent Validity = a and b measurements obtained at the same time Predictive Validity = a measurements obtained first; b measurements obtained on a later occasion

Fig. 18.5 Diagrammatic depiction of criterion-related validity

b2 b3

804

18 When and How Should I Deal with Measurements?

management potential, in advance of knowing how the employees would perform as managers. If scores on the MTBI did not strongly correlate with performance as a manager, then you would not want to use it to make selection decisions. There are two ways of assessing criterion-related validity: (1) concurrent criterion-related validity, where the predictor construct(s) and criterion construct are measured at the same time; and (2) predictive criterion-related validity, where the predictor construct(s) are measured before the criterion measurements are obtained. Concurrent validity is generally established using a simultaneous MU configuration, whereas predictive validity can be established using a sequential or longitudinal MU configuration. Regression analysis is usually the statistical tool used to establish either concurrent or predictive criterion-related validity (see Cooksey, 2014, Procedures 6.3, 6.4 7.13 and 9.8). Restriction of range can be a problem in establishing criterion-related validity, especially if the sample for the criterion measurements has been pre-screened or pre-selected in some way. For example, in research that uses a pre-existing cohort of participants, such as enrolled university students, to establish the criterion-related validity of a predictor test, restriction of range is a problem because people who have not gained admission to the university have no chance of appearing in the sample. In general, any aptitude, ability, diagnostic, personality or work sample test used to help inform decision making, such as job or managerial selection decisions, university admission decisions or course of medical treatment to follow, should demonstrate either construct validity and/or content validity as well as criterion-related validity. Face Validity Face validity (Fig. 18.6) reflects the subjective impact of a measurement instrument on participants, focusing on whether participants think the items are addressing the construct you claim or indicate is being assessed. If an instrument does not look, to the participant, as if it is measuring what you say it is or if the instrument does not look professionally developed (e.g., poorly formatted, evidence of typos, grammatical errors, awkward or unclear expression, unclear response expectations, too many ‘bells-and-whistles’), this can adversely impact face validity and may contribute to poor response rates because participants may not take the instrument or Fig. 18.6 Diagrammatic depiction of face validity

Face Validity Hypothetical Construct A

a1 a2 a3 a4

18.4

What Counts as a ‘Good’ Measure?

805

test seriously and either produce meaningless/random responses or withdraw from completing it altogether. Face validity can be an issue in cross-cultural samples, in part due to differences in language and cultural concepts. Unusual or unfamiliar wording or expression, unclear use of negative words such as ‘not’ and use of terms whose meaning is culture-specific (e.g., jargon or slang) can create confusion and, perhaps even offense, in multi-cultural samples, leading to an inability or unwillingness to treat the instrument seriously. Face validity is not quantified; rather it is built into an instrument or test through serious attention to detail in all aspects of its production and distribution.

18.4.2 Reliability There are three common approaches to estimating the reliability of a construct measure: • Internal consistency reliability addresses the question of how consistently a set of items or measurements work together to measure a common theoretical construct. • Test-retest reliability addresses the question of how stable a construct’s measurement is over different measurement occasions. • Inter-observer/inter-rater reliability addresses the question of how consistently two or more observers or raters independently record the same observations on the same entities at the same time. Bryman and Cramer (2004), Cooksey (2014, Procedure 8.1), Fogarty (2008) and Zumbo and Rupp (2004) provide more in-depth exploration of issues associated with measurement reliability. Internal Consistency Reliability Internal consistency reliability (Fig. 18.7) refers to how well a set of items (typically referred to as an ‘instrument’) works together to measure the same construct. It assumes the instrument is homogeneous, not multi-dimensional (i.e., it measures a single construct, not several). Internal consistency reliability is estimated using a coefficient called Cronbach’s alpha (a), which ranges from 0 (no reliability) to 1.0 (perfect reliability; almost never achieved; see Cooksey, 2014, pp. 372–373). The internal consistency reliability estimate can be made using data from a single sample on one measurement occasion (a single MU configuration). Internal consistency reliability can be adversely influenced by using multidimensional measures (a set of items measuring multiple constructs) but ignoring that multidimensionality. Also, adding items to an instrument measuring a construct may artificially inflate Cronbach’s alpha as can using items that are too highly correlated (which can happen if two or more items differ only by a couple of words). Generally speaking, a minimum value for Cronbach’s alpha would be 0.6

806

18 When and How Should I Deal with Measurements?

Internal Consistency Reliability Measurement items reflecting construct A

[indexed by Cronbach’s Alpha (α)] Mary

a1

Joe a2 a3

Bill

Scores for Construct A

John Jill

a4

Jean People ordered from high to low in scores on construct A

Fig. 18.7 Diagrammatic depiction of internal consistency reliability

for a new researcher-constructed measure and 0.8 for a professionally developed instrument. Cronbach’s alpha is most frequently reported in validation studies where the measurement instrument has been analysed for construct validity using factor analysis. Test-Retest Reliability Test-retest reliability (Fig. 18.8) is a straightforward and logical way to estimate reliability. Estimating it requires two measurement instrument or scale administrations to the same sample of participants (i.e., a longitudinal MU configuration) and then correlating the scores from the two measurement occasions. Test-retest reliability can be artificially inflated by memory carry-over effects (during the second test administration occasion, participants remember how they responded in the first administration occasion) and this effect is stronger the closer in time the two measurement occasions are. Conversely, if the measurement occasions are too far apart, chances are greatly increased that unforeseen Fig. 18.8 Diagrammatic depiction of test-retest reliability

Test-Retest Reliability Measurement occasion 1

Measurement occasion 2

Mary

Mary

Joe

Joe

Bill

John

John

Jill

Jill

Bill

Jean

Jean

People ordered from high to low in scores on construct A

18.4

What Counts as a ‘Good’ Measure?

807

intervening events will occur that influence the second administration occasion scores dramatically (which could change the nature of the construct being measured, thereby reducing reliability as well as validity). Because a longitudinal MU configuration must be used, the chances of participant withdrawal (also called the ‘mortality’ rate or ‘dropout’ rate) increases, which reduces the available sample size for the second administration occasion. To circumvent the difficulties created by needing to administer the instrument or test on two occasions, two single-occasion variants of test-retest reliability estimation have evolved: • split-half reliability: where the test or instrument is administered to the sample, its items are then randomly split into two ‘half-tests’, participants are scored on each half and the two half-test scores are correlated and corrected for test length, from half-test length to full-test length (see Athanasou, 1997, pp. 179–180). Split half reliability only works well for test or instruments containing a large number of items (say 20 or more) and you are willing to assume that the two halves are equivalent tests (i.e., equally valid measures of the same construct, which may depend upon the splitting method: random or odd items and even items). • parallel forms reliability: where two alternative but equivalent forms of the measurement instrument are administered to the same sample of participants on one occasion. Participants are scored on the two parallel versions of the instrument, and these two scores are then correlated (see Athanasou, 1997, pp. 178–179). This only works if the two versions of the instrument are demonstrably equivalent and you must be aware of the potential influence of boredom and fatigue on participants’ scores when they are asked to complete two highly similar instruments, essentially taking two full tests instead of one. Inter-Observer (Inter-Rater) Reliability Inter-observer or inter-rater reliability (Fig. 18.9) is important to assess in the systematic observation data gathering strategy or in any research that codes behaviour using a data recording protocol, such as coding behaviours in a video clip, camera recording or other multimedia format (i.e., the Transformative data-shaping strategy), under the positivist pattern of guiding assumptions. The observation coding instrument needs to be as clear and unambiguous as possible. This is much more difficult to achieve if observers need to code for deep/latent meaning as opposed to surface/manifest appearance. The more judgment required of observers to code or rate their observations, the lower inter-rater reliability is likely to be. To estimate inter-rater reliability, two or more observers observe and code/rate the same behaviours of the same individuals/groups in the same setting at the same time, but completely independently of each other (so there is no chance of cross-contamination of ratings between observers, which would adversely affect construct validity). The ratings from the different observers are then correlated to estimate reliability. A popular index for reporting inter-rater reliability is Cohen’s

808

18 When and How Should I Deal with Measurements?

Inter-Observer or Inter-Rater Reliability

Independently rates construct A for same sample of people in same situation at same time as Dark-Grey Observer

Mary

Mary

Joe

Joe

Bill

John

John

Jill

Jill

Bill

Jean

Jean

Independently rates construct A for same sample of people in same situation at same time as Light-Grey Observer

People ordered from high to low in observer scores on construct A

Fig. 18.9 Diagrammatic depiction of inter-observer or inter-rater reliability

kappa coefficient of agreement, especially useful where observers code behaviours into different categories (see the discussion in Cooksey, 2014, pp. 379–380).

18.4.3 Relationship Between Reliability and Validity Validity and reliability are measurement qualities that are necessarily connected to each other and their patterning leads to different judgments about measurement quality as shown in Fig. 18.10. Valid measures will always be reliable, but reliable measures may not always be valid because they may measure an unintended construct. For example, you could design an instrument that you think measures job satisfaction. You administer the instrument to employees from a non-English speaking background. Your data show the instrument to be highly reliable, but this does not mean the instrument is valid. The instrument may not be reliably measuring job satisfaction so much as reliably measuring competence in the English language. This can be especially problematic in the Cross-Cultural or Indigenous research frames, research with children and research with people with disabilities, where literacy or reading ability may be required to handle written measurement items. This fact that a reliable measure may not be valid is one reason why reporting Cronbach’s alpha for instruments in the literature provides insufficient evidence for instrument quality as this index says nothing about any form of validity. Bryman

Validity

AND Reliability

= High quality measurement of desired construct

Validity

AND Reliability

= Not possible

Validity

AND Reliability

= Reliable measurement of wrong construct

Validity

AND Reliability

= Poor quality measurement

Fig. 18.10 Relationship between reliability and validity

REJECT

KEEP

REDESIGN

18.4

What Counts as a ‘Good’ Measure?

809

and Cramer (2004), Carmines and Zeller (1979), Cooksey (2014, Procedure 8.1), Fogarty (2008) and Zumbo and Rupp (2004) provide more in-depth exploration of issues associated with measurement validity and reliability.

18.4.4 Measurement Sensitivity Measurement sensitivity concerns the capacity of a measurement system to reflect changes and differences. This is especially important in research configurations where demonstrating change over time or conditions is an important goal (any longitudinal MU configuration, for example). The key feature of a scale that impacts on its sensitivity is the number of discrete rating or measuring points it offers. For example, a Likert-type scale could offer anywhere from three to nine (or even more) rating categories. Any more than nine rating categories can create cognitive difficulties for many participants, because this number exceeds their short-term memory capacity, impeding participants’ abilities to keep the distinctions between all the different rating categories clear in their heads. If the goal of your research is to detect and demonstrate change over time, then a three-point scale would be far less sensitive to change compared to a 7- or 9-point scale. Measurement sensitivity can also be enhanced by using multiple items in an instrument to measure the same construct—a strategy that simultaneously assists in demonstrating construct validity. Using multiple items to measure the same constructs takes advantage of the fact that multiple items are generally more reliable than single items. You can add up multiple items to form an instrument score that offers a much larger set of categories to differentiate between. For example, consider the potential sensitivity of a single item measuring teacher satisfaction using a 5-point Likert-type scale (only 5 categories of possible difference) compared to an 18-item teacher satisfaction instrument, where each item uses a 5-point Likert-type scale, but adding all of the items up yields a measure that could potentially range from 18 to 90 (72 categories of possible difference). Of course, you must demonstrate/argue that all 18 items measure the same construct of teacher satisfaction and that there are no missing observations for any person in the sample in order for this process to be defensible.

18.4.5 Constructs, Variables and Models Many of the strategies for structuring people’s experiences, under the guidance of positivist or critical realist assumptions, can be associated with a statistical model. As we indicated earlier, a theory comprises a set of possibly interrelated constructs that are measured by or represented in variables thought to be either causally or non-causally connected. A statistical model provides a way of concretely translating those theoretical propositions, relationships and hypotheses into a testable form.

810

18 When and How Should I Deal with Measurements?

Any statistical model places the constructs (operationally defined as variables through measurement processes) in relationship to each other, with identifiable roles: • Measures that are theorised to be putative causes are called independent variables (IVs) and measures that are theorised to be putative effects are called dependent variables (DVs)—see Fig. 18.11. • Measures that are theorised to operate as intermediate causes between independent variables and dependent variables are called mediating variables (MedVs)—see Fig. 18.12. • Measures that are theorised to operate to alter, modify, qualify or render conditional the relationship between independent variables and dependent variables are called moderating variables (ModVs)—see Fig. 18.13. • Measures that represent constructs or events that are not controlled by or not of interest to you are called extraneous (or nuisance) variables (EVs)—see Fig. 18.14. If you allow such variables to change at the same time as a theorised IV changes, this will interfere with your ability to draw unambiguous, defensible and convincing causal inferences. This happens because changes in the EVs serve as plausible alternative explanations for why specific patterns in DV measurements were observed, a phenomenon termed ‘confounding’. Ruling out such EVs as plausible causes requires you to implement some kind of control process. • Anything that is not captured in a model in one of the above roles is lumped into model error, which comprises two components (see Fig. 18.15): – unsystematic error or random errors, which are errors that cannot be anticipated, predicted or controlled for (e.g., bad days, interruptions and distractions, one-off events); and – systematic errors, which are errors that could be anticipated, predicted and potentially controlled for in future research but which remain uncontrolled and therefore outside of the model being tested in the current research. When you are testing theoretical propositions, hypotheses and/or predictions, you will likely use something related to what is called the general linear model (which encompasses a wide range of commonly used analytical procedures ranging from simple t-tests to structural equation models). Figure 18.16 shows a diagrammatic conceptualisation of how these different categories of variables (i.e., measurements of constructs) are related to each other in such a statistical model (adapted from Cooksey, 2014, pp. 209–215). DATA (in the form of one or more DVs) are theorised to be explainable by a MODEL (comprising some function of IVs, MedVs and/or ModVs). However, any MODEL will not work perfectly, so ERROR must also be built into the conceptualisation. The representation shown in Fig. 18.11 IV to DV relationship

IV

DV

18.4

What Counts as a ‘Good’ Measure?

811

Fig. 18.12 A MedV operates intermediately between an IV and a DV

Med V

IV

DV

Fig. 18.13 A ModV changes the relationship between an IV and DV

Mod V

IV

Fig. 18.14 Changes in EV contiguous with changes in an IV create confounding; control helps to rule these EV influences out

DV

IV

DV Rule out/ control

EV1

Rule out/ control

EV2

Fig. 18.15 The composition of error

DV

Error

Unsys

Sys

Fig. 18.16 is called a ‘linear’ model because the MODEL and ERROR components are added together to explain DATA. Such linear statistical models are the most commonly employed models in the social and behavioural sciences. Systematic errors constitute uncontrolled alternative plausible explanations for patterns of DV measurements and may be handled in at least three different ways:

812

18 When and How Should I Deal with Measurements?

DATA

=

MODEL

+

Independent Variable(s) Measurements of Dependent Variable(s)

=

Mediating Variable(s) Moderating Variable(s)

DATA DV changes measured

analysis reveals

MODEL What the MODEL explains

ERROR

Systematic + Unsystematic/ random

ERROR What the MODEL cannot explain

Fig. 18.16 Conceptual representation of model building in strategies for Structuring People’s Experiences and Data Shaping (under the guidance of the positivist or critical realist pattern of assumptions) when theoretical propositions, hypotheses and/or predictions are being tested

(1) be transferred from the ERROR component to the MODEL component through deliberate inclusion as a measured construct (i.e., incorporation into the MODEL), (2) be removed from the research scenario or negated altogether through some process of control (managed exclusion by you, as researcher) or (3) remain unknown thereby continuing to contribute to the ERROR component. In addition, there may be situations where the MODEL is not appropriately specified, giving rise to other alternative plausible explanations such as, for example, when a key IV has been omitted from research consideration or a nonlinear relationship exists between an IV and the DV that is not represented or cannot be represented in the MODEL. Now, for the important caveat: your model will be convincing only when all of the measures of constructs it encompasses are both valid and reliable. If they are not, (1) this will inflate both the systematic and unsystematic error components in your model, making it much harder for you to substantiate hypothesised relationships, and (2) you will be unable to convince a reader that you have incorporated the appropriate IVs, mediators and/or moderators, meaning that the model you test will not be the model you hypothesised.

18.5

18.5

Choosing/Using a Measure Developed by Other Researchers

813

Choosing/Using a Measure Developed by Other Researchers

When it comes to measurement issues in quantitative research, it is important to realise that different considerations arise depending upon whether you employ measures developed and validated by others or you develop your own measures. There are a range of considerations that you need think about when you adopt a measure developed, validated and published by another researcher.

18.5.1 Issues to Consider Prior to Adoption • Consider context of origin versus context of application: Here, you need to carefully consider the generalisability of the constructs and the operational definitions in the instrument to the new context in which you apply it. This often implicates an international and cultural dimension where you may wish to adopt a measure or instrument, developed, for example, by researchers in the US, to use in an Australian context or perhaps in the context of a non-Western culture such as United Arab Emirates, Thailand, China or India. We are not only focusing on the language aspect here, although that is clearly a consideration as well in that, even in cultures as similar as the US and Australia, spoken and written English has local variations in sense and meaning that may influence the interpretability of specific items in the measurement instrument. Beyond the language dimension, however, there is the issue of the meaningfulness of constructs (and/or their sub-constructs) in the different context. The construct of organisational commitment, for example, may have very different meanings and even very different sub-components between the US and the UAE, for example. In the UAE, organisational commitment has Islamic and other religious or national cultural implications that are not captured by the US-version of the construct (as Alhosany, 2011, discovered during his professional doctorate research). Any cultural misfit between context of development and context of application can have serious implications for measurement errors in your study. This may require that you configure a component of your research to explicitly evaluate the cross-context suitability and applicability of the measure/instrument you want to use. • Examine published validation and reliability data: When adopting a measure or instrument developed by other researchers, you are obligated to examine and summarise any validation research accompanying the measure. This examination should not just focus on the reliability of scales (which is what many authors typically report) but also on any demonstrations of construct, content or criterion-related validity, as relevant. Thus, you would want to look at research that explores or confirms such things as factor structure and dimensionality of the measure and any predictive relationships the measure has with other

814

18 When and How Should I Deal with Measurements?

measures and constructs, being careful to situate each study in its appropriate context. This is so that you can make an informed judgment about the degree of appropriateness of the measure to your own planned context(s) of application. • Examine published critiques of constructs and/or instrument: When adopting a measure or instrument developed by other researchers, you should also read any published critiques or extensions associated with it. This will familiarise you with any debates about the nature and structure of the construct and how it may have evolved over time. Some constructs and their associated instruments have evolved quite a bit over the years (e.g., role conflict and role ambiguity, compare Rizzo, House, and Lirtzman (1970) and King and King (1990)) and you would want to ensure you can precisely situate the specific version of the measure or instrument you want to adopt. • Ensure you obtain appropriate permissions needed for use: Some instruments are offered by researchers without charge (some even include all items on an instrument in an Appendix to a paper or on a website) and these can be used freely if the source is appropriately cited. However, some authors have copyrighted their measure or instrument and must give their explicit permission for its use. They may even place limitations on how the instrument may be scored and interpreted (through a paid service, for example). In these cases, you need to be very careful to obtain all necessary permissions and have access to all the necessary resources to perform the requisite scoring. • Pilot test and re-validate instrument if it has to be adapted to your local context (s): Generally speaking, unless you are adopting a specific instrument without any changes whatsoever (a decision that would need rigorous defence), it is good practice to plan to conduct a pilot study on your version of the instrument and re-validate its dimensional structure and reliability or devote a part of your main study to this effort. This, of course, will influence the MU configuration you employ. Often, if application contexts are sufficiently different, you will find variations in item structures that you will have to account for.

18.5.2 Issues to Consider Prior to Use in Analyses • Know the scoring and interpretation procedures: Once you have adopted a measure or an instrument, you need to ensure that you fully understanding any scoring and interpretation procedures associated with it. For some instruments, this may be as simple as adding up responses to a set of items. For other instruments, a more complex scoring procedure may involve such operations as reverse-scoring specific items before a final score is calculated or weighting items in a specific manner to compute the final score. If you re-validate an existing instrument, you may have to create your own set of scoring and interpretation procedures.

18.5

Choosing/Using a Measure Developed by Other Researchers

815

• Verify item behaviour: We mentioned earlier that it is good practice to re-validate an instrument you adopt. This can be done using a pilot sample or as a set of analyses of your main sample, conducted before you begin analyses to address research questions involving the measure. In such validation analyses, you will want to either explore, using exploratory factor analysis, or confirm, using confirmatory factor analysis, the dimensional structure and reliability of the measure. It would be important to then compare your results to published validation data and try to understand and explain any variations that emerge. Remember that if you do find any differences between the structure and quality of the instrument, as applied to your context and sample, and those of published validation research, all is not lost! The mark of a good researcher is acknowledging the local variations they find with a construct and/or its method of measurement and building an account for why those variations may have occurred. Sound arguments will set the stage for you to then use your local validation results as the basis for addressing your research questions. In some cases, it may be appropriate to argue that the published instrument and its associated validation results are adequate for your intended use without modifications to interpretation. However, such arguments would need to be convincing; more so if the context(s) for application appear very different from the context(s) of development. This would be a decision you would want to consider carefully in close consultation with your supervisor(s). • Score participants on any factors/dimensions of the measure or instrument: Once you have a firm grip on the local item structure of your measure or instrument, you will then need to score every person in your sample on the dimensions you have identified. Generally speaking, unless specific scoring instructions for the instrument recommend otherwise, it is not a good idea to compute factor scores for participants (statistical packages such as SPSS can do this at the press of a button, but don’t). There are three reasons for this: (1) factor scores do not cross-validate very well, meaning that they are optimised only for your local sample; (2) factor scores explicitly include responses on all items in the calculation, irrespective of whether or not the items actually substantively contribute to that factor; and (3) for certain methods of factor analysis (namely the common factoring methods, see Cooksey, 2014, Procedure 6.5), unique factor scores cannot be computed. A better practice is to give participants factor scores by averaging, not adding, only those items that you have concluded define a dimension or factor (easily done using the Compute Variables … function in SPSS, for example). Creating factor scores by averaging defining items ensures that (1) items for which responses are missing are excluded from the computation for specific individuals (adding scores would give an artificially low total if one or more items were missing for that person) and (2) the resulting score is expressed in the original scale of measurement, so it can be easily interpreted against the original item scale. Cooksey (2014, pp. 164–165) provides additional insights into this debate about factor scores.

816

18 When and How Should I Deal with Measurements?

18.6

Developing and Validating Your Own Measure

There are a range of different considerations that you need think about when you decide to develop and validate your own measure.

18.6.1 Issues to Attend to During Development • Appropriate conceptualisation of the construct(s): You cannot just sit down one day and write a set of questionnaire or instrument items off the top of your head. To design a good measure, you need to ensure that you have a clear idea of the construct(s) you are trying to measure, including any sub-constructs that might be associated with them. Diagramming or mind mapping your construct and its desired structure in a manner similar to Fig. 18.1 can help. From both a practical and a statistical perspective, each construct or sub-construct you want to measure should have at least three distinct items written to tap it. However, this is the absolute minimum. A better practice is to write at least 5–6 items per construct. This will allow for scale revision and possible deletion of poorly performing items during the validation phase. As you write items, remember you are effectively trying to sample the domain of behaviours, attitudes or outcomes that you think will reflect the influence of the target construct or sub-construct. Almost inevitably, the first draft of your measure will be longer than the final validated version. • Write good quality items: Writing good items for a questionnaire or instrument takes time, effort and anticipation. The anticipation aspect refers to always thinking ahead to how the types of participants you intend to sample might react to and cope with each item you write. This requires careful attention to item wording and complexity of language and vocabulary used. Simply and straightforwardly written items will generally work best. Here are some other tips: – Avoid trying to shorten your instrument by writing double-barrelled items. A double-barrelled item is signalled by the use of the clause connectors ‘and’ or ‘or’ and offers more than one target for rating (e.g., “Most employees need good salaries and working conditions to be satisfied.”). Such items create adverse impacts on construct validity and renders interpretations ambiguous. – Also try to avoid asking leading or loaded questions that suggest a ‘preferable’ response (e.g., “It is wrong to take stationery supplies from your workplace”). Again, such items have adverse impacts on construct validity. We have previously mentioned that it is important to design your response scales for items with the anticipated types of participants and cognitive demands clearly in mind. When you design Likert-type response scales, keep in mind that unipolar (Fig. 18.2, item 6) and bipolar (Fig. 18.2, items 4 and 5) questions have different scale design requirements. Response dimensions, such as ‘disagree , agree’, ‘dissatisfied , satisfied’ and ‘like , dislike’, are clearly bipolar in orientation and others, such as ‘not important ) very important’, ‘not at all ) ‘a great deal’

18.6

Developing and Validating Your Own Measure

817

or ‘never ) always’, are clearly unipolar. However, some dimensions may need more careful consideration. For example, creating a ‘dissatisfied , satisfied’ bipolar scale is one option. However, a unipolar version could also be designed: ‘not at all satisfied ) very satisfied’. You would need to decide what you want to know and what you want participants to reflect in their ratings. The unipolar version can only reflect degree of satisfaction and any inferences about dissatisfaction would be not be sustainable simply because a participant ticks ‘not at all satisfied’; that is, not being satisfied may not be conceptually the same idea as being dissatisfied. There are some considerations in item writing that can be considered debatable or controversial. Krosnick (1999) and Bryman and Cramer (2004) provide useful discussions of some of these debates in their discussions of scale and questionnaire construction. Table 18.1 explores some of more common debatable questions that tend to arise and offers some recommendations. • Attend to the logical ordering and professional appearance of a self-administered instrument or questionnaire: Writing the content and response scale for each item in a questionnaire or instrument is one thing, getting its overall presentation and appearance right is another. Here it is critical that you be able to put yourself in participants’ shoes and ask yourself, for example, ‘would I complete this questionnaire if I received it in the mail or encountered it on the internet?’ In terms of the ordering of questions, the best strategy is to start with easy non-threatening questions (such as questions of a demographic nature, but not about income) then move onto the more complex rating aspects and close the questionnaire with a simpler question or perhaps an open-ended question. This strategy will ease participants into the instrument, begin to build trust and create the impression that your questionnaire is serious and professional before hitting them with the harder, perhaps more sensitive or probing items. • The physical appearance of a questionnaire or instrument is a major determiner of response rates in self-administered or web-administered questionnaires and close attention to overall layout and design will not go astray. If a questionnaire looks slapdash and has errors in it, this will encourage the cessation rather than completion of participation. Issues worthy of attention are: – Make it the right length (i.e., minimise the time the questionnaire will take to complete). This may be judged by sheer thickness or weight for a mailed questionnaire or amount of time spent downloading pages for a web questionnaire. If the questionnaire appears long, it will discourage participation and/or encourage withdrawal thereby impacting negatively on response rates. Remember that download and upload time should be counted as part of the overall time it will takes to complete a web-based questionnaire. – Make response requirements as easy and unambiguous as possible: If participants cannot clearly understand what kind of responses they need to make to items, the data you obtain will be riddled with measurement errors and you will perhaps have a higher rate of missing responses, simply because participants were confused. This goes not only to the clarity of the construction of response scales themselves, but also the clarity of the instructions for their use.

It could be argued that the relative trade-off risks to measurement quality of using an even number versus an odd number of response categories favour an even number (forcing a choice around the middle). Any frustrations could be mostly offset by: • making sure that categories around the middle only slightly deviate from the middle in either direction, and/or • offering a Not Applicable or Unable to Rate category that is visually and/or numerically separated from the rest of the scale. These strategies are illustrated in item 4 in Fig. 18.2. There are two strategies for reducing the chances that the middle category will be used as an ‘out’ for participants to avoid revealing their true feelings: • offering a Not Applicable or Unable to Rate category that is visually and/or numerically separated from the rest of the scale, and/or • phrasing the item more indirectly if it addresses a sensitive issue (redirect the focus away from the participant to ‘other people’). These strategies are illustrated in item 5 in Fig. 18.2. If you offer a neutral point, then using negative numbers for the negative half of the scale, “0” for the neutral point and positive numbers of the positive part of the scale will likely produce a better cognitive match between number and meaning for participants (even though there is no true zero point). If you don’t offer a neutral point, then the use of negative numbers will simply be confusing because there is no specific “0” point. Items 4 and 5 in Fig. 18.2 illustrate these contrasting scaling styles. For unipolar scales where a conceptual zero point is flagged (e.g., where the lowest category is something like ‘not at all important’ or ‘never’), use “0” rather than “1” to reflect this lowest category. It is not a real zero-point (it doesn’t mean you have a ratio scale) but, from a participant’s perspective, “0” will have a better cognitive match to the meaning of the category (see item 6 in Fig. 18.2).

If a neutral category is deemed important to include, then steps should be taken to reduce the chances that the category would be ‘mis-used’ by participants to reflect something other than a genuine neutral feeling. The problem is that many researchers just use an odd number of categories without thinking seriously about the potential implications of including the middle category

This issue goes to the heart of the psychological meaning of the categories on the scale. Research (e.g., as reported in Krosnick, 1999) has shown that the numbers used do make a difference. However, this is only an issue if you require participants to circle or tick a numbered category rather than circle or tick a verbal abbreviation for the category (e.g., SD, D, N, A or SA for a disagree-agree scale)

This issue also goes to the heart of the psychological meaning of the categories on the scale

Should bipolar Likert-type or semantic differential scales have a centre or neutral point? This raises the issue of whether an even or an odd number of response categories is preferable

If a ‘neutral’ (central) category is offered on the scale, how can you be sure a ‘neutral’ response really means neutral? This follows naturally from the above question, depending upon stance taken

In a 5-point bipolar scale, for example, should you use the numbers 1–5 or −2 to +2 to reflect the categories?

In a unipolar scale, should the lowest category be scaled as a 1 or a 0?

(continued)

Recommendations

Why an issue? Researchers have different views on this issue. An even number of categories forces participants to choose around the middle, making it harder to opt out of responding by ticking the middle category. However, if a middle category is not offered, this may frustrate some participants who genuinely feel ‘neutral’. An odd number of categories allows for expression of a genuine neutral feeling but may also be used by participants to opt out of revealing their feelings. If the latter happens, the scale becomes distorted and measurement error increases

Question

Table 18.1 Debatable issues associated with the design of measurement items

818 18 When and How Should I Deal with Measurements?

The general recommendation is that an effective Likert-type or semantic differential scale should include a minimum of 3 categories and a maximum of 9; 5, 6 or 7 distinct categories seem to work best. More than 9 categories make it very difficult for the participants to keep the category differences clearly in mind. Generally speaking, if sufficiently differentiated verbal anchor labels can be devised, then all response numbers or categories should have a verbal anchor (see Krosnick, 1999). Sufficiently differentiated should be taken to mean that the labels chosen would divide the scale into approximately equal-sized chunks. Note that distinct verbal labels are harder to come up with if you have more than 7 categories, especially at the extreme ends of the scale where ‘bunching’ might occur (e.g., ‘very satisfied’, ‘highly satisfied’, ‘extremely satisfied’). These types of questions should be avoided if possible because different participants will likely use different reference points (because of their differing experiences) and you will have no way to ensure that the comparisons they made were against equivalent standards. Go for an absolute rather than a relative rating. For pattern responders (who respond only in the middle, at the extremes or randomly): Design the instrument as a whole to engage participant interest but keep it short if possible. More specific instructions to consider and use the full range of the scale as they respond, and clearer assurances and reinforcement of anonymity may also help. For social desirability (where responses are influenced by the need to have one’s views be seen in a socially acceptable light): One strategy is to explicitly measure social desirability tendency using an instrument such as the Marlowe-Crowne scale so that the tendency can be statistically controlled for during data analysis (see, e.g., Fischer & Fick, 1993). Another strategy is to modify item wording so as not to induce the tendency. For example, avoid using potential trigger words such as ‘should’, ‘must’ or ‘ought’ to avoid inducing a social desirability mind set.

This raises the issue of how finely or precisely you want to define the meaning of each category. It may also invoke considerations of a trade-off between instrument or questionnaire layout simplicity or complexity.

These types of scales are commonly used in evaluation research, for example. However, they raise a question regarding what the actual comparison benchmark(s), standard (s) or reference point(s) were for the participant when they made the rating. This question arises from concerns about the motivations of participants to answer questions honestly and seriously. Participants who do not take your questionnaire or instrument seriously (recall the earlier discussion of face validity) or differ in their level of cognitive complexity or trust in you, as the researcher, they may respond to your items in atypical yet patterned ways. The risk is greater for items that touch on sensitive territory, such as ethics, honesty, religion, sexuality, race relations and so on. Different strategies are required for different types of response bias.

How many discrete scale categories should be used in Likert-type or semantic differential scales?

Should every response number or category on a Likert-type scale have a verbal anchor?

Should benchmark or standard comparison scales be used? An example might be: Compared to all the previous supervisors I have experienced, I would rate my current supervisor as: below average average above average

Should I design my items to reduce the chances of response set bias?

Developing and Validating Your Own Measure (continued)

Recommendations

Why an issue? This question points to the interaction between cognitive demands made on participants and the need for measurement sensitivity

Question

Table 18.1 (continued)

18.6 819

In general, no more than 7–9 ‘objects’ can reasonably be ranked by participants without cognitive heuristics or shortcuts being invoked. If your list of possible things to be ranked is large, ask the participant to rank only the top 3–5 While the answer to this question may ultimately depend upon your research purposes and pattern of guiding assumptions, one strategy that can be recommended, irrespective of the positioning of your research, is to include at least a few open-end questions in your instrument, even if not central to your research questions. The reason is that, for many participants, being asked for their views enhances their engagement in the questionnaire and commitment to completing it. It signals that you are interested in their views more personally rather than in solely what ticks they make or numbers they circle. In short, this strategy could enhance response rates, especially if you flag, in your covering letter, that there will be opportunities for them to express their views on the questionnaire or instrument. Drawbacks here include increasing the overall appearance of length in the questionnaire (to provide sufficient space for them to write) and possible illegibility. Some participants may use the question as an invitation to vent their spleen if they are unhappy about something.

This concern arises in the design of certain types of ordinal, interval or ratio scales, such as displayed in items 2 and 9 in Fig. 18.2 This question emerges from tensions between positivist demands for high control over item content and responses (to ensure more valid and reliable measurement) and more humanistic concerns that the views of participants should be sought

Should you include some open-ended items in your instrument? Such questions invite the participant to write their own views in response to a specific question

For acquiescence (where participants try to give you what he/ she perceives you want, such as agreeing with every item): A strategy, which not everyone agrees with, is to sprinkle negatively worded items through the instrument or measure to help spread responses across the rating scale. The scores on such items should then be reversed prior to analyses. A risk in this strategy is that if positively and negatively worded items are factor analysed together, you may end up with all the negatively wording items coming together, regardless of content. This is an artefact called a ‘method’ factor and it can interfere with clear validation of your construct. Negatively worded items can also create problems in multi-cultural samples, where English is not the first language of participants.

How many objects can be ranked, compared or sorted (with a rank attached to each object)?

Recommendations

Why an issue?

Question

Table 18.1 (continued)

820 18 When and How Should I Deal with Measurements?

18.6

Developing and Validating Your Own Measure

821

– Get the right balance between text and white or blank space: A questionnaire or web page that looks too crowded with text will make your instrument look long and complicated. This is especially true if you use a two-column format. A good balance between text and white space enhances readability. Clear visual separation between number or category columns for Likert-type ratings and good visual alignment of items with their response options and verbal anchors will both improve the overall appearance of the questionnaire and reduce completion errors. Part of what can help here is an appropriate selection of type and size of font: too small looks complicated, spacing will appear crowded and may be hard to read; too large and ‘non-academic’ looking (e.g., using a handwriting font or comic sans font) may suggest that the questionnaire is not serious and may insult the participant’s intelligence. A san serif font is generally more readable on a web page. On a web page, the text needs to stand out from the background colour on the page. Brighter contrasting colours should be avoided because they will reduce readability and some colour combinations should be avoided because of colour ‘bleed’ on many computer screens (e.g., red font on a dark blue background). – Minimise the number of ‘bells and whistles’: For paper-based questionnaires, coloured paper should generally be avoided as people will react differently to different colours of paper and black print does not always work well on some colours of paper. White paper generally works best. Heading structures on paper-based questionnaires should be simple and add to the visual segmentation and logical organisation of the instrument. For web-based questionnaires, try to keep the fancy web design gimmicks (e.g., blinking fonts, moving pictures, transition effects) to a minimum—they distract more than add value and, in some cases, add significantly to download time. – Make it look official: For many participants, what encourages their participation is knowing that the questionnaire or instrument (whether paper-based or web-based) has the official imprimatur of a profession or an organisation. Thus, the use of appropriate letterhead or logos on correspondence, envelopes, web pages and cover letters add to this overall impression of professionalism. If the questionnaire is seen to be under the guidance of a professional person (such as your supervisor) or research-oriented organisation (such as a market research company or a university), this will add to the overall impression of professionalism as well. • Instructions and cover letters need to be clear, simple and inviting: When designing an instrument, you must also attend to the materials that will accompany your questionnaire such as the cover letter or statement inviting participation (and, perhaps, for obtaining informed consent), the instructions for the questionnaire and for all relevant sections, and the closing material at the end of the questionnaire. Your cover letter, if designed well, will get your ‘foot in the door’ with a participant; if designed poorly, will make it easy for a participant to decline. The cover letter has several functions: to establish your credentials, set out the purpose for the questionnaire and why you are approaching the person to participate, specify all protections that will be provide for their identity and responses,

822

18 When and How Should I Deal with Measurements?

indicate what ethical approvals have been obtained, specify how to return or submit the questionnaire, specify about how much time the questionnaire should take to complete (a pilot test can help establish this) and when you would like to have their responses returned by. If you are offering an incentive for their participation (e.g., entry into a draw for a holiday), this needs to be spelled out as well. The letter should be semi-formal, highly readable and free of jargon, and should appear on official letterhead, if possible. A counter-signature from your supervisor will add to the professional impression. The instructions for every part of the questionnaire must be clear, highly readable and, again, free of jargon. Participants must know, every step of the way, what is expected of them and how they are to respond. They need to know if only one or more than one category can be ticked for an item. They may need to know what they should be thinking about (i.e., their mind set) as they respond. They may need to know how they should allocate numbers to options if that is demanded of them. If items are response-keyed (i.e., answering a question ‘yes’ means they should go to question X; if ‘no’ they should go to question Y), then you need to ensure that all possible responses will guide participants clearly through the entire questionnaire without skipping vital parts. The closing statement on the questionnaire should thank people for their participation and reiterate how the questionnaire should be returned. • Cross-cultural instruments in languages other than English need special attention: If you anticipate administering your instrument to a multi-cultural sample or simply to people from a non-English speaking culture in a Cross-Cultural research frame, great care needs to be taken in the design of every facet of the instrument and its surrounding materials. For a questionnaire to be administered in a multi-cultural context, but where participants are expected to have some competency in the English language, your questionnaire needs to be written to cater to a wide range of reading levels and skills. You need to be careful that the content of all questions will be suitable (as in not being confusing or offensive) for any potential cultural groups in your sample. For a questionnaire that is to be administered in a language other than English, you need to plan a series of steps in your research methodology for constructing and ensuring the equivalence of all versions of the questionnaire and this will invariably add to your research timeline. Suppose you plan a questionnaire to go to people in Australian and Chinese organisations and you want to administer the questionnaire in both English and Mandarin. One useful strategy for developing the Mandarin version is to write the instrument and all accompanying materials (instructions, cover letters) in English first, then get an expert in both English and Mandarin to translate them into Mandarin. Once done, get another expert in English and Mandarin to translate the Mandarin version back into English. If the two English versions agree, then you can be assured of the equivalence of the instrument and accompanying materials. This is a process called back translation. Where equivalence is not achieved, you will need to look carefully at whether you have assumed equivalence in concepts across cultures where it does not exist, or you have inadvertently used jargon or a mode of expression for concepts that have no counterparts in the other language.

18.6

Developing and Validating Your Own Measure

823

• Pilot test and validate the instrument: No matter how carefully you have designed a questionnaire and its accompanying materials, you can be sure there will be bugs somewhere. That is why it is critical to pilot test and validate any instrument you develop yourself. Developing your own measures means that you must plan for your methodology to include stages that will permit you to: (1) pilot test drafts of your instrument and all accompanying materials (e.g., cover letters, instructions), (2) solicit feedback from participants on those drafts; (3) revise the instrument and accompanying materials based on that feedback and potentially re-test the modified versions of them; and (4) validate the final instrument, either in a new sample or as part of your analytical strategy for your main sample. For your pilot test, you should use participants that are like those you will approach to participate in your main study. You don’t need a huge sample, just the right kind of people in the sample. Plan to interview every pilot test participant after they complete your questionnaire and probe them about issues such as: how easy it was for them to complete the instrument, were they offended or confused by any parts, were the instructions clear or confusing, impressions of length, appearance, readability and complexity, and their views on the cover letter.

18.6.2 Issues to Attend to Prior to Use in Analyses • Identify items to be reverse-scored: Prior to any validation analysis, you will need to reverse score any items that you designed to be reverse-worded. SPSS, for example, can do this easily with the Compute Variables … function. The simplest way to reverse score an item is to create a new variable by subtracting the old variable value from a number that is one plus the highest category number or measurement on the scale. So, for a 7-item Likert-type scale, you can reverse the scores by computing 8 minus the original rating for every participant. Sometimes, however, you may not be able to anticipate whether an item should be reverse-scored because this might depend upon the context and construct for which it is used. In such cases, exploratory factor analysis will quickly reveal the need to reverse score an item through a high negative factor loading for that item on the factor it defines. Simply reverse score the variable(s) and rerun the factor analysis; the signs will be reversed, and you will have the new variables you need to score participants with. • Validate the instrument: Thoroughly explore the behaviour of the items on your instrument using exploratory factor analysis. For validation purposes, you will need at least 5 times (preferably 10 times) as many participants in your sample as you have items on your instrument. It may take you several runs of factor analysis to get a clean version of your instrument (deleting poorly performing items each time). If you deliberately designed a sub-construct structure into your pool of items, then you will want to see if that ‘designed-in’ structure emerges in the analysis; if it does, that adds weight to your evidence for construct validity. Once

824

18 When and How Should I Deal with Measurements?

you have established the factor structure of your items, you will need to clearly interpret what each factor represents so that all subsequent analyses using scores on those factors can be clearly and unambiguously interpreted. You will need to run reliability analyses on the resulting factors and then score your participants on those factors in a fashion similar to if you had adopted someone else’s measure. Some researchers argue that you should not only run exploratory factor analyses to validate a newly developed instrument, but also a confirmatory factor analysis to ‘confirm’ the final structure. It is poor practice to undertake such a strategy using the same sample for both analyses. The better practice is to obtain one sample on which the exploratory analyses will be run to identify the initial factor structure, and then use a second sample to confirm that structure. If gathering two separate samples is not feasible, then you can opt to draw a very large sample to start with (double the size you would normally need) and randomly divide that sample into two sub-samples of equal size. Run the exploratory factor analyses on one sub-sample and the confirmatory factor analyses on the other sample—a process called cross-validation. Cooksey (2014, Appendix B) explores all these issues more fully.

18.7

Key Recommendations

We have covered a lot of territory in one short chapter, but there are a few key things to keep in mind as you consider measurement issues as they potentially intersect your data gathering strategies. • Measurement is one aspect of data gathering that has some definite anchors to specific guiding assumptions, namely the positivist pattern of guiding assumptions. If you plan to use quantitative data in positivist research, then you must attend closely to the requirements of good measurement practices to build toward convincing research conclusions. Those requirements include: – sound operational definition of constructs where the process for translating a construct from its theoretical status to concrete measurement procedures are clearly thought through and spelled out; – appropriate development and measurement design processes and choices where level of measurement, scaling process (if any), procedures for controlling for various types of potential response biases, procedures for ensuring cultural equivalence and meaningfulness and appropriate pilot testing (especially if you develop the measurement instrument yourself) are carefully worked through; – demonstration of appropriate type(s) of validity, construct validity being most crucial, where logical, empirical, statistical and/or predictive approaches are implemented to help you make the arguments; and – demonstration of reliability where appropriate indicators are reported (e.g. Cronbach’s alpha measure of internal consistency) but not in lieu of or as a surrogate for information about validity.

18.7

Key Recommendations

825

• Quantitative data gathering may occur in the context of research guided by an interpretivist/constructivist pattern of assumptions, but generally not with the intention of ‘measuring’ anything. More than likely, you would simply be counting things as a way of characterising attributes of participants, prevalence of a theme, extent to which a particular perspective appears to be shared, and so on. • When adopting measurement instruments developed by other researchers, you should not simply take their instrument as a given. Ask critical questions about the instrument’s quality (in terms of validity, reliability and sensitivity—look at both the pros and cons of the instrument to make your decision), its suitability for your research context(s) and intended participants, what permissions are needed to use the instrument and how to produce scores on the instrument and interpret what those scores might mean. In most cases, some pilot testing and re-validation of the instrument in your own context(s) will be essential to make your research arguments convincing. • When designing and developing your own measurement instrument, you cannot simply write a bunch of items, put them together and use the resulting collection as your ‘instrument’. The onus is on you to implement sound measurement and instrument design principles in order to ensure good instructions, sound item design, clear response requirements, attractive but professional overall appearance as well as to demonstrate the psychometric qualities (e.g. validity and reliability) of the instrument you design (DeVellis, 2016, provides excellent guidance in this regard). A pilot test is essential if you use self-developed measures and you need to clearly defend your instrument construction and scaling choices in areas where debate exists as to the best way forward. • Remember that, in the end, your measurements must work from the participant’s perspective. If participants don’t take your instrument seriously or if your measurement process is too complex or convoluted, you can have little faith in the quality of the resulting data. You will also probably suffer a reduced response rate by way of reaction to a poorly designed instrument. Remember also, that participants may make their judgments about whether they will take their participation seriously based on some very surface cues, which is why instrument length, clarity of response requirements and instructions, readability and even layout details such as line spacing, font type and size are important to look after.

References Alhosany, A. (2011). Exploring organisational citizenship behaviour in the federal hospitals in the United Arab Emirates: A cross-cultural research study. Unpublished Doctor of Health Services Management thesis, School of Health, University of New England, Armidale, NSW, Australia. Allen, N. J., & Meyer, J. P. (1990). The measurement and antecedents of affective, continuance and normative commitment to the organization. Journal of Occupational Psychology, 63(1), 1–18. Andrich, D. (2005). Rasch models for ordered response categories. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 4, pp. 1698–1707). Chichester, UK: Wiley.

826

18 When and How Should I Deal with Measurements?

Athanasou, J. A. (1997). Introduction to educational testing. Katoomba, NSW: Social Science Press. Bergkvist, L., & Rossiter, J. R. (2007). The predictive validity of multiple-item versus single-item measures of the same constructs. Journal of Marketing Research, 44(2), 175–184. Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum Associates. Brewer, J., & Hunter, A. (2006). Foundations of multimethod research: Synthesizing styles. Thousand Oaks, CA: Sage Publications. Bryman, A., & Cramer, D. (2004). Constructing variables. In M. Hardy & A. Bryman (Eds.), Handbook of data analysis (pp. 17–34). London: Sage Publications. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Thousand Oaks, CA: Sage Publications. Cooksey, R. W. (2014). Illustrating statistical procedures: Finding meaning in quantitative data (2nd ed.). Prahran, Victoria: Tilde University Press. de Vaus, D. (2002). Analyzing social science data: 50 key problems in data analysis. London: Sage Publications. DeVellis, R. F. (2016). Scale development: Theory and applications. Los Angeles: Sage Publications. Fischer, D. G., & Fick, C. (1993). Measuring social desirability: Short forms of the Marlowe-Crowne Social Desirability scale. Educational and Psychological Measurement, 53(2), 417–424. Fogarty, G. J. (2008). Principles and applications of educational and psychological testing. In J. A. Athanasou (Ed.), Adult educational psychology (2nd ed., pp. 351–384). Rotterdam, Netherlands: Sense Publishers. Gardner, D. G., Cummings, L. L., Dunham, R. B., & Pierce, J. L. (1998). Single-item versus multiple-item measurement scales: An empirical comparison. Educational and Psychological Measurement, 58(6), 898–915. Hattie, J. A., & Cooksey, R. W. (1984). Procedures for assessing the validity of tests using the “known groups” method. Applied Psychological Measurement, 8, 295–305. King, L. A., & King, D. W. (1990). Role conflict and ambiguity: A critical assessment of construct validity. Psychological Bulletin, 107(1), 48–64. Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41 (4), 212–218. Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537–567. Lamprianou, I. (2008). Introduction to psychometrics. The case of Rasch models. In J. A. Athanasou (Ed.), Adult educational psychology (2nd ed., pp. 385–418). Rotterdam, Netherlands: Sense Publishers. Lee, J. A., Soutar, G., & Louviere, J. (2008). The best–worst scaling approach: An alternative to Schwartz’s values survey. Journal of Personality Assessment, 90(4), 335–347. National Institute of Standards and Technology (NIST). (2001). SI units: Length. Retrieved March 29, 2018, from https://www.nist.gov/pml/weights-and-measures/si-units-length. Ostini, R., & Nering, M. L. (2006). Polytomous item response theory models. Thousand Oaks: CA Sage Publications. Rizzo, J. R., House, R. J., & Lirtzman, S. I. (1970). Role conflict and ambiguity in complex organizations. Administrative Science Quarterly, 15(2), 150–163. Robinson, S. B., & Leonard, K. F. (2019). Designing quality survey questions. Los Angeles: Sage Publications. Zumbo, B. D., & Rupp, A. A. (2004). Responsible modeling of measurement data for appropriate inferences: Important advances in reliability and validity theory. In D. Kaplan (Ed.), The Sage Handbook of quantitative methodology for the social sciences (pp. 73–92). Thousand Oaks, CA: Sage Publications.

Chapter 19

How Do I Manage the Sampling Process?

19.1

Key Considerations Surrounding Sampling

Sampling is about making choices related to data sources that will be the focus of your data gathering strategies. This includes choices of the data sources themselves as well as choices of the circumstances, contexts and occasions in which data sources are encountered. The most common misconception about sampling in research is that it refers only to selecting individual participants as data sources for research purposes. In social and behavioural research, sampling refers to much more than selecting human participants, especially when patterns of guiding assumptions, specific research configurations and specific data gathering strategies are considered in conjunction with the research context, research frames and associated research questions and/or hypotheses. Beyond simply choosing which individuals you intend to invite to participate in your research, sampling may concern: • how you select organisations and/or groups to approach for participation in case studies and other types of field-based or naturalistic research (may be relevant under a variety of patterns of guiding assumptions); • how you choose which handiworks, documents (e.g. text, digital, multi-media) or other artefacts to collect/read/view/analyse in your research (may also be relevant under a variety of patterns of guiding assumptions); • how you choose which contexts, situations, times, places and/or events to observe/participate in and which conversations to engage in or record (especially relevant in interpretivist/constructivist research; for example, theoretical sampling issues for grounded theory would be relevant here); and • how you choose which participants to use as examples to illustrate/support your interpretations as well as how you choose which quotations or paraphrases to report (specifically relevant in interpretivist/constructivist research).

© Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_19

827

828

19

How Do I Manage the Sampling Process?

It is important to note that in social and behavioural research, sampling human participants adds much more complexity to your research process, compared to sampling non-human data sources and artefacts. This complexity arises because human participants, acting as data sources, are motivated, perceiving, thinking, feeling beings and they don’t cease being so when they participate in your research. Rather it is their motivations, perceptions, cognitions and feelings that you must account for in your sampling plans and, downstream, your data gathering activities. Human participants may act and react positively or negatively toward you and the research process. Positive actions would include volunteering to or agreeing to participate; positive reactions would include persisting in their participation role until completed (i.e., not withdraw from your research once they have agreed to participate). It is in your best interests to try and ensure that positive actions and reactions are more likely to occur as this will result in higher rates of participation. However, even positive actions and reactions can have negative consequences if a participant’s motivation for volunteering or agreeing to participate runs contrary to their provision of high quality valid or authentic data. Negative actions would include declining to participate or refusing to volunteer, which ethical principles dictate people have a right to do and you, as the researcher, have the obligation to protect. Negative reactions can take on many forms including active sabotage of results (i.e., deliberately providing poor or misleading information, which means you will have to throw data away and reduce your sample size accordingly), withdrawal from participation once it has commenced (another ethical right that must be protected), elevated stress or emotional levels (an ethical right for participants to avoid) and/or attempting to anticipate what you hope to find and trying to give you information consistent with that expectation (what are often called demand characteristics in research guided by the positivist pattern of assumptions). Obviously, you should try to minimise these potential negative actions and reactions as the last thing you need is your data quality and participation rates to suffer because of something you do to participants in your research or because of something that participants think they have figured out you are after. Your sampling strategy should maximise the chances of accessing the data you need to address your research questions and/or hypotheses. A sampling strategy can strongly influence how convincing you can be in addressing research questions or hypotheses and how far you can extend your findings beyond the specific boundaries of your study (recall the extensional reasoning meta-criterion discussed in Chap. 9). Sampling questions have one of two main interests (which may or may not overlap): (1) sample size and (2) sample composition. In research that gathers quantitative data guided by the positivist pattern of assumptions, both sample size and sample composition are important to consider: sample size because it influences statistical efficiency, effectiveness and stability (larger samples are better for these purposes); composition because it influences the extent to which defensible generalisations to the population can be made. Sample size is much less of a concern in research that gathers qualitative data guided by an interpretivist/constructivist pattern of assumptions. In this case, it is not how large the sample is that wins the day but how, who, when, where and what has been sampled. Positivist researchers often

19.1

Key Considerations Surrounding Sampling

829

make the mistake of criticising interpretivist/constructivist research for having samples that are too small. However, this is a misplaced criticism because it imposes the positivist value of requiring larger samples for more precise statistical estimates and stronger generalising capability on a research context where these processes have little or no meaning. Samples can be very small (even one person or event) in interpretivist/constructivist research and still be sufficient to provide a convincing story. Thus, what counts as a ‘good’ sample depends upon pattern of guiding assumptions, type of data gathered, type of research configuration and associated data gathering strategies being implemented and research questions and/ or hypotheses. We can identify several focal emphases that can potentially influence the nature of the sampling strategy you employ. In the following discussion, we not only review each focal emphasis, but also foreshadow how that focus might relate to specific choices of sampling strategy, which will be discussed in more detail later in this chapter.

19.1.1 Focus on Statistical Efficiency = Precision Here, you use a sample of a size sufficient to provide statistical information about the population within a specified margin of error. Statisticians and polling organisations tend to focus on the precision of their estimates in their sampling logic. The focus on precision implicates extensional reasoning (via external validity—generalising statistical information from sample to the population). In terms of planning research, this translates into a decision about the size and nature of the sample required. The basic question is how to achieve the desired precision most efficiently. It is important to differentiate between the population of interest, which is the ultimate target of your generalisations (e.g., all employees who have ever worked in a specific industry) and the sampling frame (see Schaefer, Mendenhall, & Ott, 2012; not to be confused with a research frame), which is that subset of the population you have practical access to for drawing your sample (e.g., all employees currently working in a specific industry in a specific city). Technically, you sample from a sampling frame, not from the population at large. To obtain a sample with precision as the focus, you need to understand a few fundamental concepts about the statistical estimation of population parameters. The first fundamental concept involves terminology: (1) a statistic is defined as a numerical measure that summarises some characteristic or relationship in a sample (such a proportion or percentage of people ticking yes to a question, a mean, a variance or standard deviation, a difference between two means, a correlation, a regression coefficient); (2) a parameter is defined as the corresponding unknown value for that characteristic or relationship in the population from which the sample was drawn; and

830

19

How Do I Manage the Sampling Process?

(3) a random sample is one where each member of the population has an equal chance of being drawn for the sample and where each sample of a given size has an equal chance of being drawn. We therefore use random sampling to yield information (in the form of calculated statistics) to estimate, i.e., make inferences about, unknown parameter values in the population. Inferences are thus made from sample statistics to population parameters. The converse of precision in statistical estimation is error (i.e., more precision equals less error and vice versa) and to increase precision, we focus on reducing errors. As we noted in Chap. 18 with respect to statistical models, any statistical estimation process can make errors and error in statistical estimation comprises two components: (1) a random (or unsystematic) error component, which encompasses completely unpredictable and idiosyncratic errors, often reflecting the vagaries and complexities of human life; and (2) a systematic error component, which is potentially controllable, but which, if left uncontrolled, remain inextricably conflated with the random error component, driving the total amount of error upward. In positivist research gathering quantitative data, greater statistical precision is generally easier to achieve if a random sampling process is used to select the data sources from which data are gathered. Random sampling enhances precision by minimising systematic error and ensures that the rules of probability, rather than your own biases and preferences, influence who ends up in your sample. Thus, random sampling is a form of control procedure that has the goal of trying to ensure that any characteristics of data sources cannot influence your sampling process. This then reduces systematic error, thereby shrinking the total amount of error associated with statistical estimates. The outcome is that your statistical estimates are more precise. The second fundamental concept is called the confidence level. The confidence level is set by you, as the researcher, and quantifies the likelihood of you drawing a correct inference about the accuracy of a random sample statistic as an estimate of the value of a population parameter. The confidence level can take a value anywhere between 0 and 100% but is typically set at a high value such as 90, 95 or 99%. [Interestingly and conversely, 100% minus the confidence level quantifies the risk of making an error, called the Type 1 or alpha (a) error, in claiming that a random sample statistic is an accurate estimate for the value of a population parameter.] The third fundamental concept is called the confidence interval and defines the upper and lower bounds for how far away a statistical estimate might be from its corresponding parameter value in the population (usually expressed as some statistic plus or minus an amount of error, as for example, when a political poll shows that 52% of voters would vote Labor in an election, plus or minus 3%). Thus, a confidence interval signals the level of precision you desire in terms of how close your random sample statistical estimates are to their corresponding population parameter values. The confidence level signals the likelihood that the population parameter value falls between the upper and lower limits of the confidence interval.

19.1

Key Considerations Surrounding Sampling

831

Both the confidence level and confidence interval are conscious choices you make when considering sample size. In Fig. 19.1, we provide a conceptual display of the relationships between these three fundamental concepts. Each circle, with a slice missing, signals a confidence level you might desire (we show 90% (diagrams a, b and c), 95% (diagrams d, e and f) and 99% (diagrams g, h and i) as the choices, but other choices are also possible, depending upon your needs). The desired confidence level is shown by the black double-headed arrow spanning the distance between vertical lines (i.e., the upper and lower boundaries). The X in each diagram depicts where the relevant population parameter might fall. For the diagrams in Fig. 19.1, the population and sampling frame are shown as fixed sizes (light and medium colour rounded

90%

(a)

95%

(d)

X

99%

(g)

X

5%

X

5%

5%

Population

Population

Population

Sampling Frame

Sampling Frame

Sampling Frame

(b)

90%

X

(e)

95%

X

4%

Population Sampling Frame

90%

(h)

99%

X 4%

4%

Population

Population

Sampling Frame

Sampling Frame

(f)

95%

(i)

99%

X

X

X

3%

3%

3%

Population

Population

Population

Sampling Frame

Sampling Frame

Sampling Frame

Sample

Sample

Sample

Sample

(c)

Sample

Sample

Sample

Sample

Sample

Fig. 19.1 Conceptualising the relationship between sample size, confidence levels and confidence intervals for random samples from a population of a fixed size

832

19

How Do I Manage the Sampling Process?

rectangles, respectively) and the sample is shown as a dark rounded rectangle that varies in size inside the population. The basic pattern can be expressed as follows: with a given confidence level, a larger sample is needed to achieve a narrower confidence interval (= more precision); with a given confidence level, a larger sample size is needed to achieve a more certain confidence level. Computational procedures, often with the aid of a computer support system, are used to estimate the size of sample required to achieve a certain level of precision with a specific level of confidence, but their use requires often some intelligent simplifying guesses about the variability in measurements. To simplify things, some authors (e.g., Bartlett, Kotrlik, & Higgins 2001; Cohen, Manion, & Morrison, 2011, p. 147; Gill & Johnson, 2010, p. 130; Saunders, Lewis, & Thornhill, 2012, p. 266) have produced tables and simplified formulas to help with selecting a desired sample size.

19.1.2 Focus on Statistical Effectiveness = Power A focus on precision is most appropriate if your interest is in the precision of point estimates (statistic to parameter, especially important in polls). However, if your intention is to demonstrate or test for the existence of a relationship (e.g., group differences, correlation, prediction; generically termed an effect), then the precision focus is generally not adequate, especially for stakeholders who may be interested in or wish to use/apply your research findings (such as doctors interested in the outcomes of a clinical drug trial). Behavioural and medical researchers tend to focus on power in their sampling logic. When demonstrating or testing for a relationship, you are more interested in having a high likelihood of finding evidence for your hypothesised relationship, using random sample information, if that relationship truly exists in the population. This is the question of power and it changes the focus from efficiency in estimation (have we achieved a precise enough estimate?) to effectiveness in estimation (have I found a genuine relationship?). The power focus is still predicated on the use of random sampling, but we now look at setting sample size at a level that will provide enough power to detect an effect of a desired size. A useful metaphor for statistical power is a microscope. You will not be able to see details in an object of a certain size unless your microscope has enough magnification (i.e., enough power) for resolving details of that size or smaller. The focus on power implicates the internal coherence (via internal validity —demonstrating the desired relationship(s)) as well as the extensional reasoning (via external validity—generalising relationships from sample to population) meta-criteria. Figure 19.2 provides a conceptual visualisation of the relationship between power, sample size and effect size (which is generically referred to, in standardised form, as d in the literature; e.g., see Cohen, 1988, 1992; Smithson, 2000). The population size is fixed in all diagrams and the effect (black line with end points) line length corresponds to effect size, d. Sample sizes vary according to conditions (symbolised by the dark rounded rectangles) and if an effect falls outside

19.1

Key Considerations Surrounding Sampling

833

Sufficient power Population large effect size (d)

Sampling Frame

Sample

d

Sufficient power

Insufficient power

Population

Population medium effect size (d)

Sampling Frame

Sample

Sampling Frame

Sample

d

d

Insufficient power

Insufficient power Population small effect size (d)

Sampling Frame

Population Sample

d

Sampling Frame

Sufficient power Population

Sample

d

Sampling Frame

Sample

d

Fig. 19.2 Conceptualising the relationship between sample size, desired effect size detection and power for random samples from a population of a fixed size

of the sample symbol, then the effect cannot be detected by information from the sample. The general pattern is that progressively larger samples are required to have enough power to detect progressively smaller effect sizes. For greater depth of discussion of statistical power and its implications for research gathering quantitative data, see Cohen (1988), considered the classic text on the subject. Computational procedures, often with the aid of a computer support system, are used to make power and sample size calculations (e.g., G*power; SYSTAT; SPSS SamplePower; PASS; Stata; for online power and sample size calculators, see http://powerandsamplesize.com/). The use of such systems generally requires an intelligent guess about the effect size you want to detect. A reasonable guess may be gleaned from prior literature, stakeholder interests and/or your hunches/ expectations. A well-known and often-used statistical package for performing power calculations is G*Power 3 (a free downloadable program for Windows or Mac, from http://www.gpower.hhu.de/en.html; see also Faul, Erdfelder, Lang, & Buchner, 2007; Faul et al., 2009). This program can compute the desired total sample from given inputs of desired power level, confidence level and effect size for a wide variety of statistical tests and procedures (what is termed ‘apriori power analysis’ in G*Power 3). Figure 19.3 shows a screenshot of the G*Power 3 interface and shows the results of calculating the minimum required sample sizes for a two-group comparison study where the input parameters (lower left side of the screenshot) are the desired power level (0.95), the desired alpha error (= 1 minus the desired confidence level of 0.95 = 0.05) and the desired effect size to be

834

19

How Do I Manage the Sampling Process?

Fig. 19.3 Illustrative screenshot from the G*Power 3 software package for power analysis, showing calculations of required sample sizes for a two-group comparison study

detected (0.5 = medium effect size) [the allocation ratio refers to the ratio of the sizes of the two groups, which, in the example, is 1, meaning that the groups are to be of equal size]. The key outputs (lower right side of screenshot) from the calculation are the required sizes of each group, which is 105, yielding a required overall sample size of 210. G*Power 3 can also compute the evident power level from published research investigations (what is referred to as ‘posthoc power analysis’ in G*Power 3). It has now become a routine expectation that researchers use power analysis to make informed choices about sample sizes in their proposals to funding organisations and research ethics committees, especially for health and

19.1

Key Considerations Surrounding Sampling

835

medical research, and G*Power is a frequently used and well-accepted tool for informing such choices (see, for example, Cunningham & McCrum-Gardner, 2007, for a discussion of the UK research ethics oversight context). Some authors (e.g., Cohen, 1988, 1992; Smithson, 2000, p. 207) have produced simplified tables for roughly establishing the desirable random sample size assuming a specific confidence level and size of effect to be detected. Note that, like the focus on precision, we still consider the confidence level desired (which directly relates to the level of risk or Type 1 error attached to incorrectly concluding that random sample information accurately reflects a pattern in the population), but instead of a confidence interval, the emphasis shifts to the size of effect to be detected. There are some general relationships between effect size, sample size, confidence level and power: • if you want to detect a small effect, you will need a larger sample size, assuming your confidence level and power remain constant; • if you want to have a better chance of finding significant effects (i.e. more power), you will need a larger sample size, assuming your confidence level and desired effect size remain constant; • if you want stronger protection against false claims of significance (i.e. a higher confidence level), you will need a larger sample size, assuming your power and desired effect size remain constant; and • in general, your sample size requirements dramatically drop when your focus shifts toward detecting larger effect sizes. One interesting dilemma with the power focus is that it is possible to have too much power conferred by very large sample sizes. What this means is that with a large enough sample, even a minute effect size will be detected as significant, even though, in practical terms, the effect is virtually meaningless if someone plans to make policy or program changes or innovation adoptions based on your research outcomes. The dilemma hinges on the distinction between statistical significance (i.e., a difference or relationship exists in the population from which the sample was drawn) and practical utility (i.e., the difference or relationship is large enough to do something useful with). When samples get very large, statistical significance wins out over practical utility and this can have downstream adverse implications (including increased costs or wasted resources) for certain stakeholders who might wish to make social policy changes or adopt innovations based on what you find. For example, if, in a very large company, you have a random sample of 2500 employee responses to a 5-point Likert-type attitude item on a questionnaire (say, an item that asks about the degree of satisfaction the participant has with respect to their current salary: 1 = not at all satisfied; 5 = extremely satisfied), it is conceivable that a statistically significant difference between males and females of 0.03 of a scale point (in favour of males) could be found. The question this raises is ‘would it be worthwhile investing resources in a change in salary policy that would close this statistically significant gap between the sexes?’. The company’s CEO would probably say no, given the miniscule difference evident. Thus, significance does not

836

19

How Do I Manage the Sampling Process?

equate with utility. You can have a statistically significant finding that is either practically useful (best case scenario, showing a reasonable effect size) or not practically useful (worst case scenario, showing a very small effect size). Conversely, however, you cannot have a practically useful finding that is not also statistically significant. If the practical utility of your research findings using quantitative data is important (as it might be in an Evaluation, Developmental Evaluation, Transdisciplinary or Action research frame), then the power focus is to be preferred over the precision focus in sampling.

19.1.3 Focus on Statistical and Computational Viability and Stability For more complex research configurations involving the quantitative measurement of multiple constructs (often referred to as multivariate research), social and behavioural researchers typically need to shift their sampling focus from statistical efficiency/effectiveness toward a focus on sampling to maximise viability and stability of statistical estimates. Accordingly, the size of a random sample is set at a level that will more likely yield stable statistical estimates and viable solutions for the models being tested. In some instances, if your sample size is too small, the analytical procedure you are using may not be able to achieve a viable or interpretable solution. This means that sample size considerations are driven equally by power or precision concerns (which still play a part in the larger scheme of things) and by rules of thumb or heuristics that signal a desirable ratio of cases/participants to variables/predictors in the model being tested (e.g., 5 to 1, 10 to 1 or 20 to 1). Hair, Black, Babin, and Anderson (2010) suggested a heuristic minimum of a 10 to 1 ratio of cases/participants to variables for many types of multivariate analyses (including multiple regression, factor analysis and structural equation modelling) and a preferred ratio of at least 20 to 1. This means that if your questionnaire has 30 attitude items that will be analysed together (for example, in a factor analysis), your minimum sample size would be 10
30 or 300; your preferred sample size would be 20 * 30 or 600. These authors argued that, in multiple regression modelling, a ratio of 5 to 1 could be workable, but would not be desirable. van Voorhis and Morgan (2007) suggested that for group-based comparisons, the minimum sample size for each group should be 30; for regression and correlation analyses, sample size should be based on a 10 to 1 ratio of cases/participants to predictors; and for factor analyses, a sample size of at least 300 would be considered “good”. Green (1991) argued that for testing a regression model containing k independent variables or predictors, an appropriate minimum sample size would be found as 50 + 8 k; if individual predictors in a regression model are to be tested, then an appropriate minimum sample size would be found as 104 + k (if both tasks are to be done, run both calculations and take the larger of the two numbers as the minimum sample size). In general, the safest heuristic to use is a minimum ratio of 20 to 1 for any

19.1

Key Considerations Surrounding Sampling

837

research involving modelling with multiple variables. The viability/stability focus implicates the internal coherence (via internal validity—demonstrating the desired relationship(s)) as well as the analytical integrity (quality of analytical process) meta-criteria.

19.1.4 Focus on Representativeness of Sample This consideration shifts the emphasis from sample size to sample composition. In quantitative research guided by the positivist pattern of guiding assumptions, a sample can be considered representative of some population of interest if it is statistically comparable to that population in terms of the distributions of all relevant characteristics of participants. What counts as a relevant characteristic is one that (1) can potentially influence the patterns of measurements that are obtained, over and above those variables that you explicitly wish to examine (an internal validity concern), or (2) imposes constraints on the extent of permissible generalisations (an external validity concern). In other words, seeking a representative sample constitutes a form of procedural control, where appropriate sample composition helps to control for the influence of certain alternative plausible explanations for observed patterns of results or for certain limits on generalisation. Figure 19.4 visually depicts the concept of representativeness, from a positivist perspective. Assume the population (larger circle with lighter textured sectors) has members that belong to one of four mutually exclusive categories (e.g., these could be income brackets, ethnic or cultural backgrounds, political affiliations, industries or geographical regions). Ideally, a representative sampling frame should be identifiable within this population (smaller circle with darker textured sectors). A randomly drawn representative sample from the sampling frame should appear like circle (a) in Fig. 19.4 (where the proportions of sample members in each

(a)

Representative Sample

(b)

Sampling Frame Population

Fig. 19.4 Illustration of representative versus non-representative samples

Non-Representative Sample

838

19

How Do I Manage the Sampling Process?

textured area match the proportions in the sampling frame/population). An undesirable non-representative sample is illustrated in circle (b) in Fig. 19.4, which contains much greater proportions of ‘woven texture’ and ‘granite texture’ members relative to the sampling frame/population and, correspondingly, much smaller proportions of ‘wood grain texture’ and ‘water droplets texture’ members. Interestingly, while random sampling is ideally intended to produce a representative sample, it does not always do so. By chance alone, an unrepresentative sample like circle (b) in Fig. 19.4 can result from a random sampling process. This is the primary reason why alternative sampling schemes such as stratified random sampling or quota sampling have been developed. They are intended to enhance sample representativeness by requiring you to sample separately within each relevant category of population member. The representativeness of sample focus implicates the internal coherence (via internal validity—demonstrating the desired relationship (s)) and the extensional reasoning (via external validity—generalising relationships from sample to population) meta-criteria. Under positivist assumptions, the representativeness of a sample is an important determiner/delimiter of generalisability from the sample to some population of interest. The more representative the sample, the stronger the case for generalisability. From the perspective of an interpretivist/constructivist pattern of guiding assumptions, representativeness helps to address extensional reasoning via the sufficiency quality criterion through having you focus on ensuring that important/ relevant/marginalised/silenced/alternative voices, perspectives and/or data sources are sought and heard during your research journey. The goal here is to ensure that you broaden your focus beyond the majority or dominant perspectives you may be exposed to so as to achieve a more well-rounded and nuanced understanding (purposive and theoretical sampling schemes can help in this regard). Under an interpretivist/constructivist pattern of guiding assumptions, the representativeness of sample focus implicates the extensional reasoning (via the quality criterion of sufficiency) as well as the internal coherence (via the quality criterion of authenticity) meta-criteria.

19.1.5 Focus on Representativeness of Experiences This consideration is specific to positivist research gathering quantitative data involving human participants and to data gathering strategies that provide structured experiences for people. It shifts the representativeness focus from the sampling of participants to the sampling of experiences (i.e., tasks, conditions or situations) that sample members will experience during their participation in your research. This is an especially important issue to consider in laboratory-based research and in research intended to impose control over context in field settings. Brunswik (1952, 1956; see also Hammond & Wascoe, 1980, for discussion and examples) persuasively argued that positivist researchers, who employ the Manipulative, Structuring or Immersive experienced-focused strategies that offer structured experiences for

19.1

Key Considerations Surrounding Sampling

839

people, should design the tasks, conditions and situations under which research participants will be quantitatively measured to be representative of (i.e. similar to, in all relevant features) those tasks, conditions and situations the participants would likely encounter in their normal life or work circumstances. Such tasks/conditions could include: decisions to make (e.g., which job applicants to make redundant, hire or promote), problems to solve (e.g., diagnosing medical illnesses; assessing the threat level of an unidentified aircraft), evaluations, judgments or forecasts to make (e.g., grading students’ homework, forecasting severe hail or wind shear at airports), preferences to rate (e.g., preferred products to purchase or apartments to rent), work tasks to complete (e.g., performance appraisals), tasks conducted under stressful conditions (e.g., flying an aircraft through severe weather or during an emergency event; identifying enemy aircraft by radar under battle conditions) or tasks with deadlines or with conflicting, inaccurate or missing information (e.g., deciding upon guilt or innocence of a defendant, where testimonies from witnesses conflict). Egon Brunswik called this principle representative design and argued that it was as important—and in some cases, more important—for researchers to focus on achieving this than a representative sample. This is because generalisations from artificially highly-controlled tasks, undertaken under artificially manipulated conditions (as in a laboratory experiment), to the ‘real world’, with all its complexities and messiness, in which participants must live and work are very hard to defend. The central philosophy behind representative design is understanding how each individual person behaves under conditions and in situations that approximate what they would encounter in their normal lifespace before attempting to generalise behavioural patterns to other individuals. [Brunswik, 1952, argued this implies a need to analyse the behaviour of each individual in a sample (idiographic analysis) across a sample of representatively designed tasks or situations before making generalising inferences (nomothetic generalisation).] Figure 19.5 provides a visual illustration of three possible representative design strategies. Strategy 1 for implementing representative design principles in your research is to sample a range of tasks, conditions and/or situations from participants’ natural environments, each having different features, (generically represented by different font/shape characteristics of each experience ‘letter’ in Fig. 19.5) and then use that sample to provide the research experiences for participants. Strategy 2 for implementing representative design principles in your research is to statistically assess the essential features/characteristics/properties of a sample of tasks, conditions and/or situations from participants’ natural environments (generically labelled according to specific font/shape characteristics of each experience ‘letter’ in Fig. 19.5), then use those characteristics/properties to construct realistic but simulated research experiences for participants (represented by the font/shape with a dashed outline to indicate a simulated rather than a sampled real experience). Strategy 3 for implementing representative design principles in your research involves the acquisition and application of expertise regarding and/or experience in tasks that are relevant to the lives of potential research participants. Implementing this strategy usually requires a separate investigation that observes and analyses (e.g., using task analysis or cognitive task analysis, see, for example, Drury, 1983; Schraagen,

840

19

How Do I Manage the Sampling Process?

h

Experiences in Participants’ Life/Work Contexts

A

h

e d

C

e

C

Key Features 1 – ‘shape’ 2 – ‘background’ 3 – ‘size’ 4 – ‘underscored’ 5 – ‘bold’ 6 – ‘italics’ 7 – ‘shadow’

J

Strategy 2

k

d

f

G

Strategy 3

J b

Use Expert or Experiential Knowledge

Strategy 1 Participants’ Experiences in Research Context

e

h

C

Acquired through: • Practice/experience • Observations/interviews • Task analyses • Cognitive task analyses

Participants’ Simulated Experiences in Research Context

A

G

C

k f

J d

Fig. 19.5 Illustration of three strategies for the representative design of research tasks/conditions/ situations

Chipman, & Shalin, 2000; Wei & Salvendy, 2004) relevant tasks as carried out in their natural environments or gathers data from those with prior expertise about and/ or experience in carrying out such tasks in order to use that knowledge to construct a sample of simulated tasks for research participants (thus, requiring a sequential MU configuration; this would often be the type of configuration used in conjunction with an Immersive experience-focused strategy). The key difference between Strategies 2 and 3 lies in the basis for constructing sample simulated tasks for participants: statistical properties (summarising what Hammond, McClelland, & Mumpower, 1980; Cooksey, 1996, called objective or formal task features) or expertise/experiential knowledge (encompassing what Hammond et al., 1980; Cooksey, 1996, called subjective or substantive task characteristics), respectively. Representative design is a rather more subtle but critical angle on the question of extensional reasoning; it is not only about the size and composition of the sample, but also about the meaningfulness and relevance of the experiences that sample members undergo during their participation in your research. It is also worth observing that representative design can form an important part of the contextualisation of your research, guided by the positivist pattern of assumptions, especially in the context of the Explanatory research frame. [Note that Csikszentmihalyi and Hunter (2003) devised a technique they called experience sampling which involves prompting participants (using a wearable electronic device) to make ratings of their experiences throughout the day (their focus was on emotional states

19.1

Key Considerations Surrounding Sampling

841

like happiness). This could result in a random or systematic sample (see discussions below) of evaluated personal experiences for each participant. Experience sampling is entirely consistent with Brunswik’s concept of representative design reflecting its logic in terms of enhancing the meaningfulness of research experiences to and for participants.]

19.1.6 Focus on Sufficiency Under interpretivist/constructivist patterns of guiding assumptions, representative samples and representative design do not carry the same meaning or implications. This is partly because generalisations to some population are not typically sought in such research, and partly because such research typically does not employ the deliberate manipulation of tasks and conditions as a strategy for data gathering. Instead, the most desirable sample is one that affords you access to sufficient data upon which to base your interpretations and accounts. The most desirable observations are those that are made under natural environmental and social circumstances, with all the messiness that may imply. In one very loose sense, the focus on sufficiency can be viewed as an interpretivist/constructivist take on ‘representative design’, namely that you relate to/connect with research participants under naturalistic but, importantly, uncontrolled circumstances in their own context. However, it is important to realise that participants are not the only type of sampled entities relevant to sufficiency. Sufficiency refers to whether you have sampled enough participants, handiworks, artefacts, documents, experiences, times, events and contexts to anchor, develop and sustain the interpretations and accounts that emerge from your data. The sufficiency focus thus implicates the sampling of all types of data sources and speaks directly to both the extensional reasoning (via the sufficiency quality criterion) and the internal coherence (via the authenticity quality criterion) meta-criteria. Longer immersion in a naturalistic context will generally yield samples affording greater sufficiency and this is more likely to occur in theoretical sampling strategies.

19.1.7 Focus on Contextual Relevance/Knowledge/ Experience This focus is particularly congruent with research conducted under an interpretivist/ constructivist pattern of guiding assumptions and it shifts the emphasis from sample size to sample composition, where the goal here is to ensure that the right people and other data sources (i.e., those with, or offering information relevant to, contextually relevant knowledge/experience/roles/situations) are included in your sample. Contextual, purposive and snowball sampling schemes can help in this

842

19

How Do I Manage the Sampling Process?

regard and some stages of theoretical sampling may also benefit from this focus. The goal of this sampling focus is to maximise your contextualised learning value. It may also help enhance or facilitate access to other desired data sources and contexts, as they become known to you. This focus is directly relevant to the extensional reasoning (via the sufficiency quality criterion) as well as the internal coherence (via the authenticity quality criterion) meta-criteria.

19.1.8 Focus on Transportability This focus is also particularly congruent with research conducted under an interpretivist/constructivist patterns of guiding assumptions. Transportability refers to whether the interpretations and accounts that have emerged from your current study, in its context, with the data sources that you have sampled might have meaning for or be relevant to other contexts. While it is easy to confuse this with the positivist process of generalisation, which focuses on generalising from a sample to a relevant population), transportability is not the same thing. Not every interpretivist/constructivist study has transportability as a goal, but if you do have this goal, your sampling must be such that it would facilitate the convincing achievement of cross-contextual meaning and relevance. Negative case sampling can be very important to implement in this regard because it allows you to evaluate transported meanings from one context against the perspectives and views encountered from data sources in another context. Cross-site participant observation studies, for example, achieve transportability through sampling in each context and then cross-comparing the interpretations that emerge. One prerequisite for transportability is a deep focus on local contextual features and nuances, as these will either facilitate or inhibit cross-context transportation of meanings. This focus is primarily relevant to the extensional reasoning meta-criterion.

19.1.9 Focus on Expediency Here, the focus shifts from statistical efficiency/effectiveness with respect to sample size and/or representativeness with respect to sample composition to what is expedient/easy/feasible for you to access in building up your sample. This focus is often driven by you having to live with severe resource or access constraints. It may also provide the easiest pathway for gaining access to willing participants (online research participant pools may provide one such pathway). Volunteer and convenience sampling schemes as well as the systematic sampling scheme are approaches that implement the expediency focus. It is important to understand that the expediency focus generally means that you deliberately sacrifice potential with respect to the extensional reasoning (reducing external validity and generalisability or transportability and sufficiency) as well as the internal coherence (reducing internal

19.1

Key Considerations Surrounding Sampling

843

validity or authenticity) meta-criteria and these sacrifices will then create opportunities for acknowledgement of limitations and fertilisation of ideas for further research when your research story is being composed.

19.1.10

Hybrid Synergies

Sampling schemes can be creatively hybridised to gain some advantages against more than one focal emphasis. For example, the stratified random sampling scheme combines the benefits of simple random sampling (focused on precision or power) with the logic of a quota sample (focused on enhancing representativeness). A two-stage cluster sampling scheme explicitly combines a focus on statistical efficiency/effectiveness with a focus on expediency. To enhance representativeness while still facilitating expediency, you might combine the volunteer sampling scheme with a quota sampling scheme or a systematic sampling scheme with a quota sampling scheme. To enhance contextual relevance/knowledge/experience while still facilitating expediency, you might combine the purposive sampling scheme with a quota sampling scheme.

19.2

Probabilistic Sampling Strategies

In positivist research using statistical analysis to draw generalising inferences from a sample to a population using quantitative measurements, probabilistic sampling strategies provide a range of statistically appropriate methods for assembling a sample (Cooksey, 2014, pp. 225–229; see also the discussions in Lynn, 2002a, 2002b). In the discussions that follow, we will use the term ‘data source’ as a collective label to refer to the focal unit for sampling, be it human data sources such as a person, a group, an organisation or non-human data sources and handiworks such as documents, journal articles, YouTube clips, articles in newspapers or other media outlets, performances, photographs, drawings or other artworks. In any probabilistic sampling strategy, the defining feature is that, for every potential data source in the sampling frame of the population, the probability that the data source would be selected for inclusion in your sample is known. There are two basic types of probabilistic sample selection available: (1) random, where, as mentioned earlier, every potential data source in a specified sampling frame within the population has an equal chance of being selected in the sample; or (2) non-random, where some data sources in a specified sampling frame within the population have a greater chance of being selected in the sample than other data sources. Any sampling strategy, where the probability of data sources being selected into the sample remains unknown or is irrelevant, is termed a non-probabilistic sampling strategy; these will be discussed later in the chapter.

844

19

How Do I Manage the Sampling Process?

For the purposes of making strong statistical inferences and generalisations with known precision and/or power, a probabilistic sampling strategy is preferred over any non-probabilistic sampling strategy, a random sample is preferred over a non-random sample and an adequate sample size is a necessary precondition for sound inferences. Research investigations guided by the positivist pattern of guiding assumptions, that use a sample of inadequate size, even if that sample has been obtained using a random probabilistic sampling strategy, typically lack generalisability and external validity, and their conclusions should be carefully considered with respect to how far the authors are trying to extend their findings; i.e., extensional reasoning is constrained. The goal of a random probabilistic sampling strategy is to try to achieve a representative sample (with respect to a specified population of interest) with a given or desired level of precision in statistical estimation and/or with a given level of power to detect relationships of a specified size (thus, combining the precision and/or power focus with the representativeness of sample focus). If you adopt the Survey research frame or Explanatory research frame (under the positivist pattern of guiding assumptions), you are more likely to employ probabilistic sampling strategies.

19.2.1 Simple Random Sampling Simple random sampling is the most basic random probabilistic sampling strategy and can be accomplished via a very simple set of steps. To draw a random sample, you must first be able to: (1) describe precisely what the intended population for generalisations is and (2) describe who (or what) in that population you can feasibly gain access to, thus establishing your sampling frame. As briefly mentioned earlier, the sampling frame defines the relevant and accessible population for the purposes of your research. It may differ from the actual population to which you would like to generalise findings because the sampling frame may not contain all the members of the population (e.g., the sampling frame of a phone book or voter registry will not contain those population members who have silent or unlisted numbers or who do not have a phone or who have not registered to vote, for whatever reason). The sampling frame may also differ from the actual population to which you would like to generalise findings because of lags in its updating and maintenance. Here, the frame may contain data sources who are no longer members of the population or may not contain data sources who have most recently become members of the population (e.g., the sampling frame of enrolled students in a school or dues-paying members of a trade union may not capture those students/workers who just very recently moved to the school/joined the union or may contain students/workers who are no longer at the school/no longer dues-paying union members; a voter registry may contain registered voters who are not available for sampling (due perhaps to being out of the country)). You must assemble or procure the sampling frame, for random probabilistic sampling purposes, in the form of a list showing all potential data sources uniquely identified using some key index such as an ID number, tax

19.2

Probabilistic Sampling Strategies

845

file number or phone number. Probabilistic random sampling then occurs by choosing specific values of this key index, identified by a random number generating algorithm. If this list (say, from a census or a complete listing of all current employees in a specific organisation) can be imported into a spreadsheet statistical program like SPSS, it is a relatively straightforward matter to draw a simple random sample. Consider the illustration in Fig. 19.6 (for illustration purposes, we will consider the sampling frame to be the same as the full population of interest). The figure shows a population of 1000 data sources from which to draw the sample; each data source is simply and uniquely identified by the ID numbers 1 to 1000. This sampling frame list could easily be imported into a statistical package like SPSS. Suppose you want to obtain a 5% sample of this population for data gathering purposes. This means a sample size of 50 data sources. One approach to obtaining a simple random sample would be to use the uniform random number-generating procedure in SPSS to produce a series of 1000 random numbers, each with a value between 0 and 1; one random number associated with each data source. [A uniform distribution is rectangular, meaning that every number in the range between the lower and upper limits has an equal chance of occurring.] Now, if a 5% sample (n = 50 people) is your goal, all you need do is scan down the list of 1000 random numbers and select for your sample any data source associated with a random number value of 0.05 or less until the desired number of 50 is reached (SPSS has a Select Cases function that can do this conditional scanning automatically). If you wanted a 20% sample, selection would focus on random numbers with values of 0.200 or less, and so on. [There is a useful web-based resource called the Research Randomizer which can be used to generate random numbers for sampling as well as to randomly assign participants to experimental conditions; see https://www. randomizer.org/]. The right-hand side of Fig. 19.6 lists the ID numbers of the 50 data sources selected for the sample using this approach. These would be the data sources you would approach or access to provide your data. Advantages of a simple random sample: • It is the simplest random sampling scheme to implement, taking a whole-of-population perspective. • Every data source in the sampling frame has an equal chance of being selected regardless of their individual characteristics (i.e. the sampling process is unbiased). • It generally yields a sample with good statistical properties suitable for statistical inference and generalisation, affording adequate precision for estimating population parameters at relatively low cost. Disadvantages of a simple random sample: • You need a complete list of the sampling frame members before your sample can be drawn; if you cannot access such a list, your sampling frame (and, by implication, your intended population) may need to be revised.

846

19

How Do I Manage the Sampling Process?

Population (= Sampling Frame) List of ID Numbers [N = 1000]

5% Simple Random Sample List of ID Numbers [n = 50]

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 . . . 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430

2 8 10 21 42 46 58 100 126 128 131 166 167 194 222 224 268 290 313 351 391 429 482 512 616 646 646 667 675 680 681 681 698 706 732 781 860 864 880 885 889 908 925 930 944 959 977 994 996 999

986 987 986 989 990 991 992 993 994 995 996 997 998 999 1000

Fig. 19.6 Illustration of the simple random sampling strategy for a population = sampling frame of 1000 data sources

19.2

Probabilistic Sampling Strategies

847

• While the random sampling process itself is unbiased, it does not control for any systematic errors, relying instead upon the laws of probability to randomly distribute anything systematic throughout the sample. It may, in fact, produce a biased sample, simply because of the behaviour of probabilities (which a uniform random number generator effectively produces if you constrain it to produce numbers between 0.0 and 1.0). • The random nature of your sample can be destroyed if too many sampled human data sources either decline to participate or withdraw their participation once they start. This problem can be partially offset if you deliberately draw a sample larger than you need (a process called over-sampling) giving you a list of randomly sampled reserves that can be used to replace sampled data sources who don’t participate.

19.2.2 Stratified Random Sampling The stratified random sampling strategy is useful in situations where the simple random sampling strategy scheme, with its whole-of-population emphasis, runs the risk of under- or over-sampling data sources in specific mutually exclusive categories (i.e., non-overlapping strata) within the population. If membership in those categories is relevant to your hypotheses being tested, it becomes critical that those categories are adequately sampled. Basically, you segment your population into relevant strata (e.g., categories defined by gender, geographical location, ethnic background, income band, tax bracket, type of organisation, management level) that you suspect might influence patterns of relationships and therefore need to be controlled in some way. Stratified random sampling provides one such control mechanism through generating separate simple random samples within each stratum, each of a size that you dictate. Depending upon your needs, there are two distinct approaches to determining how large the sample within each stratum needs to be: (1) proportionate stratified random sampling, where sample sizes are proportional to the size of the stratum in the population/sampling frame (pursues proportional representation, based on stratum size, and if the strata are relatively homogeneous, results in more precise statistical estimates) or (2) disproportionate stratified random sampling, where, for example, there are several larger strata and one very small stratum (e.g., as with the Indigenous sector of the population of Australia) and the very small stratum is over-sampled to ensure adequate representation and capacity to generate sound estimates at the level of that stratum whereas all other strata may be sampled roughly proportionally. Figure 19.7 illustrates proportional stratified random sampling for a population (= sampling frame for purposes of illustration) of 1000 individuals. You divide the population into three strata (Stratum 1 comprises 45% of the population; Stratum 2 comprises 35% and Stratum 3 comprises 20%) and separately draw a 5% simple random sample within each stratum. The resulting

848 Fig. 19.7 Illustration of the proportional stratified random sampling strategy for a population = sampling frame of 1000 individuals, classified into 3 strata

19

How Do I Manage the Sampling Process?

Stratified Population 5% Stratified Random Sample List of ID Numbers List of ID Numbers [N1 = 450; N2 = 350; N3 = 200] [n1 = 27; n2 = 18; n3 = 10]

Stratum 2 1 2 3 4 5 6 7

344 345 346 347 348 349 350

Stratum 1

Stratum 1

1 2 3 4 5 6 7

3 12 24 51 70 98 115 125 159 181 201 201 203 223 229 239 240 254 281 296 298 371 388 393 416 433 440

444 445 446 447 448 449 450

Stratum 3 1 2 3 4 5 6 7

194 195 196 197 198 199 200

Stratum 2 3 24 27 34 75 100 108 117 125 148 163 192 199 228 235 252 316 322

Stratum 3 23 53 59 80 94 126 129 144 148 166

19.2

Probabilistic Sampling Strategies

849

sample sizes are proportional to the sizes of the strata in the population (N1 = 450 ! n1 = 27; N2 = 350 ! n2 = 18; N3 = 200 ! n3 = 10). Advantages of stratified random sampling: • It can produce a more representative sample and, therefore, more precise statistical estimates (i.e., with smaller error bands or confidence intervals) compared to a simple random sampling scheme. However, this occurs only where, in the population, the strata actually statistically differ on the constructs that you measure using your sample and where the proportionality of stratum membership in the population is matched to the proportion in the strata of the sample (i.e. a proportional stratified random sample). In effect, stratified random sampling gives you a procedure for controlling for the possible extraneous or confounding effects that strata membership might exert on dependent variables of interest. • It can provide you with the opportunity to make specific decisions about the proportion of data sources to sample within each stratum, to enhance the representativeness of each stratum. For example, if your population is multicultural in nature (e.g. a mixture of non-Indigenous Australians and Indigenous Australians), you may wish to deliberately over-sample the Indigenous stratum using non-proportional stratified random sampling, because it is so small relative to the non-Indigenous stratum. This will give a reasonable sample size in the Indigenous segment, at the usually acceptable cost of under-sampling the larger non-Indigenous segment. This will reduce the statistical precision of the sample somewhat but will enhance your arguments for representativeness of the sample. Disadvantages of stratified random sampling: • It is a more complex sampling strategy to implement, requires specific knowledge of strata composition of the population so that you can at least approximate the population strata proportions in your sample and requires you to have access to a population list where all members have their stratum membership included. • You must carefully choose the strata variables and categories to focus on. If your chosen strata have no relationship with or impact on the measurements that you will obtain, you will be worse off in terms of sampling quality. For example, if in the illustration in Fig. 19.7, the characteristic that defined the three strata (say, industry sector—manufacturing, retail and health) had nothing to do with what you wanted to measure about the individuals in the sample (say, training needs, organisational commitment or job satisfaction), you will pay a penalty in terms of less statistical precision compared with the less complex simple random sampling strategy while, at the same time, incurring unnecessary costs. Sometimes it may be difficult for you to know beforehand whether you have chosen the right strata-defining characteristic to focus on, unless you have access to prior research showing a relationship between the strata categories and the dependent variable(s) you are interested in.

850

19

How Do I Manage the Sampling Process?

• The stratified random sampling strategy can become costly, very unwieldy and complex if you want to sample within different types of strata in combination (e.g. gender, type of school, and location of school). Generally speaking, it will not be possible for you to sample within every potentially relevant stratum combination in the population, which means that you will need to prioritise and only focus on those strata that you most need to control for.

19.2.3 Cluster Sampling Cluster sampling is a somewhat more complicated scheme designed to enhance the feasibility of accessing a sample. For example, if your population is widely geographically dispersed, you may not have the resources needed to reach all areas for data gathering purposes. In this case, a logical sampling system would be to divide the population by geographic areas and randomly select a subset of areas to focus on. Each selected area would constitute a cluster, and you would then gather data from every individual entity within each cluster. Thus, cluster sampling is basically simple random sampling applied to clusters as the sampled entities. In general, clusters may be defined in various ways, but most often are defined by some type of geographical, physical, political or organisational boundary, such as states or territories in a country/nation, local councils, counties or shires in a state, suburbs in a city, departments or divisions in an organisation, police or fire brigade precincts, wilderness, recreation or national park areas, school districts, electoral districts, water catchment areas and so on. Figure 19.8 illustrates a 50% cluster sample of a population of 850 entities (let’s say restaurants, for illustrative purposes) identified as residing within one of four clusters (defined as the north, south, east and west regions of a major metropolitan city). The figure shows two clusters (1 and 4) having been randomly chosen and every entity (i.e., restaurant) within those two clusters would be the focus of data gathering activity (yielding a total sample of 500 entities). Advantages of cluster sampling: • You can achieve a more feasible and acceptable sample, but one that still has well-understood statistical properties in terms of the estimates it can provide. This provides a statistically defensible trade-off between sample, representativeness, and the cost of data collection. Scheaffer, Mendenhall, Ott, and Gerow (2012) indicated that cluster sampling can maximise information gain while reducing the costs of accessing that information, especially in situations where you cannot access a full sampling frame listing of population members. The cost burden increases as the separation between clusters increases. • If the cluster sample can be considered as representative of the collection of all potential clusters, then defensible generalisations from sample to population can

19.2

Probabilistic Sampling Strategies

Geographically Clustered Population List of ID Numbers [N1 = 250; N2 = 150; N3 = 200; N4= 250] Cluster 1 1 2 3 4 5 6 7

Cluster 2 1 2 3 4 5 6 7

144 145 146 147 148 149 150

Cluster 4 1 2 3 4 5 6 7

244 245 246 247 248 249 250

Cluster 3 1 2 3 4 5 6 7

194 195 196 197 198 199 200

851

50% Cluster Sample List of ID Numbers [k = 2 clusters; n1 = 250; n4= 250]

Cluster 1 1 2 3 4 5 6 7

244 245 246 247 248 249 250

Cluster 4 1 2 3 4 5 6 7

244 245 246 247 248 249 250

244 245 246 247 248 249 250

Fig. 19.8 Illustration of cluster sampling where the population is divided into four clusters, two of which are randomly chosen for the purposes of gathering data from all elements contained within the cluster

852

19

How Do I Manage the Sampling Process?

be made (generalisations are typically targetted at the element level, not at the cluster level). Disadvantages of cluster sampling: • Once you have identified and/or defined the clusters, they must remain stable or be assumed to remain stable (i.e., with boundaries and element composition unchanged) for the duration of the research project, or your capacity to generalise will be reduced. • Once clusters have been sampled, you must have access to all the potential data sources within the cluster and this may not be possible depending upon circumstances, such as where you do not have access to up-to-date lists of data sources categorised by or residing in the cluster or inability to secure a 100% participation rate within each cluster (two-stage cluster sampling, to be discussed below, can potentially compensate for this problem). • Clusters serve as a way of conveniently grouping elements together for sampling purposes; they normally do not serve any other interpretive purpose. If you wish to draw generalising inferences at the level of clusters, then you may need to combine cluster sampling with stratified random sampling (i.e., stratified random cluster sampling) so that representativeness of the sample of clusters is enhanced. This may be especially important if the number of clusters is quite large or if the clusters vary quite markedly in the number of data sources each contains.

19.2.4 Two-Stage Cluster Sampling Two-stage cluster sampling is a synergistic combination of cluster sampling and simple random sampling that is useful in situations where it is not feasible for you to sample every individual within each cluster. The process is straightforward: (Stage 1) randomly choose clusters and (Stage 2) within each chosen cluster, take a random sample of potential data sources. This sampling strategy largely negates one of the main disadvantages of cluster sampling, namely that every data source within a sampled cluster must be accessed. Figure 19.9 illustrates the two-stage cluster sampling process for a population of 850 potential data sources divided into four clusters. You randomly choose clusters 3 and 4 to focus sampling attention on and, within each cluster, select a separate random sample of data sources. Advantages of two-stage cluster sampling: • You retain the advantages of cluster sampling while, at the same time, not being required to have access to all data sources in each sampled cluster. • An even more feasible and acceptable sample can be achieved using cluster sampling, while still retaining well-understood statistical properties in terms of

19.2

Probabilistic Sampling Strategies

Geographically Clustered Population List of ID Numbers [N1 = 250; N2 = 150; N3 = 200; N4= 250]

853

50%/10% Two-Stage Cluster Sample List of ID Numbers [n3= 20; n4= 25]

Cluster 1 1 2 3 4 5 6 7

Cluster 2 1 2 3 4 5 6 7

144 145 146 147 148 149 150

Cluster 4 1 2 3 4 5 6 7

244 245 246 247 248 249 250

244 245 246 247 248 249 250

Cluster 3 1 2 3 4 5 6 7

194 195 196 197 198 199 200

Stage 1: 50% Cluster Sample List of ID Numbers [k = 2 clusters]

Stage 2: 10% Random Sample within each cluster List of ID Numbers

Cluster 3

Cluster 3 1 2 3 4 5 6 7

194 195 196 197 198 199 200

Cluster 4 1 2 3 4 5 6 7

244 245 246 247 248 249 250

Cluster 4 18 41 45 47 51 51 60 66 70 72 114 117 119 142 165 170 188 192 194 196 196 214 222 227 237

7 14 16 23 33 34 45 47 55 55 87 107 133 134 164 169 171 181 183 185

Fig. 19.9 Illustration of two-stage cluster sampling where the population is divided into 4 clusters, two of which are randomly chosen, followed by simple random sampling within each cluster

854

19

How Do I Manage the Sampling Process?

the estimates it can provide. This also provides a statistically defensible trade-off between sample, representativeness, and the cost of data collection (but with an attendant increased risk of less representativeness due to the behaviour of probabilities when random sampling inside clusters). • If the cluster sample can be considered as representative of the collection of all potential clusters, then defensible generalisations from sample to population can still be made (generalisations are typically targetted at the element level, not at the cluster level). Disadvantages of two-stage cluster sampling: • Clusters serve as a way of conveniently grouping elements together for sampling purposes; they normally do not serve any other interpretive purpose. If you wish to draw generalising inferences at the level of clusters, then you may need to combine cluster sampling with stratified random sampling (i.e., stratified random cluster sampling) so that representativeness of the sample of clusters is enhanced. This may be especially important if the number of clusters is quite large or if the clusters vary quite markedly in the number of data sources each contains. • Representativeness of the sampling within clusters may be incomplete, depending upon the vagaries of the random sampling process within clusters. This can be particularly problematic with small-sized clusters.

19.2.5 Systematic Sampling Systematic sampling is technically not a random sampling scheme since the basis of the scheme is a sequential progression through a listing of the sampling frame. To assemble a systematic sample, you randomly choose a data source from the first k data sources in the sampling frame list, then select every kth successive data source from the list. Such a sample is called a 1 in k systematic sample. It is considered a probability sample because the first selected data source has a probability of 1 in k of being chosen and, after that, every kth data source has a 1.0 probability of being selected (all data sources in between have a probability of 0.0 of being selected). You must establish the value for k on the basis of the proportion of the population desired in the sample; k thus depends upon desired sample size. Figure 19.10 illustrates the systematic sampling process for choosing a 5% (or 1 in 20) sample from a population of 1000 data sources. If you desire a 5% sample, 5% of 1000 data sources would yield a desired sample size of 50 data sources. Dividing the population size by the desired sample size yields the value for k to be used (1000/50 = 20). However, your sample cannot be considered random because every data source does not have an equal chance of being chosen for the sample. Another thing to note about systematic sampling is that if your list has an inherent ordering to it (e.g., alphabetical, by region, by date) but that ordering is irrelevant to

19.2

Probabilistic Sampling Strategies

Fig. 19.10 Illustration of a systematic sample of a population with 1000 data sources, sampling every 20th successive data source after the initial data source has been randomly chosen from the first 20 data sources in the sampling frame list

855

Population List of ID Numbers [N = 1000] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 . . . 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430

986 987 986 989 990 991 992 993 994 995 996 997 998 999 1000

5% (1 in 20) Systematic Sample List of ID Numbers [k = 20; n = 50] 14 34 54 74 94 114 134 154 174 194 214 234 254 274 294 314 334 354 374 394 414 434 454 474 494 514 534 554 574 594 614 634 654 674 694 714 734 754 774 794 814 834 854 874 894 914 934 954 974 994

856

19

How Do I Manage the Sampling Process?

the purposes of your research, then the ordering may influence the composition and therefore the representativeness of your sample. This risk can be negated by randomly reordering your list first, then assembling your systematic sample (see, for example, Bryman & Bell, 2015, p. 191). A systematic sample will not approach the representativeness of sample nor the statistical precision offered by simple random sampling (unless your sample is very large, and your sampling frame list is ordered in a meaningful way; see Scheaffer et al., 2012, for details), because once the sampling process commences, there will be many data sources that never have the chance of appearing in your sample. Advantages of systematic sampling: • Systematic sampling is often a convenient and cost-effective alternative to simple random sampling, especially if a complete listing of your sampling frame is available. It is very easy to precisely target the desired sample size simply by adjusting the value of k. In situations where there is no available list of members of the sampling frame (i.e., the sampling frame is open-ended, for example, when conducting a mall-intercept survey of shoppers or interviewing community members walking along a specific street or through a specific venue), then systematic sampling may be the only viable sampling strategy. In such cases, you first randomly determine the first data source to be sampled. This can be done by generating a list of k uniform random numbers and choosing the case number associated with the highest value random number (e.g., if k is set at 15, 15 random numbers between 0 and 1 are generated and numbered from 1 to 15; if the largest random number is the 9th number on the list, you sample the 9th data source from the time data gathering commences on a specific day). You then sample every kth data source encountered after that first choice (every 15th data source encountered after the initial 9th data source, continuing the above example). • Systematic sampling can be employed after cluster sampling to yield another type of two-stage cluster sample. It may also be employed as a stratified (but not random) sampling strategy. Disadvantages of systematic sampling: • There is a potentially high risk of obtaining a biased sample with systematic sampling, particularly if the original sampling frame list selectively includes or excludes certain types of people. For example, if you are systematically sampling using a sporting club membership roster as your sampling frame list, the roster may be out of date so that not all current members are listed, or the roster may be incomplete because only financial members (those who have paid their annual dues for the current year) are listed. These gaps thus contain types of people that will have no chance of appearing in your sample and this could adversely impact representativeness and constrain your generalisations.

19.2

Probabilistic Sampling Strategies

857

• Compared to a simple random sample, a systematic sample will tend to yield less precise statistical estimates, unless your sample is large and there is a relevant ordering to the sampling frame list. • A systematic sample is less likely to produce a representative sample simply because a large proportion of the sampling frame never has a chance of being selected. This means that generalisations to the population are harder for you to defend based on a systematic sample.

19.3

Non-probabilistic Sampling Strategies

The use of probabilistic sampling strategies is often (but not always) driven by statistical requirements for estimation, relationship testing and generalisation using quantitative data, usually in research guided by the positivist and sometimes the critical realist pattern of assumptions (if qualitative data are also gathered, they are typically supplemental to quantitative data, i.e., a hierarchical MU configuration). However, probabilistic strategies form a smaller part of a much larger set of sampling strategy possibilities that can be used to achieve a wide range of research purposes, under more diverse patterns of guiding assumptions, using qualitative and/or quantitative data. The larger portion of this set comprises non-probabilistic sampling strategies; strategies that do not depend upon knowledge of the probability of data sources being chosen for your sample and do not utilise the principle of randomness as a sample selection criterion. Instead, non-probability sampling strategies apply different sampling logics to meet your needs. Some non-probabilistic strategies (e.g., quota sampling, volunteer sampling, convenience sampling) have evolved as alternatives to probabilistic sampling strategies and are useful in circumstances where you lack the necessary resources or permission to access relevant data sources in your sampling frame or are unable to adequately define or list the membership of your sampling frame. These non-probability strategies constitute a fallback position imposed by constraints on you, as the researcher, and as such create limitations for you to address in terms of drawing convincing analytical conclusions and generalisations. In some cases, your choice of data gathering strategy may preclude the use of a probability sampling strategy (e.g., web-based internet questionnaires accessed by clicking on a weblink are difficult to target to specific individuals; rather, they tend to rely on whoever clicks the website link and accepts the invitation to participate, which means that you cannot anticipate sampling probabilities). It is possible to use a non-probability sampling strategy to achieve a statistically representative sample (e.g. quota sampling could be used for this purpose, see below), but the sample cannot and must not be considered random. Other non-probability sampling strategies (e.g., purposive sampling, snowball sampling, theoretical sampling, contextual sampling) have evolved to offer you new avenues for choosing data sources that are more consistent with the expectations of

858

19

How Do I Manage the Sampling Process?

interpretivist/constructivist, participatory inquiry and Indigenous patterns of guiding assumptions and are well-suited for research gathering qualitative data. These non-probability strategies are useful in research contexts where random or systematic sampling simply does not make sense, given the type of data you want to gather, the positioning of the participants and/or the research questions being addressed, because the sampling foci (such as precision of estimation, power, sample size, external validity and capacity to generalise) they pursue are irrelevant. Instead, sampling criteria, such as pursuing greater depth of learning, accessing the most appropriate data sources, ensuring sufficient coverage of relevant voices, evaluating emerging theoretical insights, are employed. With these latter strategies, the meaning of representativeness extends beyond considerations of sample composition to considerations of availability and authenticity of perspectives and viewpoints and relevant contextual knowledge.

19.3.1 Convenience Sampling The convenience sampling strategy, as the name suggests, involves you choosing participants that are easy for you to access, usually at much lower cost, relative to a probabilistic sampling strategy. Convenient access can be achieved when participants live in geographical proximity or work in organisational proximity to you, when you have a personal connection to specific gatekeepers who control organisational or institutional access or attendance at specific events for data gathering purposes or when you are part of a professional network or know a colleague who can facilitate access to participants. Postgraduate research students and early career researchers, for example, frequently must rely on the convenience sampling strategy because of the resource and access constraints they face. In the case of postgraduate research, your supervisor may be able to facilitate access or participants may be sourced from the local area in which you reside, study or work. University academic researchers frequently use convenience sampling (either of undergraduate students, who may be promised course credit as an incentive, or postgraduate students or students undergoing professional development, such as MBA students or student teachers) because it is an efficient way to produce research outcomes quickly. For university academics, the driving motivation behind using a convenience sample may be less about adapting to contextual constraints and more about taking advantage of contextual opportunities, perhaps as a pathway toward furthering their career. This can be problematic for a social science discipline if the use of such samples (as is common in psychological, management or marketing research) becomes the norm for theory building and testing as it means that extensional reasoning and value for learning are consistently diminished, unless the researcher is very careful and open about how they tell their story and acknowledges the limitations that their choice of sampling strategy creates. Convenience sampling may be combined with other sampling strategies such as mall-intercept systematic sampling (where you approach every kth person

19.3

Non-Probabilistic Sampling Strategies

859

Convenient or feasible researcher-accessible site/situation Data Source 7 Data Source 1 Data Source 4 Data Source 2

Data Source 3 Data Source 9

Researcher Data Source 6

Data Source 8 Data Source 5

Data Source 10

Fig. 19.11 Visualisation of the convenience sampling strategy

encountered, seeking their participation onsite at a local shopping mall or other public venue) or quota sampling (see discussion below) in your local area. Figure 19.11 shows a visualisation of the convenience sampling strategy, where you access a convenient or feasible site or situation where it is easy and cost effective for you to find 10 data sources. Advantages of convenience sampling: • Convenience sampling can be an efficient (i.e., quick) low-cost strategy that provides minimal drain on your resources. • Convenience sampling can be used under the guidance of a range of patterns of assumptions, including positivist or interpretivist/constructivist. • The convenience sampling strategy may benefit from your local contextual knowledge, if you live and work in the same area as your participants (however, this may be a two-edged sword, as discussed below). Disadvantages of convenience sampling: • Under the positivist pattern of guiding assumptions, the convenience sampling strategy greatly weakens your capacity to generalise to some population of interest relative to a probabilistic sampling strategy, because convenience places tight boundaries around your sampling frame, rather than a proper listing of the population to which findings might be generalised. It is extremely difficult to argue that a convenience sample is representative, which adds to the problems associated with generalisability.

860

19

How Do I Manage the Sampling Process?

• The convenience sampling strategy has a high risk of producing a biased sample relative to probabilistic and other non-probabilistic sampling strategies because you are leveraging your own local knowledge or someone else’s local knowledge to access data sources. This usually yields a poorer sample for generalisation or sufficiency purposes since perspectives from potential data sources in the population that you cannot access are, by definition, excluded from your sample. Therein lies the trade-off away from what is ideally desirable to what is practical and feasible: sampling data sources that are convenient to access forces you to trade away capacity to generalise (i.e. sacrificing representativeness) under positivist assumptions, or to trade away sufficiency and transportability of interpretations and accounts under an interpretivist/constructivist pattern of assumptions. Thus, if you use a convenience sampling strategy, you must to be very cautious in how far you extend the conclusions and implications from your research beyond the boundaries of that research, creating limitations to extensional reasoning. • Use of a convenience sampling strategy as an opportunistic choice rather than as a fallback adaptive response to constraints can contribute to the maintenance of a self-reinforcing system, whereby a discipline produces prodigious amounts of research but is effectively held back from achieving its full potential because the appropriateness of using research participants that are convenient to access is not systematically or critically examined or questioned. As noted above, this creates limitations with respect to two important meta-criteria, extensional reasoning and value for learning. Such a practice also serves to reinforce the divide between academic research and practitioner-based learning, reifying Mode 1 knowledge at the expense of Mode 2 knowledge.

19.3.2 Quota Sampling The quota sampling strategy is the non-probabilistic analogue to probabilistic stratified random sampling strategy and involves three discrete steps: 1. Identify key dimensions and associated categories of participants to be sampled, in very much the same way as is done for stratified random sampling. The dimensions should be relevant to establishing the representativeness of the sample for the purposes of addressing your research questions of interest and the categories of each dimension should preferably (but not necessarily) be mutually exclusive and exhaustive of possibilities. For example, the dimension of gender could be defined to have two categories (male, female) or three categories (male, female, transgender/intersex) depending upon the research issues you are investigating. Ethnic background or ancestry in Australia could be defined in several ways (e.g., identifying as Indigenous or non-Indigenous; as having parents who were of white Australian ancestry, European ancestry, Asian ancestry, Middle Eastern ancestry, mixed ancestry; parents who were from a

19.3

Non-Probabilistic Sampling Strategies

861

specific named country or region). Whatever dimensions and categories you define, these form the basis for step 2. 2. Establish the number of participants in each category of each dimension to be obtained; these establish the sampling quotas to be filled. The quota size in each category could be judged against both the total desired sample size and the desired pattern of representation in each category. Representativeness of sample will be enhanced somewhat if a proportional representation approach to deciding on the quota to be filled in each category is employed (proportions could be sourced from a national census, organisational records or some other relevant secondary data source). As a simple example, if three categories of management are of interest for an internal management survey of a large organisation employing 5000 people and organisational records indicate a typical ratio of 1 senior-level manager to every 2.5 middle-level managers to every 6 floor supervisors, then a proportional quota sample for conducting structured interviews in the organisation might be set at 20 senior-level managers, 50 middle-level managers and 120 floor supervisors, giving a total sample of 190 managers. 3. Sampling of participants proceeds until the requisite quota for each category of each dimension is filled. Various sampling strategies could be used at this stage, including systematic sampling, convenience sampling, volunteer sampling or purposive sampling. Figure 19.12 provides a visualisation of the quota sampling process for different types of participants, where each type represents one category of a four-category dimension (e.g., age groups or educational background). While the quota sampling strategy is typically used in positivist research (particularly in the context of the Survey, Cross-Cultural, Descriptive, Exploratory or Evaluation research frame), it may also be useful when undertaking interpretivist/constructivist research. A quota sample can help you ensure that you access a desirable range and volume of perspectives from specific types of people. Thus, a quota sample can help you make arguments for sufficiency. Advantages of quota sampling: • It is a cost-effective and time-efficient alternative to stratified random sampling and becomes more viable in situations where you cannot obtain the sampling frame lists needed to carry out a proper random sampling with strata. The final sample size and composition is guaranteed when all quotas have been filled, which provides you with some assurance of representativeness in the sample. • However, it is important to realise that this assurance is only ever tentative in a quota sample because sampling is not random within a category. • If a potential participant declines to participate, you simply select another participant to approach.

862

19

How Do I Manage the Sampling Process?

Quota (n1) for Participant Type 1

sampling commences

Quota (n2) for Paticipant Type 2

sampling commences

Quota (n3) for Participant Type 3

sampling commences

Quota (n4) for Participant Type 4

sampling commences

Sample until quota 1 filled

Sample until quota 2 filled

Researcher Sample until quota 3 filled

Sample until quota 4 filled

Fig. 19.12 Visualisation of the quota sampling strategy

Disadvantages of quota sampling: • Capacity to generalise under the positivist pattern of guiding assumptions is weaker with a quota sample, compared to a stratified random sample. Generalisability may be further impaired, depending upon how data sources are classified or selected within each category. If you or an associate must exercise personal judgment about who to approach to fill each quota (which can happen in mall- or community-intercept samples making judgments based on appearance, such as age, ethnic background or age, for example), there may be a greater risk of bias associated with your sampling choices. • Representativeness may suffer if decline-to-participate rates differ within different categories. A high decline rate in a category may signal reluctance of category members to participate because of sensitivities associated with what you wish to ask or learn about, which means that a perception or characteristic of your research becomes a factor in influencing participation rates (thereby contributing to systematic error). You should note the decline-to-participate proportion in each category (i.e., number of participants approached but declined, divided by the total number of participants approached in the category) to check for any patterns that might interfere with your claim of representativeness. • Establishing quotas for too many dimensions and/or categories can make your sampling process complicated, render quotas difficult to fill and, paradoxically, produce an unrepresentative sample. A single dimension for a quota sample is relatively easy to implement if there are not too many categories (3 to 5 categories would be manageable; 10 categories would require much more effort to

19.3

Non-Probabilistic Sampling Strategies

863

implement across a much larger catchment area or sampling base). However, problems escalate when there is more than one dimension for classifying potential participants as each participant will be classified into one category of each dimension. The technical term for this way of combining categories for different dimensions or variables is called a factorial sampling plan. For example, suppose you wish to implement a quota sampling strategy using two dimensions to identify undergraduate students at a major metropolitan university to interview about their university experience: dimension 1 defined as Indigenous Identification with categories of Aboriginal/Torres Strait Islander and Australian/non-Indigenous Australian and dimension 2 defined as degree program enrolled the student is enrolled in, with categories of Maths & Science, Business, Education & Health, Humanities, Social Sciences, Law and Engineering. Table 19.1 illustrates this 2-dimensional quota sampling problem, showing the population membership in each category for the entire university (along with relevant percentages in parentheses). Implementing a proportional quota sampling strategy would be difficult enough using only the ‘Degree enrolled in’ dimension, where there are 7 categories to sample within. The dark-grey cell numbers in the right-hand column show what such a proportional quota sample might look like if the desired sample size was 2000 students. The medium-grey cell numbers in the bottom row show what a proportional quota sample based solely on ‘Indigenous identification’ might look like. However, when both dimensions are considered together, 14 categories of students are created, each comprising one pairing of an ‘Indigenous identification’ category and a ‘degree enrolled in’ category. The light-grey cell numbers show what a proportional quota sample might look like when a total sample size of 2000 is desired. Notice how small the representation of Indigenous students would be in the Maths & Science, Engineering and Law degrees (all with quotas less than 30). In multi-group samples, a desirable minimum sample size for each group would be 20 to 30 participants if valid comparisons between the groups are to be made. Thus, even though a proportional quota sample is supposed to produce a more representative sample, in this instance it does not because certain groups have too few participants in them. This is a common problem when some category combinations (e.g., Indigenous students and Maths & Science or Engineering degree enrolments) are very much smaller in number than other categories. If we were to scale up the proportional quota sample so that the smallest group (Indigenous Engineering students) had at least 20 participants as their quota, the total sample size required would balloon to 10,000 students (nearly half the population). It is hard to imagine that you would have the resources to enlist enough assistants to conduct 10,000 interviews (conducting 2,000 interviews would be tough enough as it is). Note that these considerations and constraints exist for the proportional stratified random sampling strategy as well. A more feasible attack on the problem would be to deliberately over-sample Indigenous participants and under-sample non-Indigenous participants.

864

19

How Do I Manage the Sampling Process?

Table 19.1 Complex proportional quota sample showing some inherent limitations Indigenous Identification Aboriginal/Torres Strait Islander Australian

Degree enrolled in Maths/Science

Student Population (row % )

75 (3.54% )

Proportional Quota Sample Business

Student Population (row% )

Education & Health

Student Population (row% )

310 (6.74% )

Student Population (row % )

Social Sciences

Student Population (row% )

Student Population (row % )

Engineering

Student Population (row % )

Total

1,460 (5.98% ) 120

106 760 (95.00% )

800 (3.27% )

4

61 22,560 (92.35%)

153

327

304

14 40 (5.00% )

426 3,990 (16.33% )

1,300 (89.04% )

Student PopulaƟon (row %) 1,870 (7.65%) ProporƟonal Quota Sample

5,210 (21.33% )

3,710 (93.98% )

160 (10.96% )

511

390

23

Proportional Quota Sample

6,250 (25.58% )

4,770 (91.55% )

280 (7.02% )

377

464

36

Proportional Quota Sample

4,600 (18.83% )

5,685 (90.96% )

440 (8.45% )

174

351

47

Proportional Quota Sample Law

4,290 (93.26% )

565 (9.04% )

Proportional Quota Sample

2,120 (8.68% ) 167

26

Proportional Quota Sample Humanities

2,045 (96.46% ) 7

Proportional Quota Sample

Student Population within Degree (column % )

Non-Indigenous Australian

1847

65 24,430 (100.00%) 2000

19.3.3 Volunteer Sampling In volunteer sampling, you publicise the nature of your research project and provide your contact details for any people interested in volunteering to participate. In many cases, an offer of remuneration or recompense for time spent participating accompanies the request for volunteers. The people who respond and agree to participate then comprise your sample. With this sampling strategy, sample size is determined by how many people agree to participate, making it very difficult to anticipate the final sample size that will be achieved. Many laboratory-based positivist studies in the psychology, business and marketing disciplines rely on such samples (e.g. volunteers from undergraduate, postgraduate or MBA classes at a university), often in exchange for course or unit credit. Figure 19.13 provides a visualisation of the volunteer sampling process. The key distinguishing feature is

19.3

Non-Probabilistic Sampling Strategies

865

Participant 2

Participant 1 non-participant

Participant 8

Participant 3 non-participant Participant 7

Researcher non-participant

Participant 6

Participant 4 non-participant

non-participant

Participant 5

non-participant

Fig. 19.13 Visualisation of the volunteer sampling strategy, where 8 participants volunteer to participate in response to your appeal

that potential participants approach you to participate (based on information they have acquired or been given about the research) rather than you approaching the participants. There are now online resources available to facilitate researchers in obtaining a volunteer sample of relevant people. These resources often constitute very large sometimes multi-national participant pools or online panels assembled, maintained and regularly updated by various companies where researchers can access volunteers willing to participate in many types of research projects (usually with an expectation of remuneration for their efforts). This type of resource is often very useful for research in the Survey, Cross-Cultural or Explanatory research frames that is guided by the positivist pattern of guiding assumptions and seeks to gather quantitative data. Marketing/consumer, organisational and psychological research are three major areas of research where such online data participant pools are used. Examples of companies offering access to online participant panels include: OzPanel (Australian online survey community maintained by Roy Morgan Research; see http://www.roymorgan.com/services/online-research/oz-panel); Qualtrics Online Sample (see https://www.qualtrics.com/online-sample/); Survey Monkey Audience (see https://www.surveymonkey.com/mp/audience/); and SSI SurveySampling.com (see https://www.surveysampling.com/audiences/consumeronline/). The organisations that maintain such panels record a variety of attributes of their panel members so that online-based (and, in some instances, telephone-based, mobile-based or site intercept-based) questionnaires can be appropriately targeted to meet your specific needs. Incentives for panel members might include an hourly or per survey participation payment or product in lieu of payment (such as a T-shirt or hat), charitable donation or sweepstakes/lottery entry

866

19

How Do I Manage the Sampling Process?

and/or accumulation of points for a retailer gift card or product redemption. Organisations like Qualtrics and SurveyMonkey offer support for all aspects of questionnaire-oriented data gathering (and, in some cases, analysis) activities, allowing you to manage, online, all aspects of your panel or sample, from recruitment to questionnaire design and delivery to reward dispensation (see, for example, http://success.qualtrics.com/rs/qualtrics/images/Panel_Management_ Guide.pdf). Advantages of volunteer sampling: • Volunteer sampling is a cost-efficient way of assembling a sample of participants for research. This sampling strategy is often useful for researchers who lack the resources or access to the desired range of data sources, particularly if they are geographically dispersed. Utilising an online panel can extend the ease with which you can sample from a larger perhaps even national pool, against relevant participant attributes (which facilitates quota sampling, for example) and if the organisation managing the online panels has partners or branches in other countries, cross-cultural research can be facilitated (SurveyMonkey Audience Global Panel, for example, has partners in 63 different countries). • In a volunteer sample, participants tend to be more motivated to engage with data gathering tasks (such as completing a questionnaire or participating in an interview) and, therefore, tend to be less likely to withdraw their participation once they have commenced. In a volunteer sample, motivation can be stimulated by offering incentives for participation in the form of an exchange: e.g., receive desired outcome X in exchange for completion of a 30-min online questionnaire. Incentives may be tangible, as with the offer of money or a product in lieu of money, lottery entry or points that can be redeemed for a desired product, or intangible, as with satisfying the person’s curiosity about what you are trying to learn or allowing the participant to reveal something about themselves or something they know (meeting an ego-satisfaction need) or to simply be of assistance to you (meeting an altruistic need). Response rates can be enhanced if the incentives/rewards on offer are desirable to the types of participants being sought, which means the choice of incentives to offer should be strategic and, if possible a range of possible incentives should be on offer so that participants can choose their preferred outcome in compensation for their participation. • Volunteer sampling is easily combined with almost any other sampling strategy. For example, convenience or quota samples are often volunteer samples. Given the ethical climate surrounding social and behavioural research today, it is possible to view every sampling scheme as being ‘volunteer’ in nature because of each participant’s right to give their informed consent to participate. Disadvantages of volunteer sampling: • A major drawback of volunteer samples is lack of representativeness. Generalisability is, thus, severely curtailed in such samples. This is because very often there is a qualitative distinction between people who volunteer to

19.3

Non-Probabilistic Sampling Strategies

867

participate and those who don’t. These motivational differences can, conceivably, bias any measurements obtained on the sample simply because you never get access to the types of people who don’t volunteer. There are many different motivations to volunteer. For example, altruism may motivate people to volunteer because they want to help you. However, this motivation may translate into them helping by giving answers they think you are looking for, a mindset that leads to invalid measurements because it creates a demand characteristic. Participation because one is curious about your research itself can be a motivation, but if that curiosity is not satisfied or if the project is not what they thought they volunteered for, this may influence the duration and quality of their participation. If you explicitly offer remuneration or recompense, then motivation to volunteer may be driven more by anticipation of receiving the offered reward and less by intentions to participate seriously, which may render any information participants provide as suspect or as an instrumental means to gain the reward. The mindset of people who agree to be enlisted on online panels may differ from those who do not have the opportunity to or do not wish to enlist (e.g., being on a panel may provide a consistent if small source of income) and this could influence how information they provide in any research they volunteer for should be interpreted. • The incentives on offer to motivate volunteering must be seen by potential volunteers as adequate recompense for the length of time and effort required during their participation. Also, the incentives offered must be the incentives provided at the end of participation. If, in either case, there is any perceived mismatch between the incentives likely to be achieved for participation and the volunteer’s initial expectations before volunteering, response rates for participation can be negatively impacted, because withdrawal rates from your research are likely to be higher. • The volunteer sampling strategy is a passive strategy for obtaining a sample, where you simply publicise your research and need for participants and wait until volunteers contact you, which gives you far less certainty about the kinds of people that end up in your sample. If you use an online panel, this gives control over your sample composition largely to the organisation controlling access to the panels. Other strategies, such as quota sampling or purposive sampling are active strategies in that you explicitly target and approach those you want to participate in your research, giving you a bit more certainty over who to approach to participate. • If you use an online participant panel, managed by a specific organisation such as Roy Morgan Research, Qualtrics, SurveyMonkey or SSI, it should be recognised that these organisations are businesses and as such need to maintain and grow their business. You thus need to be aware that using one of these companies to sample data sources means you will be using that company’s software systems to manage the process and relying on the company’s policies for assembling and maintaining their online panels. You have no control over how these companies actually attract participants into their panels or pools (for example, Survey Monkey Audience refers to accessing its online panels as

868

19

How Do I Manage the Sampling Process?

‘buying responses’, which can cast some doubt over the potential validity of the responses from participants). Also, a specific company may not gather all the relevant attributes you might need to determine who should be approached to participate in your research.

19.3.4 Purposive Sampling Purposive sampling is a very important and commonly used sampling strategy, particularly for research guided by an interpretivist/constructivist or other non-positivist pattern of guiding assumptions. The essence of this sampling strategy is quite straightforward: you specifically and deliberately choose the precise data sources you want to connect with because they meet one or more of the selection criteria relevant to your research questions (we say data sources here because while human participants are most commonly purposively sampled, it is also possible to purposively sample events, documents, media stories, websites and so on; see the visualisation in Fig. 19.14). In implementing this sampling strategy, you might choose data sources because of their role or purpose in a specific group or organisation, because of the perspective you think they might offer, because of information they might possess (and be willing to share in the case of human data sources), and/or because of a specific characteristic or background the data source

Data Source Context

Data Source 1

Data Source 2

Data Source 8

Data Source 3

Researcher Data Source 7 Data Source 4 Data Source 6 Data Source 5

Fig. 19.14 Visualisation of the purposive sampling strategy where you identify 8 strategically important data sources to connect with

19.3

Non-Probabilistic Sampling Strategies

869

possesses or reflects (such as their historical or strategic value, gender, ethnic background or management level or leadership position). With this strategy, what is important is not how many people (or other data sources) you gain access to, but who or what you gain access to. The overall goal is to tap into those data sources that offer a perspective that might be relevant to what you want to learn in a specific context. Purposive sampling can ensure that the most appropriate data sources are included in your sample. For this sampling strategy to work well, you should be familiar with the context in which you are collecting data so that you can properly identify and target the appropriate data sources. It is a commonly employed sampling strategy within the Descriptive, Exploratory, Action, Transdisciplinary, Indigenous, Feminist, Evaluation or Developmental Evaluation research frames. The purposive sampling strategy can be used in research guided by the positivist pattern of guiding assumptions, but generally only in the context of the Descriptive, Exploratory or Action research Frames. Any inferences drawn from statistical analyses of quantitative data gathered in such cases will necessarily be more descriptive in intent and greatly restricted in generalisability. Different modes of purposive sampling strategy for choosing research participants (generically referred to as ‘cases’ to signal that choice of participants is based on some characteristic or logical basis defining the ‘case’) have been identified by Bryman and Bell (2015) and Cohen, Manion, and Morrison (2011) amongst others. Depending upon the goals of your research and the desired depth/breadth balance for learning, one (or more, especially when used synergistically) of the following purposive sampling modes may be useful: • Typical Case mode: This mode involves sampling those cases you think are the most typical or representative of perspectives available in the research context. Your goal here is to build up a picture of what is typical or ‘normal’ within the research context. The interpretive and theoretical stories that emerge from this mode of purposive sampling will tend to reflect the majority or dominant viewpoints, emphasising depth of learning over breadth of learning. • Atypical Case mode: This mode involves sampling those cases you think are atypical (e.g., you consider them to be unusual, extreme or deviant, in the minority, silenced or hidden) in the research context. Your goal here is to deliberately seek cases that will likely require you to enlarge, modify, qualify or refine your emerging interpretive and theoretical accounts, emphasising breadth of learning over depth of learning. You can use it as a mode to examine contrasts with the perspectives acquired using the typical case mode and it shares similarities with the negative case sampling strategy (to be discussed below). This contrast logic can help you meet the interpretivist/constructivist quality criterion of sufficiency as long as you take concrete steps to ensure authenticity of the voices sought and heard, since atypical case participants may perceive risks or fear consequences associated with revealing or expanding upon their atypical views with you. • Case Diversity mode: This is a weaker and more affirmational form of the atypical case mode where you purposively sample cases that will likely provide

870

•

•

•

•

•

•

19

How Do I Manage the Sampling Process?

access to the most diverse range of perspectives. It thus emphasises breadth of learning over depth of learning. This mode can help you meet the interpretivist/ constructivist quality criterion of sufficiency. Criterion-based mode: This mode involves sampling cases that meet a specific criterion that you have identified, such as employees hired within the post three months or newly enrolled students and their parents. This mode of purposive sampling can be useful in enhancing the representativeness of the views you obtain and may be synergistically combined with quota sampling for best effect. In that regard, the criterion-based mode tends to emphasise breadth of learning over depth of learning. The goal is to not only enhance sufficiency but also to enhance extensional reasoning more generally, if transportability of interpretations is important to argue. Critical/Pivotal Case mode: This mode focuses on sampling cases that you identify as critical or pivotal in the research context. The cases might be people involved in a critical incident or event in the research context, people most likely to hold pivotal knowledge about an issue or people who could provide good test cases for emerging theoretical propositions (as in grounded theory research). This mode of purposive sampling is oriented toward maximising the learning value from interpretations and theoretical accounts. Opportunistic Case mode: This mode cannot be planned or implemented from the commencement of your research; rather it is an emergent sampling mode focusing on connecting with specific data sources that were initially unknown or unavailable at the start of your research, but later become known/available to you as your research journey progressed. The opportunistic case mode is oriented toward enhancing sufficiency of your research story and may be particularly useful in the Developmental Evaluation, Indigenous, Feminist and Transdisciplinary research frames. Case Intensity mode: This mode of purposive sampling pursues cases that provide you with a deeper level of exposure to the perspectives of a specific group, event or situation, enacting a deliberate depth of learning emphasis. This mode can provide you with a richer array of data from which to draw and illustrate your emerging understanding of and theoretical account for a specific group’s perspectives. Homogeneous Case mode: This mode focuses on sampling data sources based on their similarity to each other, where you explicitly define the dimensions along which similarity will be gauged (e.g., high or low performers at work or in school). The homogeneous case mode can be useful in supporting your intentions to build up a compare-and-contrast story between different homogenous groups. It is a variant of the case intensity mode, where the homogeneous groups form the focus for intensive sampling. The Feminist and Indigenous research frames often rely on this mode for comparing and contrasting the gendered or indigenist experiences of people. The Cross-Cultural research frame can also utilise this purposive sampling mode to good effect. Reputational Case mode: This mode is a variant of snowball sampling (to be discussed below), where key data sources are identified based on the

19.3

Non-Probabilistic Sampling Strategies

871

recommendations of others, where those recommendations are based on the reputations of those data sources. Reputation may be based on a person’s possession of specialist knowledge, insights or experiences or on their political importance or power, either because of the person’s political role in some relevant context (e.g., they are involved in policymaking or regulation setting, see Farquharson, 2005) or because it would be politically prudent for you to seek that person’s views. Using this mode of purposive sampling may enhance sufficiency if the consideration for a person’s inclusion in the sample is not politically motivated. Politically-motivated purposive sampling may help you to achieve other goals beside your immediate research goals. For example, interviewing a key political player in the research context of interest may open doors to accessing new resources or other sources of data that you would not previously have had access to (leading to opportunistic case sampling) and/or may open other pathways for research dissemination and impact. The drawback here is that politically-motivated sampling can create a very complex relationship between you and your stakeholders that can be difficult to successfully navigate in the pursuit of convincing research. • Revelatory Case mode: This mode involves you seeking access to the first members/founders of a particular group or organisation or developer(s) of an innovation in order to gain preliminary developmental insights with respect to the group or innovation. This can be especially important if you need to build up the historical context behind the group or innovation to serve as a backdrop against which to gauge new learning. In this sense, the revelatory case mode has a depth of learning emphasis. This mode may be especially useful in the Exploratory, Transdisciplinary, Evaluation or Developmental Evaluation research frames. Advantages of purposive sampling, for research guided by interpretivist/ constructivist or other non-positivist patterns of assumptions: • Purposive sampling is the dominant sampling strategy for interpretivist work involving gathering of qualitative data. Its value comes from the capacity to direct your learning through targetted contact with the most relevant data sources (without having to rely on statistical probabilities to hopefully provide those data sources). Purposive sampling is very difficult to implement without some degree of contextual knowledge on your part as it is that knowledge base you draw upon to target the data sources you want to connect with. This implies that, for effective purposive sampling, you might need to undertake some preliminary exploratory research to help you build the required contextual knowledge base (an ideal situation for the exploratory sequential MU configuration). • The purposive sampling strategy, when used properly, can help you build an argument for sufficiency in your coverage of perspectives. If there are gaps in the sample, you fill them by simply targeting one or more data sources, using one or more of the purposive sampling modes to provide the targeting logic.

872

19

How Do I Manage the Sampling Process?

Representativeness in purposive samples is less of an issue than in many other sampling strategies because you can generally take active steps to enhance representativeness wherever and in whatever way it is lacking. • Purposive sampling is a very versatile strategy as it can be used synergistically with many other non-probability sampling strategies (we have also seen that different modes of purposive sampling can be used synergistically). For example, the quota sampling strategy is often implemented using purposive sampling to fill the quotas for each group. Convenience sampling can be implemented purposively. As we will see later, purposive sampling can be usefully combined with different theoretical sampling strategies as well as with the contextual sampling strategy. Disadvantages of purposive sampling, for research guided by interpretivist/ constructivist or other non-positivist patterns of assumptions: • The biggest and most difficult problem associated with the purposive sampling strategy is avoiding bias in the selection of participants. Since you are deliberately targeting specific individuals for their participation, you must be very clear as to why you are choosing them. If, for example, your purposive sampling choices comprise people you feel will open up best or be easiest to talk to, people who you may know from previous encounters, or people who you are ‘attracted’ to by virtue of similarity in worldviews, then your sample will be biased. The nature of the bias will be to slant the emerging interpretations toward perspectives that resonate more closely with your own and will signal that you have not effectively managed your own preconceptions when deciding which data sources to pursue. It negatively impacts both authenticity and sufficiency and will accordingly reduce convincingness. The purposive choice logic should always focus on who or what should you connect with to enhance and enrich your understanding of others’ perspectives (not reflections of your own perspective). Your research journal can help you to manage the risks associated with purposive sampling bias by providing a record of your reasoning behind all sampling choices and serving as a vehicle for keeping your own preconceptions separate from the research process. Downstream, this can help you to inject transparency into your research stories. • Another disadvantage of purposive sampling may occur if you seek a diverse range of perspectives (breadth of sampling) at the expense of seeking confirming or disconfirming information with respect to each perspective (depth of sampling). You need to explicitly strike the right balance between breadth and depth for your research to be convincing. This is where different modes for purposive sampling, used synergistically, can realise their full potential, value and utility. • The purposive sampling strategy can only work effectively if all potential participants sampled agree to participate and maintain their authentic participation throughout. This is especially important if your purposive sampling mode is criterion-based (such as sampling all people in leadership positions in a school or company), atypical case-based (where the risk of atypical participant decline

19.3

Non-Probabilistic Sampling Strategies

873

is much higher than for the typical case mode) or case diversity-based (where cases in the minority or on the fringes are generally less likely to agree to participate). Here we have an instance where the ethical rights of people (and your associated obligations as researcher) can potentially interfere with the intended scope and shape of your research and the sufficiency of the interpretations and stories you convey. The most effective way to manage this problem is to ensure absolute transparency in what your purpose is when seeking participants coupled with assembling very detailed notes in your research journal about the intentions, outcomes and implications from the purposive sampling strategy as it unfolded during your research journey.

19.3.5 Snowball Sampling Snowball sampling is also known as network sampling and is another strategy often used in research guided by an interpretivist/constructivist or other non-positivist pattern of assumptions. The strategy generally commences with your purposive or convenient selection of one or more key participants or other data sources and, as part of the data gathering process from those sources, you seek information on who else might be useful to approach for participation or what other data sources (e.g., documents, reports, media stories) might be useful to seek out. Figure 19.15 Researcher

Data Source 1

Data Source 3

Data Source 2

Data Source 4

Data Source 8

Data Source 5

Data Source 7

Data Source 6

Data Source Context

Fig. 19.15 Visualisation of the snowball sampling strategy leading to the cascading identification of 8 data sources

874

19

How Do I Manage the Sampling Process?

provides a visualisation of the snowball sampling process. Unlike the purposive sampling strategy, the snowball sampling strategy works well in contexts that you are not very familiar with. You rely, instead, on the knowledge of ‘insiders’ and social/collegial/professional networks to expand, extend and enrich your sample. This cascading effect of data source identification can yield reasonably large samples with relatively little effort and expenditure of resources. The trick to making the snowball sampling strategy work most effectively is knowing how to solicit the information about other data sources to connect with. It is good practice for you to be very clear about the reasons you are seeking such information and to be very open about what you hope those other data sources might be able to provide. Your goal is to invite the participant to suggest the most appropriate data sources, i.e., to leverage their contextual and personal knowledge in ways that will best benefit your research aims. The snowball sampling strategy is generally most useful for research conducted in the context of the Exploratory, Transdisciplinary or Development Evaluation research frames. In the Indigenous research frame, snowball sampling may be required to gain access to desired data sources. In this frame, you would need to first approach relevant tribal elders, who would serve in a dual role as first-level participants but also as primary gatekeepers, to not only seek permission to conduct your research but also to learn who might be the most important people to talk to or data sources to examine. Access to data sources suggested by elders may be accompanied by conditions and constraints intended to protect and appropriately delimit the knowledge they share with you; conditions and constraints that you will be ethically bound to honour. Advantages of the snowball sampling strategy: • A suitable sample can be built up relatively efficiently and cost effectively with a reasonable likelihood that you will be pointed to additional appropriate data sources. • In the absence of contextual knowledge, the snowball sampling strategy provides a way for you to quickly identify and focus on key sources of information, harnessing the social and professional networks of people within the research context. • Cohen et al. (2011) argued that the snowball sampling strategy may provide you with potential access to hard-to-reach groups/people (e.g., minorities or marginalised groups, gangs, elite executives, convicted felons, rural/remote communities, Indigenous communities) and, perhaps, even to groups/people initially hidden from or unknown to you (e.g., school dropouts, problem gamblers or drinkers, sex workers, informal groups, such as poker players or lottery group players or syndicates). If this happens, you can benefit from expansion of your contextual knowledge (i.e., you learn about new facets of the research context) as well as from potential access to new data sources.

19.3

Non-Probabilistic Sampling Strategies

875

Disadvantages of the snowball sampling strategy: • Participants will often only suggest other data sources for you to connect with that either share or embody similar perspectives to theirs, is someone they have close contact with (e.g., a friend) or a working relationship with (e.g., a colleague or peer), or is someone they think might be willing to help you out. These criteria, often unknown at the time participants suggest others to contact, can lead to biased samples that reveal only a limited range of perspectives. It may also be the case that the data sources suggested by a participant may be constrained by their own role/position within the social or organisational hierarchy of the research context (e.g., a school, an organisation, a community) and by the limits to their knowledge associated with that role or position. Thus, it can be difficult to achieve a representative sample using the snowball strategy, which then imposes constraints on sufficiency. It is possible to take steps to reduce or perhaps minimise the biases that can emerge when employing snowball sampling by using the strategy only in combination with other sampling strategies. For example, snowball sampling could be used in conjunction with specific modes of the purposive sampling strategy, so that a wider range of selection criteria for data sources can be employed. • Many institutional ethics committees will not approve research that employs snowball sampling to identify human participants. The reason typically put forward is that asking others who might be a useful person or persons to connect to violates that named person’s ethical right to privacy. Researchers in such institutions generally need to work around this constraint by employing other sampling strategies that do not require participants to reveal names and contact details for other potential participants. Often this leads to the use of the volunteer sampling strategy, which relies on self-nomination for participation, or purposive sampling, which uses a researcher-directed approach. • The fact that participants suggest specific data sources to connect with does not automatically grant you the right to access those data sources or, in the case of human data sources, to assume that they have given their informed consent to participate. Where human data sources are concerned, the protocol for obtaining informed consent must be implemented at every level of the snowball sampling process. Where non-human data sources are concerned, permission to access these, if required, must be obtained via the appropriate gatekeeper(s). • Sufficiency and representativeness are generally much more difficult to achieve using a snowball sample because of the selective nomination of other data sources to connect with. Participants can only identify only those data sources they are familiar with or have relevant knowledge about. Any data sources they don’t know about do not get nominated, which can lead to gaps in your sample.

876

19

How Do I Manage the Sampling Process?

19.3.6 Contextual Sampling The contextual sampling strategy involves (1) identifying and selecting data sources, both inside and outside the research context, that will facilitate acquiring a wider range of perspectives as well as a greater depth of understanding of that context and (2) selecting occasions and situations for data gathering purposes. With (1), you seek information from sources; with (2), you choose when and where you should seek information from sources. Thus, the general goal of this strategy is to enrich your contextual knowledge from both inside and outside viewpoints. Figure 19.16 provides an illustration of this strategy where you choose three distinct time periods in which to focus your data collection within the research context and two out-of-context data sources that can offer perspectives on your research context. Then, within the research context, you select 3 events and 3 specific situations or sites in which to gather data and, in addition, choose 2 in-context data sources to seek information from. The contextual sampling strategy can be extremely useful for research conducted within the Case Study research frame and potentially useful in the Descriptive, Exploratory, Evaluation, Developmental Evaluation, Indigenous, Feminist or Action research frames. The contextual sampling strategy may be employed in research under the guidance of a positivist pattern or an interpretivist/constructivist pattern of assumptions. Contextual sampling under the positivist pattern of guiding assumptions tends to focus more on time, event or situation/site sampling. For example, when using systematic observation as a data gathering strategy, contextual sampling of

Time/Time Period 2

Time/Time Period 3

Data Source 1

Time/Time Period 1

Data Source 2

Researcher Event 1 Event 2

Situation 2

Data Source 3 Event 3

Data Source 4

Situation 3 Situation 1

Research Context Fig. 19.16 Visualisation of the contextual sampling process embodying both an outside focus looking in at the research context and an inside focus within the research context

19.3

Non-Probabilistic Sampling Strategies

877

observation occasions and sites may be randomly or systematically structured so that some argument about representativeness of the observations can be sustained. In a time-and-motion study of a factory assembly line, for example, you could devise a scheme where you record an observation at a specific site along the assembly line every three minutes (a systematic sample) or on a random basis (simple random sampling). In a time-aligned longitudinal MU configuration, contextual sampling of time periods (e.g., months or years) may be done for econometric or financial time series research (perhaps requiring access to a secondary or archival database for sourcing the data). Contextual sampling under an interpretivist/constructivist pattern of guiding assumptions focuses much more widely in order to provide maximal contextual information within specific time periods spent in the research context. Information from outsider data sources (e.g., former employees of a company you are conducting participant observations within or media stories about the company) may also be sampled to enhance contextual understanding providing ‘outsider-looking-in or former ‘insider’ perspectives). For participant observation data gathering within the research context, the focus of the strategy shifts toward selecting relevant occasions, events or situations/sites in which to make and record observations as well as choosing data sources you wish to connect with. In combination with other sampling strategies, you could decide which meetings or tea room gatherings to attend and observe during a day, e.g., using a quota sample or a purposive sample. Advantages of the contextual sampling strategy: • The contextual sampling strategy is more of a mindset for sampling rather than a specific set of rules for obtaining a sample. For this reason, the strategy must be synergistically used in combination with other sampling strategies, either probabilistic, non-probabilistic or both. Contextual sampling is often successfully realised when used in tandem with the purposive sampling strategy, especially when it comes to selecting events and situations in which to gather data and data sources from which information will be sought about those events and situations. • In research guided by the positivist pattern of assumptions, contextual sampling is one way to implement the principles of representative design to provide representative experiences for research participants. • In research guided by an interpretivist/constructivist pattern of assumptions, the contextual sampling strategy can help you build richness and sufficiency into your contextual stories. By seeking contextual information from outside the research context as well as contextualised evidence from within, you can build a more convincing story that is authentic and sufficient as well as triangulated. • Where your resources are tightly constrained, the contextual sampling strategy provides a way of focusing in on the most important sources of information to seek (whom, when and where). For example, you may only have enough resources to gather data within a research context during two time periods

878

19

How Do I Manage the Sampling Process?

(which may be chosen to suit your convenience, e.g., between semesters of teaching, or the convenience of the organisation, e.g., during slow business periods). Data gathering from selected data sources outside the research context can occur off-site between these time periods. Disadvantages of the contextual sampling strategy: • The contextual sampling strategy, in and of itself, does not ensure that sampling will be representative or sufficient. The success of the strategy depends entirely upon the synergistic use of other sampling strategies which implies that the disadvantages of those strategies will persist here. What contextual sampling does offer is the opportunity to mitigate some of those limitations through being paired more than one type of sampling strategy (e.g., using convenience sampling to choose time periods and events and purposive sampling to choose situations or sites for data gathering and data sources to connect with). • The dual focus on extra-contextual data sources and within-context data sources may yield conflicting stories, accounts or perspectives on common issues. If you are working under an interpretivist/constructivist pattern of guiding assumptions, you would have to work assiduously to produce authentic yet sufficient interpretations, especially when your own researcher perspective also must be managed. Effective use of your research journal should help in this regard, but it may still provide cognitive challenges for you. • There is a risk that use of the contextual sampling strategy may consume additional resources if the dual focus creates a multiplying effect for the number of data sources you connect with on multiple occasions or at multiple events, especially if paired with the quota sampling strategy. The potential for this multiplying effect can be reduced if purposive sampling is employed wherever possible and if the number of time periods for data gathering is kept small. This risk is more likely under an interpretivist/constructivist pattern of guiding assumptions.

19.3.7 Theoretical Sampling The theoretical sampling strategy is most useful in the context of an interpretivist/ constructivist pattern of guiding assumptions when undertaking a grounded theory approach (see, for example, Bryant & Charmaz, 2007, Chap. 11, in particular; Corbin & Strauss, 2008, Chap. 7; Charmaz, 2014, Chap. 8; Morse et al., 2009). It can be thought of as a type of purposive sampling where emerging theoretical ideas, concepts and relationships drive the sampling logic. Theoretical sampling offers a mindset for sampling whereby you commence data collection from a preliminary sample of participants, events and so on, and commence data analysis at the same time. As you begin to form theoretical interpretations and ideas about meanings, relevant concepts and relationships, these are then used to guide your further

19.3

Non-Probabilistic Sampling Strategies

879

Research Context Researcher Event 1 Data Source 1 Situation 1 Data Source 2

Researcher forms preliminary interpretations & theoretical ideas/explanations from the data – uses these to guide the next stage Researcher Seeking data to test initial interpretations & theoretical explanations Situation 2 Data Source 3

Data Source 4

Research Context

Event 3

Data Source 5

Event 2

Fig. 19.17 Visualisation of the theoretical sampling strategy

sampling in pursuit of evidence that will help to refine, shape and even explicitly test your preliminary theoretical ideas and interpretations that have emerged. This logic is what makes the theoretical sampling strategy useful for research conducted in the Explanatory research frame, but under an interpretivist/constructivist as opposed to the positivist pattern of guiding assumptions. The theoretical sampling strategy can also be very useful in the Case Study research frame (especially where the longitudinal case-based MU configuration is used) as well as the Feminist and Indigenous research frames, where the explanatory or deeper learning sequential MU configuration might be used. Figure 19.17 provides a visualisation of the theoretical sampling process where you purposively connect with two data sources and access an event and a situation and commence analyses of the resulting data. Your preliminary analyses help learning, ideas, concepts and relationships to emerge that then signal areas where you need more data to help build up a logical and coherent interpretive explanation (the ‘grounded theory’), leading you to purposively sample 3 more data sources, 2 more events and another situation. Note how easy it would be to replace the initial stage of theoretical sampling (the top bubble in Fig. 19.17) with the contextual sampling strategy offered in Fig. 19.16. In fact, many grounded theorists (Charmaz

880

19

How Do I Manage the Sampling Process?

would be amongst these) would argue that this would be an important synergy of sampling strategies to harness, to ensure that the most well-rounded version of the grounded theory can emerge. Note that Fig. 19.17 offers a simplified representation of the theoretical sampling process involving only two stages. In practice, many more stages will likely be required, a logic that leads to the sequential/saturation sampling strategy to be discussed later. Advantages of theoretical sampling: • The theoretical sampling strategy is more of a mindset for sampling rather than a specific set of rules for obtaining a sample. For this reason, the strategy must be synergistically used in combination with other non-probabilistic sampling strategies. As indicated earlier, theoretical and contextual sampling are often used synergistically in a grounded theory investigation. Theoretical sampling is often successfully facilitated when used in tandem with the purposive sampling strategy, especially when it comes to selecting events and situations in which to gather data and data sources from which you will seek information about those events and situations that preliminary theorising suggests would be useful to focus on for evidence as well as for testing emergent theoretical ideas. • The theoretical sampling strategy is tailor-made for a grounded theory approach is that it provides an explicit method for continuously testing and refining your ideas until the emergent stories stabilise. What drives the sampling process are emergent rather than pre-existing theoretical ideas, concepts and relationships (a logic entirely consistent with an interpretivist/constructivist pattern of guiding assumptions). • The theoretical sampling strategy is an explicitly adaptive sampling strategy. Early gaps in sampling and/or gaps in knowledge provided in the first stage of sampling can be easily rectified, qualified, extended and/or expanded upon by subsequent stages of sampling (an adaptive/flexible MU configuration could be well-suited for such purposes). New directions in sampling focus can thus easily be accommodated and, with respect to the development of grounded theory, is highly desirable. The sampling adaptability inherent with this strategy extends to affording opportunities to enhance sample representativeness (recall the different modes of purposive sampling that could be brought to bear here; we explore one specific variation of theoretical sampling below with negative case sampling) as well as sufficiency of evidence (although this latter is more likely to be realised using the sequential/saturation sampling strategy). Disadvantages of theoretical sampling: • The theoretical sampling strategy, in and of itself, does not ensure that your sampling will be representative or sufficient. The success of the strategy depends entirely upon your synergistic use of other sampling strategies which implies that the disadvantages of those strategies will persist here. What theoretical sampling does offer is the opportunity to mitigate some of those limitations through the sequential logic in theoretical targeting of your sampling.

19.3

Non-Probabilistic Sampling Strategies

881

• Theoretical sampling is not something that you can completely plan for as it depends upon a sequential learning logic for its development and unfolding. The first stage can be effectively planned; the subsequent stage(s) must remain open-ended until you have begun to form some preliminary views that provide guidance for what to look for next. In this sense, planning for the allocation of resources throughout the research journey can be a bit tricky, given the unknowns that persist at the commencement of data gathering. This also means that an evolutionary MU configuration is likely to be most workable. • The theoretical sampling strategy tends to reinforce a confirmatory data gathering logic. That is, it is oriented toward seeking out further supporting data for your emerging theoretical ideas. However, you need to avoid premature closure on theoretical ideas, concepts and relationships. This is something that requires maintaining an open-mind and being willing to jettison ideas, concepts and relationships that don’t work in favour of those that do work. Unless you take active steps to counteract a confirmatory mindset, you may end up with an overly restricted, perhaps even biased, theoretical account (especially if your own preconceptions and expectations have not been effectively managed throughout the theoretical sampling process). One synergistic strategy for avoiding this problem is to combine theoretical sampling with negative case sampling (see discussion below) so that you deliberately seek confirming evidence as well as disconfirming evidence. It is the juxtapositioning of, and balance in focus between, seeking confirming and disconfirming evidence that allows a truly robust and useful grounded theory to emerge.

19.3.8 Negative Case Sampling The negative sampling strategy (also known as deviant case sampling; see, for example, Neuman, 2013) can be considered as an inverse (because it seeks to disconfirm rather than confirm) variant of theoretical sampling (in grounded theory research) as well as a mode of purposive sampling more generally. It involves the deliberate search for extreme, unusual or disconfirming evidence for specific interpretations and theoretical explanations, ideas, concepts and/or relationships. It is a strategy that is breadth-focus in intent and congruent with interpretivist/ constructivist or other non-positivist patterns of guiding assumptions. Using this strategy, divergent, marginal, silenced or other unusual perspectives, events or situations are deliberately sought to motivate you to enhance/enrich/enlarge/prune/ refine/correct your interpretations and theoretical understandings by incorporating the learning you acquire from such perspectives. Figure 19.18 displays a visualisation of the negative case sampling process (which on the surface looks very similar to the visualisation of theoretical sampling offered in Fig. 19.17) where you purposively connect with two data sources and access an event and a situation and commence analyses of the resulting data.

882

19

How Do I Manage the Sampling Process?

Research Context Researcher Event 1

Data Source 1

Data Source 2

Situation 1

Researcher forms preliminary interpretations & theoretical ideas/explanations from the data – uses these to guide the next stage Researcher Seeking data to disconfirm early interpretations & theoretical explanations so they can be refined, rounded out & enhanced Situation 2 Data Source 3 Data Source 4

Research Context

Event 2

Fig. 19.18 Visualisation of the negative case sampling strategy

Preliminary learning emerges that then points to the need to pursue disconfirming evidence for the emerging theoretical story, leading you to purposively sample two more data sources, another event and another situation. This sampling strategy will only work if you commence data analysis simultaneously with data gathering (as is typical in participant observation or grounded theory research, for example). This gives you preliminary evidence to form the initial impressions and interpretations needed to identify and pursue negative or disconfirming cases. You also need to be aware that there are several different ways to define a negative case. A negative case might be a data source that offers/provides/reveals a minority viewpoint, a disaffected viewpoint, a viewpoint that challenges the dominant stories that are emerging or a completely different perspective (perhaps one considered deviant or abhorrent by the dominant group or organisation). Advantages of the negative case sampling strategy: • The negative case sampling strategy, more than any other, reveals a stark contrast between positivist and interpretivist/constructivist guiding assumptions in the way that unusual observations and ill-fitting evidence are handled. Under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, it is often the unusual, the extreme, or the marginalised perspective

19.3

Non-Probabilistic Sampling Strategies

883

that motivates you to revise your understanding of people within a specific context. This reflects an inclusionary logic in handling unusual or ill-fitting evidence and means that such evidence should be pursued rather than avoided. Under the positivist pattern of guiding assumptions, the analysis of quantitative data using statistical methods requires that unusual observations be treated as outliers and, therefore, as something to be argued away, discounted or deleted from the sample (because they create instabilities and inaccuracies in statistical estimates if retained in the analysis). This reflects an exclusionary logic. [There is, in fact, a well-developed set of statistical approaches that target the identification and treatment of outliers.] Negative case sampling thus intentionally seeks the outliers that a positivist approach would try to ignore or delete. • The inclusionary logic of the negative case sampling strategy works to enhance not only the representativeness of your sampling but also helps to ensure sufficiency in the evidence you gather. In a grounded theory investigation, this means that you access the evidence needed to ensure that you can develop the most robust and useful theoretical account. This logic is also congruent with an ethical position that all voices in a context should be heard, when seeking to understanding what is going on in that context. You seek to negate the tendency for marginalised or disenfranchised voices to be ignored or silenced. However, this logic need not be reserved for grounded theory development in the context of the Explanatory research frame. It can also be very useful in research frames that are guided by a critical social science or Indigenous pattern of guiding assumptions (e.g., the Action, Feminist or Indigenous research frames). Disadvantages of the negative case sampling strategy: • One problem associated with negative case sampling is how do you identify and connect with relevant negative cases? Sometimes, it will be obvious to you where you should find negative cases because of information you encounter in earlier stages of your research (e.g., participants you have interviewed early on may have hinted at or even suggested that there are one or two individuals who do not seem to fit with the rest of the department and you seek to connect with them in order to achieve a more well-rounded understanding). In another example, you may realise that your early data are giving rise to a theoretical account indicating that teachers with specialist postgraduate training background (i.e., with PhDs; they might even be referred to around the school as ‘the nerds’) tend to be socially disconnected, not held in high regard around a particular high school and generally dissatisfied with their lot in work life at the school. You then seek to identify and connect with a sample of such people in order to test these relational propositions from your emerging theory and discover whether there is evidence regarding the social disconnection of these people can be found, whether data can be found showing what ‘not being held in high regard’ actually implies and whether there is evidence for general dissatisfaction. This latter example illustrates that a specific differentiating characteristic (having a PhD) that emerges as being of theoretical interest which might then be used to

884

19

How Do I Manage the Sampling Process?

anticipate where negative cases are likely to be found. However, in other instances, it may be quite difficult to identify relevant negative cases, especially from marginalised or silenced minority groups, and even trickier to encourage their participation in your research. In such instances, you may have to probe around a bit, perhaps even using a snowball sampling strategy and asking participants if they know anyone holding views that strongly contrast with their own views or with mainstream views or accessing documents that may point to the existence of and perhaps even the location of such views. • With negative case sampling, you may run the risk of encountering perspectives that have other agendas than helping you learn and understand their point of view. Some marginalised participants may try to co-opt you into supporting their perspective, publicising it and, perhaps, even promoting it. This transforms their participant role into a political stakeholder role and is more likely to occur in research contexts where there are evident on-going hot-button power-oriented issues and conflicts (e.g., strong conflict with a union over work place conditions or ethical conflict with an aspect of the business). At its most extreme, the psychological pressures you experience to buy into a marginalised perspective and its accompanying political agenda may be so overwhelming that you lose your researcher attitude and ‘go native’. At that point, you cease to successfully manage your preconceptions, expectations and beliefs and your buy-in becomes complete. • Another potential problem with negative case sampling is a greater likelihood of not accessing authentic perspectives from a marginalised or disenfranchised data source, particularly if the majority view is that that data source holds a deviant, criminal or unethical perspective or engages in behaviours that could be characterised as such. You will need to work very hard to ensure that the marginalised or disenfranchised data source, if it is a person, feels safe in providing their views; otherwise lack of authenticity will be a very real concern. Encouraging participation by such people may require compromises on your part such as connecting with them outside of the research context, promising anonymity and promising not to record or directly quote anything they say.

19.3.9 Sequential/Saturation Sampling The sequential/saturation sampling strategy is the logical iterative extension of theoretical sampling taken to a concrete endpoint. It is generally the most common manifestation of fully-realised theoretical sampling in the context of grounded theory research under the guidance of an interpretivist/constructivist pattern of assumptions. Sequential/saturation sampling ideally embodies the continuous iterative interplay between theoretical sampling and data analysis where interpretations and theoretical explanations, concepts and relationships are progressively enhanced, extended, revised and improved, stimulating the gathering of additional data, and so

19.3

Non-Probabilistic Sampling Strategies

885

on. The iterative sampling/analysis process affords opportunities to achieve both breadth and depth of learning in context. Eventually, a point will be reached in the iterative process where you learn nothing new that requires you to make further changes to your emergent grounded theory. This is the point of saturation and serves as a signal that you can cease gathering data (a diminishing returns-type of logic that has implications for your resourcing requirements). Saturation is more likely to be achieved using participant observation data gathering strategies where you maintain an in-depth and long-term immersion in the research context (as can happen in ethnographic research, for example). Long-term contextual immersion enhances the chances of achieving a high level of sufficiency and completeness of interpretations and theoretical explanation. In cases where long-term immersion is not possible, you may be able to carry out repeated short-term contextual immersions, reflecting a longitudinal sequential or case-based sequential MU configuration. Figure 19.19 provides a visualisation of the sequential/saturation sampling process where you purposively connect with two data sources and access an event and a situation and commence analyses of the resulting data. Your preliminary learning that emerges then points to the need to pursue confirming as well as disconfirming evidence for your emerging theoretical story, leading you to purposively sample two more data sources, another event and another situation. Learning from this stage moves you to gather further data from another data source and another event and so on in iterative fashion, until nothing new is gleaned from new data that would require you to further modify your theoretical account (i.e., saturation is achieved). Advantages of the sequential/saturation sampling strategy: • Both breadth and depth of learning can be achieved using the sequential/ saturation sampling strategy. The longer you can maintain contextual immersion, the greater likelihood you will encounter or experience unusual or rare contextual events, situations and perspectives. Breadth of learning can be especially enhanced if you couple the sequential/saturation sampling strategy with the negative case sampling strategy. • If you can achieve saturation, then you can argue, on strong grounds, that your emergent theoretical account is sufficient. If emergent theoretical concepts and relationships drive your sampling process to seek confirming and disconfirming evidence, the sufficiency argument can be strong. If analyses always remain focused on the data gathered and the meanings/stories they convey, arguments for authenticity can also be made. However, if you do not effectively manage your own preconceptions and expectations throughout the sequential/saturation sampling process, sufficiency and authenticity arguments will no longer be convincing. The bottom line is that you should make all sequential/saturation sampling choices on the basis of what will add value to development of your emergent theoretical account (i.e., ‘what are the data and the emergent theory telling me I need to do in terms of seeking more data?’ as opposed to ‘what

886

19

How Do I Manage the Sampling Process?

Research Context Researcher

Event 1

Data Source 1 Situation 1

Data Source 2

Researcher forms preliminary interpretations & theoretical ideas/explanations from the data – uses these to guide the next stage Researcher Seeking data to testt initia initial interpretations ini ns & theoretical explanations

Situation 2

Data Source 3 Data Source 4

Research Context

Event 2

Researcher forms further interpretations & theoretical ideas/explanations from the data – uses these to guide the next stage Researcher Seeking data to further her test interpretations ions & theoretical explanations Data Source 5

Event 3

Research Context

Researcher ceases sampling when nothing new is learned that would cause additional elaborations or changes to interpretations & theoretical explanations = Saturation Researcher Fig. 19.19 Visualisation of the sequential/saturation sampling strategy

additional data do I think I need?”). Your research journal will help you to keep track of the bases for your sampling choices. Disadvantages of the sequential/saturation sampling strategy: • The sequential/saturation sampling strategy is very resource intensive to implement. The longer the contextual immersion, the more resources you will require. This implies that saturation may not be achievable because of resource constraints in that you may simply not be able to stay immersed in the research context long enough to achieve saturation. Employing an alternative MU

19.3

Non-Probabilistic Sampling Strategies

887

configuration, where several distinct short time periods in the research context can be scheduled for data gathering purposes, may help you to use available resources more effectively. The drawback here is that, for each return to the research context, there will be a settling-in period for you in order to ‘get your head back in the game’. Furthermore, you will need to be alive to the possibilities that changes may have occurred in the time periods between visits and this may have implications for available data sources and access to specific events or situations. This may mean that you will need to sample at least some data sources who could shed light on those inter-visit periods (this would not constitute sequential/saturation or theoretical sampling; it would be a form of contextual sampling). • It may not always be clear when or even if the point of saturation will or can be reached. This uncertainty may emerge because the research context is simply too complex and multi-faceted (offering many potential data sources, events, situations to consider). You will often need to use your judgment to make a trade-off, balancing the benefits of the learning achieved against the costs of gathering new data. The principle of diminishing returns for benefits and costs can be employed to establish a quasi-saturation point where new learning is considered minimal or inconsequential rather than zero (i.e., full saturation). This uncertainty creates resource implications as well since you cannot continue gathering data indefinitely simply because those data keep yielding new insights. Where you face time constraints (e.g., as with postgraduate research or a research contract), you may not reach the point of saturation before you reach the point where data gathering must stop and writing up must be completed. Resource and contextual complexity constraints imply that absolute stability in your emergent theoretical account will likely not be achieved. This outcome will not be ‘fatal’ to your research project. Instead, what it means is that you will need to be more cautious and circumspect in drawing conclusions and be very clear about the limitations that not achieving saturation imposes on your research. • Using the sequential/saturation strategy generates research intensity and effort because you are always ‘learning more and having to get more’. The longer the iterative sampling/analysis cycle continues, the more emotionally, socially and cognitively taxing the research can become for you and the greater the risk that you will lose your researcher attitude (which may be revealed in ‘going native’, increased sloppiness in sampling/data analysis management or in sheer exhaustion and loss of motivation). This means that you need to ensure that adequate personal (e.g., for managing work-life balance, emotional, social and physical health) and professional (e.g., collegial advice, sounding boards for issues) support systems are in place from the start to help you to mitigate the risks that such detrimental outcomes will emerge.

888

19.3.10

19

How Do I Manage the Sampling Process?

Synergistic Combining of Sampling Strategies

Throughout our discussion of the various sampling, we have highlighted the potential benefits to be achieved by using synergistic combinations of more than one strategy, either simultaneously (as when purposive sampling is used with a quota sampling strategy) or sequentially (as when convenience sampling is used to choose outsider contextual data sources and researcher-suitable time periods for data gathering, then purposive or theoretical sampling is used within the research context). Onwuegbuzie and Collins (2007) and Teddlie and Yu (2007) discuss a range of potential synergistic combinations of sampling strategies, focusing on sampling to yield quantitative and qualitative data (i.e., sampling for traditional mixed methods research). These strategies are usually implemented as multi-stage sampling strategies, which means they can be very useful when different patterns of guiding assumptions are adopted for different stages of the research (e.g., in sequential, hybrid and evolutionary MU configurations). Note though that synergistic use of sampling strategies need not be tied to obtaining mixtures of different types of data (quantitative and qualitative); they may be equally valuable for obtaining data of a single type (quantitative or qualitative). The nature of the synergies to be realised by combining different sampling strategies depends upon the specific strategies combined, their advantages and disadvantages, available resources and the overall relationship between sampling strategies and your research goals. Every synergistic combination will offer benefits and impose costs, which means when planning for their use, you must consider both costs and benefits. For example, theoretical sampling is most effective if combined with negative case sampling as this helps you to achieve a more well-rounded and sufficient story, but this synergy becomes more resource-intensive if saturation is being pursued. Quota sampling paired with either convenience sampling or purposive sampling can be a very resource-efficient way to build up a sample, but at the cost of loss of sufficiency. Stratified random sampling can be effectively paired with simple random sampling, but where small-sized strata exist, representativeness of your sample may suffer. Some researchers have argued that you can counteract this problem to some extent by over-sampling so that you can wear the losses and still end up with a reasonable sample. Even here, though, you must check that there is no systematic bias in who is declining or who is agreeing to participate. Whatever synergistic combination is used, you will need to ensure that you convey a transparent story about the logic, benefits and limitations of the specific sampling processes you employed.

19.4

19.4

Constraints on Sampling Quality

889

Constraints on Sampling Quality

No matter what your adopted pattern of guiding assumptions is, two critical constraints can impact upon any sampling strategy, no matter how meticulously planned and implemented. As it turns out, both constraints are tied to the involvement of human beings in your research and apply irrespective of the research frame being employed. Constraints invariably mean that you will have to make one or more trade-offs. The trick is to make such choices so as to most effectively deal with or circumvent the constraints while not sacrificing too much in convincingness. Firstly, ethical constraints frequently rewrite the most ideal of sampling plans (effectively turning them into a type of volunteer sample), privileging the rights of human participants and imposing obligations on you, as the researcher. For example, even the most sophisticated random sampling strategy can be rendered non-random or non-representative (and therefore less than ideal for statistical and generalisation purposes) simply because potential participants must have the unimpeded right to decide whether they will participate (through giving or withholding their informed consent) and they must have the unimpeded right to withdraw or cease their participation at any time. Response rates can therefore be impacted by a high number of declines and/or withdrawals and this, in turn, can disrupt representativeness and restrict opportunities to generalise. Ethically, you cannot force people to participate, even when they have been properly sampled per your sampling strategy. You may control the choice and implementation of your sampling strategy, but potential participants control whether they will be, or remain part of, your sample. You must take steps to ensure that you do nothing during the course of approaching and gathering data from participants that might cause them to decline to participate or withdraw their participation. If this occurs, representativeness or sufficiency of the sample will be negatively impacted as an artefact of the way your research is being conducted. In short, participants will certainly use their own judgment to determine whether they participate and persist in your research project, but you should not be inadvertently providing them with reasons to decline or withdraw. The existence of gatekeepers who control access to desired participants compounds the problem imposed by ethical constraints. In some cases, political or cultural issues (e.g., a gatekeeper expecting quid pro quo for allowing access to their research site for sampling and data gathering purposes; a requirement to discuss accessing an Indigenous community with one or more tribal elders) may accompany the ethical constraints. Of course, no gatekeeper can ethically require that any individual will or should participate, that remains the individual’s absolute right to decide (i.e., participation cannot be coerced by an employer, but it may be strongly encouraged, especially if the research is being done to help meet organisational goals). However, a gatekeeper can shut down or open up the opportunity for you to invite those individuals to participate and if that opportunity is shut down, you will have no recourse other than to rethink your sampling strategy (this

890

19

How Do I Manage the Sampling Process?

is one reason why having a Plan B for sampling purposes is so important and will be discussed in more detail below). Secondly, feasibility constraints will strongly influence your choice of sampling strategy(ies) and these constraints are determined by available resources (time, funding and personnel) and geographical location as well as by access to the relevant population(s) and/or potential sample members. Resources, location and access will often play off each other such that constrained resources will likely restrict your access to desirable participants or organisations as well as force you to focus on a less dispersed geographical area. Better resources (funding and data collection support, for example) may permit the use of more sophisticated sampling strategies, but with the perhaps undesirable consequence of extending the timeline or increasing the complexity of your research. Sample size needs to be considered in light of data gathering strategies you anticipate using. For example, in-depth semi-structured qualitative interviews (conducted under an interpretivist/ constructivist pattern of guiding assumptions) take time to carry out and require a post-interview time period for recording interviewer thoughts and observations and you can only do so many in a day (they are tiring interviews to conduct). Targetting a large sample, by doing a number of interviews in a number of organisations, will take a great deal of time to carry out, unless a team of interviewers is available, which requires additional resources. Lack of access to a list of population members to form a sampling frame for a simple random sample may lead you to rely on a convenience sample or a volunteer sample imposing restrictions on generalisability. In general, you want to construct a sampling plan that is feasible for you to implement with a reasonable chance of success given your resources and location, and that anticipates that some potential participants or organisations will either decline to participate or decline to go the distance in their participation. If you don’t have the resources to conduct your research across a wide geographical area, you may be forced to focus, for example, on your local area (convenience sample) or use a cluster sample, and you will simply have to wear the attendant restrictions on your ability to generalise.

19.4.1 Plan B Sampling Strategy A Plan B sampling strategy is a back-up plan; one that is implemented if your initial sampling strategy(ies) fall apart. A Plan B may be necessary because you cannot fully anticipate whether some organisations or participants will decline to participate or withdraw their participation, or some gatekeeper will deny you access to the data sources you need. A Plan B may also be necessary if the resources you need to carry out your intended sampling strategy(ies) end up not being available or satisfactory. A Plan B for sampling is usually accompanied by sacrifices in sample quality or representativeness to enhance feasibility. A Plan B might be something as simple as a substitution map: having another organisation ready to pursue access with, or a new source or site for seeking participants. In a substitution map,

19.4

Constraints on Sampling Quality

891

replacement data sources are held in reserve in case primary data sources become unviable. The sampling strategy may remain the same, but the sampling frame is generally modified. It can also be useful to have a Plan B available for situations where specific data gathering strategies will be used in which response rates are known to be problematic, such as with mail or internet administered questionnaires in a Survey research frame. In these situations, you need to prepare avenues for obtaining additional participants. This might mean implementing an alternative data gathering strategy to help shore up the response rate problems with the initial strategy, what could be termed an MU augmentation Plan B. For example, you may initially plan to use a mail questionnaire but achieve only a 7% response rate. Plan B might be to sample (randomly or otherwise) potential participants who were sent a mail questionnaire but did not complete it, contact them and perhaps conduct a phone interview. If an ethics review board has rejected approval of your research project because you proposed to employ a snowball sampling strategy, Plan B might be to employ a different sampling strategy altogether, such as a purposive or convenience sample (which could necessitate altering your research questions and perhaps even your pattern of guiding assumptions). This sampling strategy adaptation Plan B will generally be less desirable than your original sampling strategy but would be better able to meet the ethical constraints. If your research is being conducted within the Case Study research frame, one consequence might be that you have to change focal organisations (the case), from one you are less familiar with (hence the need for snowball sampling) to one you are more familiar with (where a purposive sampling strategy would work but the organisation is less optimal with respect to your research goals/questions). The time and place to build Plan B is at the commencement of your research project. It requires thinking ahead to try to anticipate potential obstacles to the preferred sampling strategy(ies) and formulating Plan B to circumvent those obstacles. Such planning must be done in full knowledge of what you gain and lose by having to rely on your Plan B and what the implications of that cost/benefit trade-off might be for the convincingness of your research. In some cases, implementing Plan B may require the implementation of other processes as well (such as new data gathering strategies) to help offset some of the sacrifices it requires, and these will have associated resource implications.

19.5

Key Recommendations

This chapter has considered a wide range of issues and strategies associated with sampling in postgraduate and other research projects. However, we can distil a few essential ‘gems’ of learning to keep in mind from this exploration of sampling. • Sampling is not just about choosing which individuals you will approach for your study. It is broader and encompasses additional issues such as where, when and how you will engage with participants and can even influence how you will

892

•

•

•

•

•

•

19

How Do I Manage the Sampling Process?

choose examples in your write up. It may also implicate how you construct the conditions or contexts that participants will experience (the principle of representative design; especially relevant in the context of Manipulative, Structuring or Immersive experience-focused strategies). Which sampling approach or combination of approaches you use will and should be significantly influenced by your choice or research frame, pattern of guiding assumptions, your contextualisation and positioning strategies, choice of research context(s) and your research questions/hypotheses. Probability sampling strategies commonly used under the positivist pattern of guiding assumptions include: simple random sampling, stratified random sampling, cluster sampling, multi-stage sampling and systematic sampling. Non-probability sampling strategies that can be used under either the positivist or an interpretivist/constructivist pattern of guiding assumptions include: convenience sampling, quota sampling, volunteer sampling, contextual sampling, purposive sampling, snowball sampling, theoretical sampling, negative case sampling and sequential/saturation sampling. Keep in mind that, as part of your ethical obligations as a researcher, participants have the option of declining to be involved in your research as well as having the right to cease their participation in your research, at any time. This will have important implications and impose important limitations on your sampling schemes. For probability sampling schemes, participants’ exercise of these rights will impact on the randomness and representativeness of your resulting sample. For non-probability sampling schemes, participants’ exercise of these rights will impact will impact on the representativeness and/or sufficiency of the interpretations you can generate. There are a variety of options available to you when constructing a sampling plan. Your choices amongst these options should be guided by the issue of feasibility as well as by what you will require to produce a convincing research project. This means you will have to construct a sampling plan that meets, as far as possible, your needs in terms of research quality while at the same time being a plan that can be successfully implemented within your time, access and resource constraints. Under an interpretivist/constructivist or other non-positivist pattern of guiding assumptions, representativeness carries a different meaning, focusing on the type and range of participant voices you gain access to/connect with, and, as a consequence, sample size is much less of an issue. Sufficiency becomes the driving quality criterion for constricting a sampling plan under such assumptions and this is where non-probability schemes are strongest. Where appropriate, transportability across research contexts may emerge as a secondary quality criterion to focus on. Under the positivist pattern of guiding assumptions, representativeness, focusing on the character of the sample composition as a foundation for generalisations, becomes the driving criterion for constructing your sampling plan. Additionally, the reliance on statistical analysis to derive meaning from any quantitative measurements means that your sampling plan should, as far as possible, support

19.5

Key Recommendations

893

the drawing of generalising inferences from your sample to your intended population. Probability sampling procedures become an imperative in this regard. Where this is not possible (e.g. web-based questionnaires, relying on volunteers), constraints on sample representativeness will likely emerge and this will affect your ability to generalise. • Under the positivist pattern of guiding assumptions, where you are using statistical inference tests, there is a substantive interplay between the probability of falsely concluding that you have found a significant effect (Type I or alpha (a) error), the probability of correctly identifying that a significant effect truly exists (power of the test), size of the effect you are looking for and the size of sample you will need. • Keep your options open because not everything goes according to plan. Consider having an alternative sampling plan (Plan B) ready should this need to be implemented. Discipline yourself to think of contingencies and anticipate potential problems with accessing participants and/or other data sources that you might encounter down the road.

References Bartlett, J. E., Kotrlik, J. W., & Higgins, C. C. (2001). Organizational research: Determining appropriate sample size in survey research. Information Technology, Learning and Performance, 19(1), 43–50. Brunswik, E. (1952). The conceptual framework of psychology. Chicago: University of Chicago Press. Brunswik, E. (1956). Perception and the representative design of psychological experiments (2nd ed.). Berkeley, CA: University of California Press. Bryant, A., & Charmaz, K. (Eds.). (2007). The Sage handbook of grounded theory. Los Angeles: Sage Publications. Bryman, A., & Bell, E. (2015). Business research methods (4th ed.). Oxford, UK: Oxford University Press. Charmaz, K. (2014). Constructing grounded theory (2nd ed.). London: Sage Publications. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). New York: Routledge. Cooksey, R. W. (1996). Judgment analysis: Theory, methods, and applications. San Diego: Academic Press. Cooksey, R. W. (2014). Illustrating statistical procedures: Finding meaning in quantitative data (2nd ed.). Prahran, VIC: Tilde University Press. Corbin, J., & Strauss, A. (2008). Basics of qualitative research (3rd ed.). Los Angeles: Sage Publications. Csikszentmihalyi, M., & Hunter, J. (2003). Happiness in everyday life: The uses of experience sampling. Journal of Happiness Studies, 4, 185–199. Cunningham, J. B., & McCrum-Gardner, E. (2007). Power, effect and sample size using GPower: Practical issues for researchers and members of research ethics committees. Evidence-Based Midwifery, 5(4), 132–136.

894

19

How Do I Manage the Sampling Process?

Drury, C. G. (1983). Task analysis methods in industry. Applied Ergonomics, 14(1), 19–28. Farquharson, K. (2005). A different kind of snowball: Identifying key policymakers. International Journal of Social Research Methodology, 8(1), 345–353. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. Gill, J., & Johnson, P. (2010). Research methods for managers (4th ed.). Los Angeles: Sage Publications. Green, S. B. (1991). How many subjects does it take to do a regression analysis? Multivariate Behavioral Research, 26(3), 449–510. Hair, J. F., Black, B., Babin, B., & Anderson, R. E. (2010). Multivariate data analysis: A global perspective (7th ed.). Upper Saddle River, NJ: Prentice Hall. Hammond, K. R., McClelland, G., & Mumpower, J. (1980). Human judgment and decision making: Theories, methods, and procedures. New York: Praeger. Hammond, K. R., & Wascoe, N. E. (Eds.). (1980). Realizations of Brunswik’s representative design. San Francisco: Jossey-Bass. Lynn, P. (2002a). Principles of sampling. In A. Greenfield (Ed.), Research methods for postgraduates (2nd ed., pp. 185–194). London: Arnold. Lynn, P. (2002b). Sampling in human studies. In A. Greenfield (Ed.), Research methods for postgraduates (2nd ed., pp. 195–201). London: Arnold. Morse, J. M., Stern, P. N., Corbin, J., Bowers, B., Charmaz, K., & Clarke, A. E. (2009). Developing grounded theory: The second generation. Walnut Creek, CA: Left Coast Press. Neuman, W. L. (2013). Social research methods: Qualitative and quantitative approaches (7th ed.). Boston: Pearson Education. Onwuegbuzie, A. J., & Collins, K. M. (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12(2), 281–316. Saunders, M., Lewis, P., & Thornhill, A. (2012). Research methods for business students (6th ed.). Harlow, UK: Pearson. Scheaffer, R. L., Mendenhall, W., III, Ott, R. L., & Gerow, K. G. (2012). Elementary survey sampling (7th ed.). Boston: Brooks Cole/Cengage Learning. Schraagen, J. M., Chipman, S. F., & Shalin, V. L. (Eds.). (2000). Cognitive task analysis. New York: Psychology Press. Smithson, M. (2000). Statistics with confidence. London: Sage Publications. Teddlie, C., & Yu, F. (2007). Mixed methods sampling: A typology with examples. Journal of Mixed Methods Research, 1(1), 77–100. van Voorhis, C. R., & Morgan, B. L. (2007). Understanding power and rules of thumb for determining samples. Tutorials in Quantitative Methods for Psychology, 3(2), 43–50. Wei, J., & Salvendy, G. (2004). The cognitive task analysis methods for job and task design: Review and reappraisal. Behaviour & Information Technology, 23(4), 273–299.

Chapter 20

How Should I Organise My Data?

Data collection and analysis should not proceed until you have carefully planned for how you will organise and manage the collection, recording and storage of your data and how you will prepare your data for analysis. One aspect of this process will involve planning for recording your data in as close to their original form as possible. For qualitative data, this will mean deciding on data gathering strategies and media for data recording, which may include Textual and Multi-media artefacts as well as the Transformative data-shaping strategy. For quantitative data, this will generally mean deciding between data gathering strategies that focus on any of the data-shaping strategies and/or the archival/secondary artefact-based strategy. If you are accessing one or more secondary quantitative databases, you will need to devise rules/processes for how to sample, extract and store just those cases, variables and measurements you need into your own database for analysis (the eViews program can facilitate such processes for financial and econometric research projects). Once the data have been obtained or recorded, you will need to decide if and how the data will be transformed from their original recorded form into a form that you can use for data analysis purposes (the hallmark of the Transformative data-shaping strategy). In many cases, it is unlikely that you will analyse the data in their original recorded form. Another aspect of this planning process will involve you making decisions as to whether you will employ a computer support system of some type to help you manage your data storage and analysis. Every support system, even a manual one, will have its own requirements that will constrain how you manage your data and you will need a system of procedures to help you manage your data within those constraints. If you do decide to use a computer support system, you will need to know which system(s) you will have access to (e.g., SPSS, SYSTAT, NCSS, Stata, eViews, SAS or Excel for quantitative data and/or MAXQDA, NVivo, dedoose, Decision Explorer, ATLAS.ti, or The Ethnograph for qualitative data) and what their associated requirements will be for data entry, formatting and storage. If you don’t know how to use your chosen program(s), you will have to learn how to use it before you can commence data entry and analysis. It is not ethical, in the context of © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_20

895

896

20

How Should I Organise My Data?

postgraduate research, to hand your data to someone else to analyse—you must analyse your data yourself. However, it is entirely appropriate to have someone else help you to transform your data from their original recorded form into the form you will need for analysis, as long as you have properly briefed them as to precisely how you want them to do that translation. You may not be able to anticipate all the decisions you need to make about organising your data, but the more you can anticipate and plan for, the smoother the transition will be from data recording to data analysis. At the same time, the risks of introducing errors into the translation process will be minimised. In order to handle some types of emergent data problems (e.g., anomalous or ambiguous responses, partial responses, disruptions to or degradations in recording), you may have to make on-the-spot decisions as you encounter them. For such problems, your research journal will prove invaluable in helping you to record and later consistently apply the handling methods you improvise. No matter what type of data you collect, you can be certain that you will have to make choices that deliberately sacrifice some information in the data in order to retain or enhance other relevant or desirable information.

20.1

Preparation of Quantitative Data

20.1.1 Coding Rules For quantitative data, preparation means transforming your raw data (e.g., ratings made on a questionnaire, ticks in boxes, responses to open ended questions) into a form that can be readily analysed and entering the resulting numerical data into a spreadsheet for analysis. Many statistical programs such as SPSS, NCSS, Stata or SYSTAT come with a built-in spreadsheet data entry system and provide extensive variable defining and labelling capabilities. Prior to data entry, you will need to make decisions about how you will deal with the process of transforming your raw data, in whatever form they come to you, into the numerical form that will be used for analyses. You should always record these decisions and the logic behind them in your research journal. You will likely need to devise rules for the following issues that could emerge during your data entry activities. • How to handle non-serious or pattern responders? In self-report paper-based or web-based questionnaires, there is always the risk that some participants will either not take your instrument seriously or be unwilling to reveal their true feeling or position on items. For example, they may respond according to some systematic pattern down a page or screen full of Likert-type attitude items (e.g., zig zag or progressive left-to-right or right-to-left responses). Some participants may only respond only using the centre ‘neutral’ category if your scale offers an odd number of response options (‘middle-of-the-road’ response pattern), they

20.1

Preparation of Quantitative Data

897

may respond only at the extremes of a scale (‘bipolar’ or ‘extreme’ response bias) or they may strongly agree with every item (‘acquiescence’ bias). In such cases, your decision will be to either exclude such individuals from your sample (a decision you would have to defend) or include them in the sample, but only attend to the responses that do not reflect their pattern biases (such as responses to demographic items). The first decision will reduce your overall sample of usable responses; the second decision is a judgement call and may lead to problems with missing data for some analyses. In general, the second choice may be more defensible since you will be able to look for any potential demographic patterns associated with the non–serious or biased responders (see also missing value analysis which is discussed below). • How to deal with ambiguous responses? One type of ambiguous response, especially for paper-based questionnaires, is where a participant circles two numbers or ticks two or more categories on a scale when only one was to be circled or ticked. Another type of ambiguous response occurs when participant makes up their own response scale (e.g., the participant places an X between the two middle numbers on an even-numbered bipolar (e.g., agree-disagree) scale in order to signal a neutral response where none was being offered by the scale itself). Ambiguous responses contribute to measurement error. This means the most defensible rule for dealing with them is to treat them as missing data for that specific item. There are other rules you could implement, such as averaging two circled numbers or flipping a coin to choose which category to record if several were ticked, but such rules involve artificial manipulations of your intended response scale, making them harder to defend. Another choice you could make is to use a web-based questionnaire, where the question interface can be designed in such a way as to preclude ambiguous responses. • How to handle participants who omit or neglect items or whole sections of a questionnaire? This is a classic cause of missing data in a sample and is a risk mainly for paper-based instruments. For example, participants may skip an item they don’t want to respond to, they might write an illegible answer to an open-ended question, they may inadvertently jump over an item (a risk with small font questionnaires having many items close together on a page) or they may forget to turn a page over to answer questions on the reverse side. Your choice of rules is basically: (1) use what data they do provide, coding their unanswered questions as missing data or (2) delete the participant from your sample altogether. Rule 1 will allow you to retain as much data as possible but assumes that the missing responses are essentially random events rather than deliberate omissions. Rule 2 involves a judgement call regarding how much of the questionnaire has actually been omitted or neglected. If the proportion is substantial (e.g., whole pages or sections or a whole series of items in a group), then you may decide to remove the participant from your sample (neglecting whole sections then becomes a basis for defining a non-usable questionnaire). In some cases, you may need both rules 1 and 2, working in tandem, to deal with a range of missing data anomalies. Again, a choice you could make is to use a web-based questionnaire, where the question interface can be designed in such a

898

20

How Should I Organise My Data?

way as to preclude skipping of questionnaire items or sections and production of illegible responses to open-ended questions. • Following on from the previous two questions, will it be important for you to be able to distinguish truly missing data (where a participant does not provide a response to a question) from inappropriate responses (where a participant circles two rating numbers or ticks more than one category where only one was to be circled or ticked)? If you do want to distinguish them, you could decide to leave truly missing responses blank in your database but record a unique number outside the legitimate scale range to signal inappropriate responses (see de Vaus, 2002, pp. 3–4). For example, you could record a 9 for an ambiguous response to a 7-point Likert-type scale. The 9 can then be labelled as a second type of missing value for analyses targeting your research questions/hypotheses (this can be done quite easily in SPSS, for example). But because you have distinguished blank from 9 ‘responses’, you have given yourself the capacity to analyse the people who scored 9’s separately from those whose responses were truly missing using items where they did provide legitimate responses (e.g., demographic items or other attitude items). If your scale has a built-in response category of ‘Not Applicable’ or ‘Unable to Rate’, then you could use something like a ‘9’ or ‘-9’ outside the main response scale to indicate such a response (see de Vaus, 2002, pp. 72–74). This would then facilitate later analysis of participants who were unable or unwilling to give a response within the scale for specific items. • How to code open-ended responses? Open-ended responses include written responses to “Please specify” after a participant has ticked the “Other” category for an item (e.g., a demographic item, asking about religious background, offering the choices of ‘Christian’, ‘Islamic’, ‘Jewish’, ‘None’, as well as an ‘Other—Please specify ____________’’ category) as well as written responses to open ended questions (e.g., ‘What does effective leadership mean to you?”). de Vaus (2002, pp. 4–8) provides an excellent discussion of approaches to coding open-ended questionnaire responses. In general, for quantitative analyses, handling open-ended responses is a type of content analysis, where responses are categorised according to thematic content. Then the category codes are assigned numbers (usually only at the nominal or ordinal scale of measurement) and these numbers become the data to be entered. The category codes you develop may emerge from the data themselves or you may wish to adopt a coding system used by other researchers—this would be a decision you would make early on. You will have to make some decisions as you go, regarding the fineness or coarseness of the categories you want to retain (see de Vaus, 2002, pp. 33–39). Finer coding systems will generate more distinct categories which can make analyses more complicated and problematic, especially for small samples. Whatever coding system you devise or employ, always make sure that the categories are mutually exclusive (i.e., non-overlapping so that responses can be unambiguously classified into only one category) and exhaustive (so that every response can be classified into a category). Note that this type of positivist content analysis coding is distinctly different from the

20.1

Preparation of Quantitative Data

899

types of coding you would employ for qualitative data gathered under the guidance of an interpretivist/constructivist or other non-positivist pattern of guiding assumptions. One process you would be expected to undertake during the coding of open-ended responses in an investigation guided by the positivist pattern of assumptions is checking inter-rater or inter-coder reliability (recall the discussion of this process for assessing reliability in Chap. 18). To check inter-rater reliability, two or more people work through the same raw data forms and independently code the content of open-ended responses into the categories that have been devised. The coding patterns for pairs of coders are then compared, usually with a measure of correlation or agreement (see Cooksey, 2014, pp. 379–383) to assess the extent to which they coded the same content into the same categories. Disagreements are resolved through discussion and consensus and coding comparisons and refinements in definition continue until a high degree of reliability can be demonstrated. • How to numerically code the categories of categorical variables? For nominal or ordinal scale items where only one category response is permitted, you will need to decide how to numerically code category membership for each participant. Some researchers build the numbering scheme into their instruments (yielding a pre-coded questionnaire), but this can often be distracting for participants. Normally, the numbering scheme is applied at the point of data entry —you just need to settle on the rules before that step occurs. For example, for a demographic nominal scale item measuring ‘sex of participant’, you might decide to code females as ‘1’ and males as ‘2’. [For such dichotomous (two category) items, there may be analytical advantages to using ‘0’ and ‘1’ to code the two categories, rather than ‘1’ and ‘2’—a process called dummy coding. A dummy-coded variable can be included in calculations of Pearson correlations, giving what is technically called a point-biserial correlation if correlated with an interval/ratio scale measure or a phi coefficient if correlated with another dummy-coded variable (see Cooksey, 2014, pp. 121–129). A dummy coded variable can also be included in a regression equation as a predictor, which will yield a regression coefficient that measures the difference between the dependent variable mean for participants in the ‘1’ category and those in the ‘0’ category (see the discussion in Cooksey, 2014, pp. 211–212). If you compute an average for a dummy coded variable, the average reflects the proportion of your sample that is coded in the ‘1’ category.] If your nominal or ordinal scale has more than two categories (as would a questionnaire item asking about religious background or highest level of education achieved), then simply using the cardinal numbers ‘1’ to ‘k’, where k is the number of category options on offer, will suffice. The one thing to remember here is that your numbers will not represent an interval measurement scale, so they should not be used in any parametric analysis that would treat them as such (e.g., computing means and standard deviations, correlations, regressions). However, it is possible, once the data have been entered, to dummy code a

900

20

How Should I Organise My Data?

multi-category variable—a process described in detail by de Vaus (2002, pp. 368–373) as well as by Cooksey (2014, pp. 211–212). Once this has been done, then it becomes permissible to use the resulting dummy-coded variables in correlation and regression-type analyses (see, for example, the discussion in Chap. 3 of Miles & Shevlin, 2001). • How to code multiple answers to a question where multiple answers are permissible? Some items are intended to have multiple responses and you will need to be clear on how you want to code each response. For a categorical item where several categories can be ticked by a participant (e.g., an item that asks participants to tick all of the categories of education and training activities they have participated in for their current employer), you would create a separate variable in your spreadsheet database for each possible category. Then you would enter a ‘1’ if the participant ticked that category or a ‘0’ if they did not tick it. Essentially, you would create a set of dichotomous items. For an ordinal scale item where a rank number must be allocated by the participant for each option offered (e.g., “From the following list, rank in order of importance from 1 to 5 the five most important reasons you think people leave your current employer”). For such an item, you would first assign each item in the list a number. Then you would need five variables to code the 5 ranking positions (‘First’, ‘Second’, etc.). Finally, you would enter the appropriate item number in the variable corresponding to the ranked position of each item. de Vaus (2002, pp. 10–16) provides a very good discussion of a range of coding options for multiple response items.

20.1.2 Data Entry All the coding rules you devise must be consistently applied to all participants when entering their data into the computer spreadsheet. It is ethically permissible to have someone else (e.g., a data entry professional) enter your data for you; however, you will need to thoroughly train them on your coding rules so that the data are entered the way you want them. If you enter the data yourself, make sure you ‘Save File’ often so that you do not lose any of the data entry work that you do. Statistical software can be twitchy and occasionally ‘hang your computer’. If this happens, anything you have done since the last ‘Save File’ will be lost—saving very regularly will ensure any losses are minimal. Also, as you enter the data, periodically double check that you are in the right row and column of the spreadsheet when entering a specific data value—it is easy to get out of line or in the wrong column when entering data, so ‘looking up at the screen’ periodically will ensure you are not entering ‘apples in the oranges column’. If you are employing a web-based internet questionnaire or instrument, you may be able to program some of your coding rules into the web page so that the downloaded data file will have the structure and content that you want. If you are employing a mixture of paper-based and web-based questionnaires and/or

20.1

Preparation of Quantitative Data

901

instruments, you will need to take steps to ensure the proper alignment of variables from the two data sources and to ensure coding consistency between them. This need often arises because designing paper-based and web-based questionnaires require different design decisions, and this may affect how various measurements or responses are obtained or even the order in which they are obtained. Variable alignment between the two data sources ensures that when you have a single database containing all participants, every variable contains the correct scores, correctly coded. Your final spreadsheet database will have a common layout, irrespective of the software package you employ for data entry. There will generally be one row in the data base for each participant or case and at least as many columns (usually called ‘variables’) in the database as there are items (or items plus multi-category measures) on your measuring instruments. The first column should contain a unique identifying number for each participant so that you can correctly link the data in the row back to the correct participant’s original recorded raw data record, which means, of course, that you will need to write or have written this unique identifying number on the participant’s original recorded raw data record as well. If you have 1200 participants responding to a questionnaire with 50 questions, you will thus have a 1200 by 51 (including the extra column for the participant ID) matrix of data. It is possible to create your database in Excel then import the Excel file into whatever statistical package you will be using. However, if you do this, you will not be able to take immediate advantage of the extended variable labelling facilities offered by the statistical package. Instead, we recommend using the spreadsheet interface of the statistical package (e.g., SPSS, Stata or SYSTAT) itself. Make sure that you set up the database format before you commence entering the actual data. This means naming and defining all the variables needed to hold the responses provided by participants. As you format your database, make full use of the software package’s variable and category labelling facilities (IBM Corp, 2016, Chap. 3; Field, 2018, Sects. 4.5 and 4.6, both provide very clear discussions of how to prepare an SPSS database using the full range of formatting functions). It is a good idea to use short mnemonic names for your variables, so that you easily remember what they measure. Once your database format has been created, you can enter your data in one of two directions: by moving down columns, entering one variable at a time, or across rows entering one participant’s data at a time. Data entry across rows is always preferred as this means you will be entering all the data for a single participant or case, before you move onto the next participant or case (i.e., the next row). Figure 20.1 displays a screenshot from SPSS (version 24) showing the ‘Variable View’ window for the practice data set of Worldwide Toy Company middle and senior management questionnaire responses described in Cooksey (2018).1 There The Worldwide Toy Company is a fictitious company, created by Cooksey (2018), to provide data for learning/practicing quantitative and qualitative analyses. All the data in the Worldwide Toy Company learning scenario are fictitious as well. The Worldwide Toy Company scenario, in brief, concerns a researcher’s investigation into a restructuring event in the company and its impact

1

902

20

How Should I Organise My Data?

Fig. 20.1 Screenshot of the SPSS (version 24) ‘Variable View’ window for formatting a quantitative database, prior to data entry

are 11 columns of variable characteristics available for formatting with respect to each variable. Variable names are created and entered in the first column. SPSS has specific rules for creating variable names: they must be less than 256 characters in length, include no blank spaces and only use letters, numbers and the underscore (_) symbol. Extended variable labels (= abbreviated questionnaire item content) can be entered in the ‘Label’ column and labels for the values of categorical and Likert-type items can be entered in the ‘Values’ column via ‘drop-down’ dialog boxes. One such ‘Values Labels’ dialog box is shown as open in the lower part of the Figure and illustrates how the categories of the ordinal scale variable ‘mgmt_lev’ are to be labelled. The column labelled ‘Missing’ is where you would enter other numbers (such as “9” for ambiguous or ‘Not Applicable’ responses) to be considered as missing in addition to the default ‘SPSS system-missing’ value. ‘None’ in this column means that no additional missing value codes, besides the default ‘SPSS system-missing’ value (represented by a blank variable entry for a participant), have been defined. The column labelled ‘Measure’ is where you enter the measurement scale for the variable, which is required for certain analyses. on management staff within a hybrid Explanatory Survey research frame. In the scenario, an explanatory sequential MU configuration yielded quantitative data from a questionnaire sent out to all 184 middle and senior managers in the company (146 usable questionnaires were returned) and qualitative data from transcribed story-telling interviews with 15 questionnaire participants who agree to participate in a follow-up interview (3 ex-managers, made redundant by the restructuring event, were also interviewed). The full training scenario contextual document can be obtained from the first author, Ray Cooksey, via email request: [email protected].

20.1

Preparation of Quantitative Data

903

Fig. 20.2 Screenshot of the SPSS (version 24) ‘Data View’ window showing the data from the first 30 participants’ entered into the spreadsheet formatted as defined in Fig. 20.1

Figure 20.2 displays a screenshot from SPSS (version 24) showing the ‘Data View’ window associated with the ‘Variable View’ window shown in Fig. 20.1 (see tabs in lower left of figure). Note that the first variable, ‘Inspector ID’, contains a unique identifying number for each questionnaire participant. All other variables are as defined in the ‘Variable View’ window. Missing responses for a variable are simply left blank in the database (see, for example, participant 17 who is missing responses on four questionnaire items). The last two columns in the spreadsheet are reserved for those participants who agreed to participate in a post-survey interview and the pseudonym assigned to each such participant (two such participants are shown in Fig. 20.2, Mary and Fran). The next stage of quantitative data preparation involves double-checking the completed spreadsheet database against participants’ original responses. This step is important, irrespective of whether you or someone else has entered the data, primarily because it is at this point that most data recording errors can be identified and rectified. If left uncorrected, data entry errors can seriously distort the outcomes from any statistical analysis procedure as well as magnify measurement errors in general. For example, consider the implications of finding the average age of a sample of 75 participants in a questionnaire where three participants had their ages accidentally recorded as 93 years old instead of 39 years (because the data entry person inadvertently reversed the digits “3” and “9”). If you have a reasonably small sample (say, less than 100 participants), it will pay to double check every participant’s original data recording form (e.g., completed questionnaire) against the data entered in the spreadsheet. However, if you have several hundred

904

20

How Should I Organise My Data?

participants, double checking everyone will be very tiring. In such cases, a sampling strategy could be employed, such as checking random participants or every 5th participant in the spreadsheet. If two people are involved in data checking, one to read the data line from the spreadsheet and the other to read along and confirm the values in the original raw data recording form, the data checking process will be more efficient and less tiring, especially if you rotate roles periodically. Obviously, you do not want to destroy the original recorded forms for the raw data until you are certain that the data in the spreadsheet are exactly the way that you want them.

20.1.3 Final Data Checking Once your data have been completely entered and double-checked in your spreadsheet, there is still one more stage of preparation that is extremely useful to undertake before embarking on your analyses. This stage involves what is called data screening and missing values analysis and it relies on using specific statistical functionalities provided in more powerful software support systems such as SPSS or SYSTAT. Tabachnick and Fidell (2013, Chap. 4) and Cooksey (2014, Procedure 8.2) provide thorough discussions and illustration of this stage of data preparation and checking. In brief, what data screening involves is using visual displays such as histograms and bar graphs and tabular displays such as frequency counts to summarise the distributions of each variable in the database (see Cooksey, 2014, Procedure 8.2, for some concrete illustrations). Even with thorough data checking, you can be sure that some errors will remain; you can also be sure that there will be other data anomalies you will have to deal with. • Data screening provides the opportunity to checking for things such as normality or skewness in your variables and outliers or extreme data values. Some extreme data values may be data entry errors (e.g., a mis-keyed value of ‘4’ instead of ‘1’ in the ‘gender’ variable for a male participant) and some may be extreme simply because of the nature of the variable itself (e.g., measures of speed for humans performing tasks will often be ‘positively skewed’, meaning that while there will be a large clump of reasonably fast people, there will also be relatively few but very large values, linked to very slow people). If you anticipate analysing a skewed (i.e., non-normal) variable using a parametric statistical test, you may need to consider using either a nonparametric test or transform the variable (e.g., by taking its logarithm) to make it more normal (see the discussion in Cooksey, 2014, pp. 104–108, 207–208, 385–390, for more on the issue of handling non-normal variables). • Missing values analysis is a specific procedure you conduct on all your variables to see if there are any anomalous patterns of missing data (e.g., items that tend to get omitted together) or if any specific sub-groups of participants tend to be missing data on certain variables. Missing values analysis also offers you the opportunity to decide whether you want to retain your data as missing values or

20.1

Preparation of Quantitative Data

905

estimate what the data values should have been (a process called ‘imputation’). You will need to be careful here as there are several different methods for imputing missing data, each with their own advantages and disadvantages and each requires certain assumptions about the nature of your missing data (see the discussions in Chap. 9 of de Vaus, 2002 and pp. 62–72 of Tabachnick & Fidell, 2013). If you do not have large amounts of missing data for variables or problematic missing data patterns, it is probably best to live with them as missing rather than trying to estimate them. The drawback here is that, for certain analyses, this may create instabilities in statistical calculations or sample size problems. Missing values analysis is discussed and illustrated more fully in Cooksey (2014, pp. 390–396).

20.1.4 A Final Word! Once your database has been checked, double-checked and screened, you should ensure that you make a back-up copy to keep in a safe place. Keeping a printed or hard copy of the database would also not be a bad insurance policy (although big data files may fell a lot of trees in the printing, so be careful here!). When you eventually write up the story of your quantitative data preparation for your Methods section or chapter, you can rely on the record in your research journal to assist your recall of specific details.

20.2

Preparation of Qualitative Data

Preparation of qualitative data implies two distinct but interactive stages in the data management process: (1) data gathering and recording (‘data creation) and (2) data transcription. The success of stage 2 is critically dependent upon decisions and processes implemented in stage 1. The data transcription process (one manifestation of the Transformative data-shaping strategy) can only encompass what is included in the data creation process; if data creation is of poor quality, then transcription will necessarily be of poor quality and data analysis will be degraded considerably. Yin (2011, Chap. 7) provides a very useful discussion of different approaches to recording qualitative data.

20.2.1 Data Creation The term ‘data creation’ provides a useful way to think about how qualitative data are collected and recorded. ‘Data creation’ signals that the data are not just ‘out

906

20

How Should I Organise My Data?

there’ to be gathered, but that you play an active role in their creation. If you conduct qualitative interviews, your data may be created via digital or camera recording and/or field notes. If you implement participant observation or other ethnographic methods, then field notes may be your primary mode of data creation. If you are doing document analysis, data creation may be in the form of downloading email, Word or pdf files from internet websites or physical (including scanned) copies of documentation. If you are taking a multi-media approach to your qualitative investigation, then your mode of data creation may be via audio, camera, video or webcam recording, web pages, or via. doc, .docx, .rtf, .txt, .pdf, .ppt. .pptx, .xls, .xlsx, .csv, .tif, .jpg, .png, .gif, .mp3, .mp4, .wma, .flv, .avi and .m4a internet file downloads and file formats. Each mode of data creation has its own benefits and drawbacks and each mode has implications for how the data are represented in a form suitable for analysis. Qualitative data creation is a rich and inherently unpredictable activity–you never know what you will discover. For this reason, it will be prudent for you to design a coherent and adaptable system for labelling and cross-linked different sources of data (Bazeley, 2013, Chap. 3, Miles, Huberman, & Saldana, 2014, pp. 50–52, p. 71; Richards, 2009, Chaps. 2 and 3, each provide a useful discussion of issues associated with the management of qualitative data, but from slightly different perspectives). Computer software support systems, such as MAXQDA, NVivo or dedoose, are explicitly designed to help you in this regard–a good reason to consider their usage if you are creating a reasonable volume of data. For example, Fig. 20.3 displays a screenshot from MAXQDA (version 12). The

Fig. 20.3 Screenshot from MAXQDA (version 12) showing the ‘Document System’ window, the ‘Document Browser’ window and an open ‘Memo’ window linking the memo to “Jean’s” storytelling interview transcript, which is open in the document browser window

20.2

Preparation of Qualitative Data

907

screenshot shows the ‘Document System’ window (left side) with all 18 qualitative storytelling interviews from the Worldwide Toys research scenario imported (from . rtf files). Organising documents into sets, in the document system window, allows you to group transcripts according to some criterion of interest. “Jean’s” interview transcript is highlighted in the window and that transcript appears, in its entirety, in the document browser window to the right. Memos can be attached to any document imported into MAXQDA and the memo attached to “Jean’s” transcript is open at the bottom right. Once transcripts have been imported, coding and analysis can commence. MAXQDA operates only on copies of transcript files, never altering the original data files.

20.2.1.1

Audio Recorded Data Creation

Audio recorded data creation focuses on your attention primarily on the verbal content of an interaction. Some non-verbal aspects will also be audible during the interaction, including aspects of paralanguage (e.g., interruptions or talking over, pauses, stutters, intonation, pace of speaking). However, there will be a lot of non-verbal information you will sacrifice access to if you opt for audio data recording. In particular, access to body language, gestures and larger scale environmental and contextual cues will be lost. If you choose this mode of data creation, you need to be sure that you are not sacrificing information that could be potentially useful to you (use of field notes may help here). The following are some points to remember/consider when using audio qualitative data creation: • Invest in a good quality digital recorder. When you choose a digital recorder, make sure it has the memory capacity to capture at least 90 min of interview data and a long-rated battery life. If the device has USB connectivity, this will facilitate file transfers to your computer. The smaller the device, the less intrusive it will be in the interview context. The higher the storage capacity, the less distracting fiddling you will have to do with the device during an interview. • Invest in several sets of long-life alkaline or lithium batteries, unless your digital recorder has a built-in rechargeable battery. Alternatively, you may find that fast rechargeable NiMH batteries (such as Varta 15 min Charge & Go batteries) provide an economical as well as a more environmentally friendly advantage. Ensure that you start each interview with a fresh set of batteries or with a fully-charged recorder. Nothing is more disruptive to an interview than to have to change dead batteries mid-stream or to plug your recorder into a wall socket or USB port for recharging. As well, you will likely lose some data between the time the batteries give out and the time you realise that the batteries have died and replace them. If you have a rechargeable model, make sure you take charging equipment with you for emergencies. • Establish a quiet environment in which to conduct your interview, free from background noise and at low risk of interruptions from phone calls, disturbances

908

•

•

•

•

•

20

How Should I Organise My Data?

outside windows and so on. Ensure that you and your interviewees turn off or silence mobile phones before commencing the interview. Position your recorder’s microphone in a way that will maximise sound quality from both interviewer and interviewee. A trial interview will help you to establish the optimal positioning. If you need to use stimulus materials as part of the interview, such as cognitive map building materials, question cards or paper stimuli, ensure that any extraneous noises, arising from any manipulation of the materials by you or the interviewee, do not degrade the quality of the recording (this may largely be a recorder placement problem). You should have plans in place for what you will do if your data recorder fails halfway through an interview or if a participant declines to be recorded. In such cases, you will need to have a manual recording alternative (perhaps with a shorthand system) that you can rely on to take notes during the interview. Keep your research journal close at hand for making notations, brief memos and mnemonic observations about the interview and how it is unfolding. If you do make such notations, try not to let it intrude on or interfere with the interview process itself. An alternative practice would be to wait to record such observations until immediately after the interview is concluded and the interviewee has left the room. You may find such notations handy when it comes time to finalising data transcripts. If you encounter a reference to a significant person, event or relationship during an interview, you may find it prudent to make a brief note of this to follow up later with a view to finding out more about it, documenting it and cross-linking it back to the interview where you first learned about it. When the interview is complete, label the recorded file or type with a unique identifier so you can trace the file back to the original source of data. This identifier can also be used as part of a reference ID for any transcriptions of the file. At the end of the day, if you use a digital recorder, download your audio files to your computer and make a backup copy straight away.

20.2.1.2

Data Creation in Field Notes

Data creation via field notes will allow you to attend to and record a wide range of contextual, verbal, symbolic and non-verbal information. Virtually anything you can attend to and perceive becomes potential data for recording. However, you need to realise that, because you, as the observer, are the recording ‘device’, your attention can only be focused on a small range of potential inputs. If you rely solely on field notes, you may run the risk of missing key events, interactions and environmental events that occur while your attention is directed elsewhere for observations—a risk that makes pairing field notes with other data creation modes highly attractive. If you choose this mode of data creation, you need to be sure that you are

20.2

Preparation of Qualitative Data

909

not overly selective in what you attend to and observe. The shorter the time you are on-site observing, the greater this risk becomes. The following are some points to remember/consider when using your field notes for qualitative data creation: • Record all field notes (either in your research journal or a dedicated field notebook) using what is called ‘low inference descriptor’ language. That is, record your contextual and interactive observations in language that is as descriptive and ‘objective’ as possible, focusing on verbal, non-verbal and environment/context-specific aspects. You want to avoid recording observations in language that suggests inferences, suppositions, interpretation or filtering on your part. Think of yourself, in your ‘observer’ role, as if you were simply a recording device. You simply want to record, in a kind of shorthand storytelling mode, what you see, hear, taste, smell and feel. If you are recording what you have heard, trying to record as close to a quote as possible (i.e., close paraphrasing). If you see someone pushing or yelling at someone else, describe the actions and events you see without making assumptions as to motives or intentions. Yin (2011, pp. 161–166) discusses how to develop your own transcribing language for taking abbreviated field notes. • Paradoxically, while you are recording your field notes using low inference descriptors, you may also be having thoughts and feelings about the data that are being created and about your part in their creation, aspects of your ‘participant’ role. You may also be forming preliminary analytical and interpretive ideas. You don’t want to lose these threads of thinking, so you should segment your field notes into (low inference) observational data and self-observations, interpretive notations and commentary (along the lines suggested in Chap. 3 for your research journal). These two segments form near simultaneous records of observations but from two different role perspectives: the observational record itself (in your ‘observer’ role) and your reflections on that observational record and your part in it (in your ‘participant/researcher’ role). Recording field notes with this kind of simultaneous yet dual perspective takes some practice and effort to get used to. Richards (2009, pp. 48–50) provides a useful discussion of this dual process focus in field notes. • It is preferable to generate field notes on the fly as you observe, but whether this is feasible depends upon the participant observer role you have adopted. Obviously, if you are a covert participant observer, you cannot record field notes in context. Even if you are an overt participant observer, it may be too disruptive to record on the fly. In such cases, you will need to record your field notes retrospectively, perhaps at the end of the day. This means you will have to rely heavily on your memory. Try to devise shorthand methods or mnemonic devices for creating concrete reminders of key events, issues, conversations and interactions during the day. These reminders can then be used as memory prompts when you record your field notes. • Ensure that you make back-up copies of all field notes so that an original data record can be preserved in a safe location.

910

20.2.1.3

20

How Should I Organise My Data?

Data Creation from Documents

Data creation from documents focuses your attention on visual, verbal and some non-verbal information that has been produced by other people. This means you will have access to information where you have had no impact on its production (Charmaz, 2014, pp. 48–49, refers to such information as ‘extant documents’). In this sense, the data will be unaffected by your role as a researcher, unlike other modes of data creation (which yield what Charmaz, 2014, pp. 47–48, refers to as ‘elicited documents’). The drawback here is that you must take the documents at face value with the content and in the style in which they were written; they may not contain all you hoped they would contain and may obscure more than clarify some things. For historical documents, there may be no opportunity to contact the author of the document to achieve clarification or expansion on issues. [This is the fundamental problem that the branch of interpretivism called ‘hermeneutics’ is concerned with.] The following are some points to remember/consider when using documents for qualitative data creation: • Secure appropriate permissions to use the documents if they are not in the public domain. Documents available on the internet can generally be considered as public domain and will not require permission to use as data sources. Other documents may be considered as copyrighted, private or commercialin-confidence (e.g., journal articles, cartoons or newspaper articles, emails, minutes of meetings, watermarked document drafts, development plans) and may require permission to access and use. Furthermore, gaining access to such documents may require you to give certain assurances about citing the source or de-identifying the source or specific provider of the document. • Ensure that you have the capacity to safely store and back-up all the documents you obtain. Again, you are preserving the ‘raw’ data here. Storage may be on a hard disk or USB drive in the case of documents in electronic form and in a physical cabinet in the case of hard-copy documents. • You will need to determine how to directly access the content of each document you procure for analysis. In the case of pdf files, the files may have been ‘locked’ against text copying. This will mean that you will have to work on a hard copy, retype the document or scan the document into electronic form for coding and analysis. There may be copyright issues associated with scanning a document, so ensure you clear this process with someone familiar with copyright legislation, such as the Copyright Officer at your university. For electronic documents that are not locked against text copying and where you may want to use only selected parts of the document, you can simply select and copy the desired text into a new file and save it, noting, of course, all the relevant citation information you might need. If the documents have pictorial information (e.g., photos, graphs), you will need to decide if you want these to become part of the data you analyse. If so, you will need to preserve them in any new documents you create.

20.2

Preparation of Qualitative Data

20.2.1.4

911

Multi-media Data Creation

Multi-media data creation is often a highly technologically-dependent mode of data creation. It may allow for the exploration of multi-sensory information in real time, with access to at least visual, verbal and non-verbal cues as well as to some larger scale environmental and contextual cues. If the multi-media data record is something produced by someone else outside the context of your research (e.g., a recorded TV interview with a prominent CEO that you download from the internet), then what you have is essentially a richer form of data creation than documents. If you are producing the multi-media data record yourself, then you can pick and choose what you focus on, which may expose you to the risk of missing important things while you focus your attention elsewhere (essentially, the field notes problem). Note that there are modes of multi-media data creation which are not necessarily technology-dependent, such as cognitive or cause mapping (although there are software packages that can be of assistance here, such as Inspiration or Decision Explorer) or rich pictures (soft systems methodology), where the research participant actively produces a data representation of their own (recall the Visualisations participant-centred strategy). The following are some points to remember/consider when using multi-media qualitative data creation: • You need to ensure that you have the requisite storage capacity for the type of media being collected (via internet download, CD-ROM, DVD or Blue-ray) or generated (via digital camera or webcam). The corollary to this is that you have the multi-media data stored in a format that the software on your computer can read without errors. • If you are actively generating the multi-media data record using a digital handheld camera or mobile phone camera, for example, you need to ensure your device is freshly charged and that you have a spare fully-charged battery pack. You will also need to ensure you have the correct type and capacity of storage media for the camera. If you are employing a cognitive mapping methodology and your participants are actively generating a map via drawings, whiteboard or post-it notes, you will want to keep a permanent record of the map as the participant has produced it. A digital or mobile phone camera (with a 5 mega-pixel camera or better) would be ideal for photographing and storing the map image. • If you are actively generating the multi-media data record using a digital handheld camera or a webcam, you may wish to edit the file to remove irrelevant segments, intrusions or blown ‘takes’ (transparency is essential when you do this). This means you will need to have appropriate editing software you can use and the appropriate expertise to use it. Every file should have a unique identifying number or label attached to it so that you can trace any transcribed material back to the appropriate file. • You should also consider how your qualitative data analysis support software (e.g., MAXQDA, NVivo, dedoose, ATLAS.ti) requires multi-media files to be stored and flagged for analysis.

912

20

How Should I Organise My Data?

20.2.2 Data Transcription For many types of qualitative data, preparation means transcribing the data, or at least the audible portion of it (in the case of multi-media data) into a text suitable for analysis (either in hard copy or electronic copy—the latter being required if a software support system like MAXQDA, NVivo or dedoose is to be used). Qualitative data are seldom gathered in the final form that will be used for analysis. For recorded interviews, for example, the actual recorded files are almost never analysed. You first prepare transcripts of each recorded interview (see Kuckartz, 2014, pp. 122–129). These transcripts, whether in hard copy printed form or stored electronic form (e.g., Microsoft Word files), become the final form of the data to be analysed. For multi-media data creation, the information contained in even a short segment can be overwhelming. To cope with this burden, you will make decisions about what is important to focus on in each data record and perhaps generate a system of memos or notes to capture this focus. The resulting memos or notes would then become the form of data to be analysed. If documents are being analysed, they may be usable/codable in original form or may have to have extracts taken from them which contains the information directly relevant to your research questions. Whatever mode(s) of qualitative data creation you employ, you will need to plan for the transcribing or data representation process itself. Here are some issues you will need to think about and plan for: • You must decide what aspects are important (within each recorded interview, for example) to preserve and reflect in the transcript. You will have to decide things like: – Do you want contextual details about the interview (e.g., who, when, where, time of day, day of week) included in the transcript, perhaps set off by a special format, or in a separate file (which would then have to be clearly cross-linked to the interview transcript it is associated with)? – Do you want the emotional tone of the interview or of specific utterances reflected in the transcript as far as possible? – Do you want interruptions, stutters, mumblings and onomatopoeia (words that imitate sounds they are describing like ‘um’, ‘blam’, ‘buzz’, ‘meow’ or ‘whoosh’) recorded? – Should your transcript have line numbers, and should the text be segmented in particular ways? – Do you need pauses or gaps timed and recorded? – How will you reflect the intrusion of background noises or the fact that you could not understand what a participant was saying in the transcript? – How will you protect identities of people, named during interviews, in the transcripts (may involve creating pseudonyms or some other way of de-identifying people and organisations)?

20.2

Preparation of Qualitative Data

913

– How will you distinguish what the participant says verbatim from your paraphrasings/understandings of what they have said? – How you will distinguish between your talk and the talk of the interviewee in the transcript? – For group-based or focus group interviews, how will you identify and distinguish between different speakers in the transcript? Your decisions with respect to these and other emergent questions will partly depend upon the paradigm approach you have adopted for your research. For example, conversation or discourse analysis would require different things to be represented in a transcript compared to a grounded theory or more general thematic approach. The latter would be more concerned with thematic content and emotional emphasis whereas the former would be more interested in paralanguage, turn-taking and the general conduct of the dialog between interviewer and interviewee. There are some insightful discussions of issues associated with what to transcribe available in Bazeley (2013, pp. 73–81), Elliott (2005, pp. 50–56), Kuckartz (2014, Chap. 5, focusing on transcription to facilitate use of a computerised qualitative data analysis support system), Richards (2009, pp. 58–61) and Silver and Lewins (2014, Chap. 4, also focusing on transcription to facilitate use of a computerised qualitative data analysis support system). Easton, McComish, and Greenberg (2000) discuss some of the pitfalls associated with qualitative data transcription. Irrespective of your specific paradigm perspective on qualitative data, effective transcription will require you to create a coding system to represent each non-verbal aspect of an interview in the transcript so that it is readily obvious and can be dealt with appropriately during analysis. These codes should represent low-inference descriptors (i.e., simply denote what happened) rather than interpretations. The important thing to realise here is that it is you, as the researcher, who makes these coding decisions; there is no rule book to tell you what is important. Virtually all reputable qualitative data analysis software packages can cope with rich-text files and other enriched file formats which means that additional text formatting features such as font type, colour and size, boldface, underscore and italics can be used to define codes for transcribing documents. However, using these additional types of formatting features will likely add time and costs to the transcription process, so the most efficient types of transcription codes and symbols to use come from the standard ASCII text character set, which means that the standard typing keyboard can be used for an entire transcription. Table 20.1 shows some simple transcription coding symbols, defined using only ASCII characters, which can be used to reflect several different features of an interview in a transcript. You can see that there are several non-verbal aspects of interviews that can be represented. [Kuckartz, 2014, p. 126 presents a similar table displaying what is called the Jefferson Notation System for transcription, citing work by Jefferson, 1984; Peräkylä & Ruusuvuori, 2018, offer a more comprehensive of possible transcription symbols, especially useful if you are doing conversation or discourse analysis.]

914

20

How Should I Organise My Data?

Table 20.1 Some illustrative codes that could be incorporated into transcripts of qualitative interviews; other codes may be devised as needed Code

Illustrations in the context of a transcript

Meaning

>Jane< >Dick< ⋮

I hated it when >Jane< told me I had to fire >Dick
Acme< ⋮

We had real trouble when our major competitor >Acme< released their new cleaning products on the market I: Why do you think this has happened? P: I guess it happened because my boss has trouble taking feedback on board F: What do you think about this product? P1: I think it is really functional and well-made P2: I think its rubbish

Insertion of pseudonyms corresponding to actual organisational members, Regina Bailey (>JaneDickAcmepseudonympseudonympseudonympseudonymJohn< 12/09/06 interview, lines 12–15).

This type of transparency in qualitative data referencing is highly desirable but is only achievable if your transcript files are pre-structured to have line numbers. Some software support systems like dedoose import data files and automatically apply line numbers to them so that the results of any text searches can be corrected located and reported. Others, such as MAXQDA, automatically number paragraphs, but have a facility where you can change to line numbering if you wish. • Document data (e.g., emails, reports, minutes of meetings, strategic plans, annual reports) may be analysed directly in hard copy or imported electronically (e.g., .pdf or Word document files). However, you may have to de-identify the documents, if they are considered confidential or you have promised anonymity to whoever provided you with access to them. • Double check the originals of all, or at least a sample, of the data records against their transcribed versions to ensure that correct transcription has occurred. Inaccurate transcription leads to analyses that do not keep faith with the participant’s perspective. Furthermore, if you are using a software support system, double check that the file import process has successfully imported the file you wanted in the form that you wanted. Always keep a hard copy as well as an electronic copy of all document files stored in a safe place.

20.2.3 Memoing/Transcribing Maps and Multi-media Data Sources Some types of qualitative data do not need to be transcribed so much as they need to be summarised (see Richards, 2009, pp. 58–61). For multi-media data sources, including performances, your data summaries may take the form of field notes taken as you play the multi-media file or watch performances or memos you assemble as you watch or listen (you might link these memos to specific readings on an elapsed time counter so that they can point you right back to a data source location). These summaries or memos can then be typed into a Word document and coded and analysed just like any other source of data. Alternatively, if you use a computer support system, like MAXQDA, NVivo or dedoose, you can take advantage of their extensive memoing capabilities, where the memos themselves can be separately

20.2

Preparation of Qualitative Data

917

coded and analysed. Transcription rules aren’t needed here, just a clear idea of what you are interested in looking for/at. If you are employing a drawing, soft systems/rich picture, cognitive or cause mapping qualitative methodology (see, for example, Eden & Ackermann, 2002; Chap. 10 in Jackson, 2003; Stiles, 2004), you may have provided a whiteboard (perhaps with electronic copying capacities so the map can be printed), paper, flip chart, coloured pens, post-it notes or other moveable composition media for the participant to use to help them generate their map or picture. Once they have completed their display, you will need to decide what characteristics of the maps or pictures will be of interest for analysis (e.g., number of concepts, hierarchy, links, strength of links, themes, content of concepts) and you may need to scan or transcribe and perhaps formalise the structure of their map into an electronic form using Decision Explorer, Inspiration or PowerPoint. Alternatively, you may choose simply to photograph the map or picture they have produced and use it as a multi-media document. Generally, you will want a permanent record of the map as originally generated by the participant, if possible. However, in cases where you have interviewed the person as they produced their map, you may wish to elaborate or embellish their map using references to their interview. Here, you would be much more collaboratively involved in the creation of the final map and would have to make display and content choices that reflect this collaborative effort (Bryson, Ackermann, Eden, & Finn, 2004, provide a very interesting and informative review of cause mapping as a qualitative methodology as does McDonald, Daniels, & Harris, 2004).

20.3

Key Recommendations

This chapter has discussed a range of issues to consider with respect to the organisation of your data and preparation for analysis. These issues tend to get lost in the larger landscape of research conduct, so it is important to review some of the key points and recommendations here. • Data will almost never be analysed in exactly the form in which you gather them. Therefore, some forethought as to how the data will be organised and prepared for analyses is essential to ensure that they are in the correct form you need and are accurate with respect to all essential details. Once the original records for gathering data (e.g. completed questionnaires, interview digital files) have been destroyed, it is too late to rectify any problems. Part of your forethought must be clear consideration of the software support systems you plan to employ, if any. • For quantitative data, there are several important things to get right to ensure data accuracy: – Clearly set out the rules you will use to translate the data from original recorded form into the form you will need for statistical analysis. These

918

20

How Should I Organise My Data?

coding rules should anticipate how to deal with problematic responses and with missing data. These rules should also address how to code categorical responses and responses to open-ended questions. Record these rules in your research journal. – Part of the process for preparing quantitative data for analysis should include organising the data entry spreadsheet for the software package (e.g. SPSS, SYSTAT, Stata) you intend to use. Make full use of all variable labelling facilities offered by the package so that you have a very transparent data record that does not rely on your (fallible) memory for details. To do this, you need to be fully aware of the capabilities and limitations of your chosen software package. – Quantitative data entry is a process that should be tightly controlled where at least a sample of entries, if not all (preferred), are double-checked against original data sources for accuracy. Once data entry has been completed, you will need to screen your data for anomalies, missing value patterns, and possible violations of statistical assumptions before commencing your main analyses. • For qualitative data, there are also several important things to get right to ensure that the data are in a form amenable to analysis while remaining as close as possible to their original form and senses of meaning: – Data creation and data preparation are intimately linked—how you do one influences how you do the other. For example, audio recording of interviews ensures that verbatim utterances as well as some emotional and non-verbal content from participants as well as from yourself, as researcher, are available for transcription. Make sure you get the mechanics of the recording process worked out before you commence data gathering so that the recording process is as reliable and unobtrusive as possible. Using field notes or multi-media recordings as a data record means that you are working some distance away from the original events observed and conversations heard or engaged in and you must be clear as to what types of content you are interested in. Here it is essential to use low-inference descriptor language to try and minimise the influences of your own judgements as a researcher on the nature of the data you are preparing for analysis. In short, you don’t want to be ‘pre-analysing’ the data with your own preconceptions while preparing the data for analysis. If pre-existing documents are a source of qualitative data, you need to ensure you have the correct permissions to access and use them and that you have some understanding of the context and circumstances under which they were created, and if possible, by whom, when and for what purpose(s). – Data transcription is an important step in the preparation of many forms of qualitative data. You need to devote a fair amount of forethought as to how you will want the original data transcribed into the final form you need for analysis. Such transcription should capture and represent all the relevant details in the data that you wish to be available for analysis. Develop a set of

20.3

Key Recommendations

919

rules that you and/or your data transcriber can consistently employ to produce your final data records for analysis and record these in your research journal. A useful step will be to double-check at least the first few efforts at transcription, so you can verify the process is going as you have planned. Make sure you clearly understand the capabilities, limitations and requirements of any software support package you intend to use so that your data files are prepared and stored in the correct format(s). – For some forms of qualitative data, transcription may be less relevant than summarisation or image conversion. Here, the rule is stay as close to the original form of the data as possible.

References Bazeley, P. (2013). Qualitative data analysis: Practical strategies. Los Angeles: Sage Publications. Bryson, J. M., Ackermann, F., Eden, C., & Finn, C. B. (2004). Visible thinking: Unlocking causal mapping for practical business results. Chichester, UK: Wiley. Charmaz, K. (2014). Constructing grounded theory (2nd ed.). Los Angeles: Sage Publications. Cooksey, R. W. (2014). Illustrating statistical procedures: Finding meaning in quantitative data (2nd ed.). Prahran, VIC: Tilde University Press. Cooksey, R. W. (2018). The Worldwide Toy Company: A research data analysis training scenario. Unpublished training document. Armidale, NSW: UNE Business School, University of New England. de Vaus, D. (2002). Analyzing social science data: 50 key problems in data analysis. London: Sage Publications. Easton, K. L., McComish, J. F., & Greenberg, R. (2000). Avoiding common pitfalls in qualitative data collection and transcription. Qualitative Health Research, 10(5), 703–707. Eden, C., & Ackermann, F. (2002). A mapping framework for strategy making. In A. Huff & M. Jenkins (Eds.), Mapping strategic knowledge (pp. 173–195). London: Sage Publications. Elliott, J. (2005). Using narrative in social research: Qualitative and quantitative approaches. London: Sage Publications. Field, A. (2018). Discovering statistics using SPSS (5th ed.). Los Angeles: Sage Publications. IBM Corp. (2016). IBM SPSS statistics 24 brief guide. Armonk, NY: IBM Corporation. Retrieved April 3, 2018, from ftp://public.dhe.ibm.com/software/analytics/spss/documentation/statistics/ 24.0/en/client/Manuals/IBM_SPSS_Statistics_Brief_Guide.pdf. Jackson, M. C. (2003). Systems thinking: Creative holism for managers. Chichester, UK: Wiley. Kuckartz, U. (2014). Qualitative text analysis: A guide to methods, practice & using software. Los Angeles: Sage Publications. McDonald, S., Daniels, K., & Harris, C. (2004). Cognitive mapping in organizational research. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 73–85). London: Sage Publications. Miles, J., & Shevlin, M. (2001). Applying regression & correlation: A guide for students and researchers. London: Sage Publications. Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis: A methods sourcebook (3rd ed.). Los Angeles: Sage Publications. Peräkylä, A., & Ruusuvuori, J. (2018). Analyzing talk and text. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (5th ed., pp. 669–691). Los Angeles: Sage Publications.

920

20

How Should I Organise My Data?

Richards, L. (2009). Handling qualitative data: A practical guide (2nd ed.). Los Angeles: Sage Publications. Silver, C., & Lewins, A. (2014). Using software in qualitative research: A step-by-step guide (2nd ed.). Los Angeles: Sage Publications. Stiles, D. R. (2004). Pictorial representation. In C. Cassell & G. Symon (Eds.), Essential guide to qualitative methods in organizational research (pp. 127–139). London: Sage Publications. Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston: Pearson. Yin, R. K. (2011). Qualitative research from start to finish. New York: The Guilford Press.

Chapter 21

How Should I Approach Data Analysis and Display of Results?

21.1

Preliminary Thoughts

We have mentioned in earlier chapters the need to plan for the processes of analysing your data. The intent, nature and trajectory of data analysis emerges from considerations of the research frame, research configuration, pattern of guiding assumptions and type of data (quantitative or qualitative) as they intersect your research questions/hypotheses. The basic goal of all forms of data analysis is to build meaning from the raw data and convey that meaning to one or more specific audiences. Data analysis provides key pathways for telling the stories about what you have learned through your research journey by helping readers/users connect evidence, including strategic data displays, with those stories. Quantitative analysis deals with data in the form of numbers, measurements and indices whereas qualitative analysis deals with data that are in non-numerical form, which can include recordings, documents and transcripts, images, websites and films/videos. For certain purposes, via the Transformative data-shaping strategy, qualitative data may be transformed (i.e., ‘quantitised’) into a quantitative form prior to analysis, e.g., participants, words and codes can be categorised, counted, ranked or rated, yielding quantitative data. Equally, quantitative measurements can be ‘qualitised’ such that richer interpretive meaning is attached to the numbers. No matter what type of data you have gathered, analysis will almost always transform, condense, aggregate, re-represent, thematise or categorise the raw data to build meaning. For research guided by the positivist pattern of guiding assumptions (within an Explanatory, Survey or Evaluation research frame, for example), your analytical approach will likely be constrained by your research questions/hypotheses and the types of quantitative measures you create and implement to assess constructs of interest. There is some flexibility here, but not very much; there are only certain analyses that make sense for certain kinds of data, samples and hypotheses. In this case, your planning can be very specific, and you can get an idea of the resources and skills you will need very early on in the research process. This will also help © Springer Nature Singapore Pte Ltd. 2019 R. Cooksey and G. McDonald, Surviving and Thriving in Postgraduate Research, https://doi.org/10.1007/978-981-13-7747-1_21

921

922

21

How Should I Approach Data Analysis and Display of Results?

you to anticipate how you will have to prepare your data for analysis. For research guided by an interpretivist/constructivist pattern of guiding assumptions (within an Exploratory, Explanatory, Indigenous, Feminist or Action research frame, for example), your analytical approach may be influenced partly by your research approach (e.g. grounded theory, phenomenography, social constructionism, discourse analysis, conversation analysis) and partly by your need to dig into the meanings carried in the data in order to address your research questions. With qualitative data, there is much more flexibility in how you can approach your data; there may not be one right way or one set of rules to guide you and there is often ample room for creativity. This very flexibility makes it a bit more difficult to plan specifics regarding your approach to the data. However, you can anticipate some of the coding and sense-making approaches and data display methods you might like to try as well as deciding, early on, if you will use a computer-supported approach to your analyses. This anticipation can also help you to make some important decisions such as, for example, exactly what information—verbal and non-verbal— you want represented in the transcriptions of recorded qualitative interviews prior to analysis (discussed in Chap. 20). Of course, it may well be that you gather both quantitative and qualitative data through a pluralist approach to your research (as would be likely in the Action, Evaluation, Transdisciplinary, Developmental Evaluation research frames). This will mean you need to not only think about how best to deal with each type of data on its own but also about how you might bring their stories together or how you might use both data types jointly to convey a richer story. This will demand that you expand and enlarge your skill set to meet such challenges. The same challenge confronts you if you gather data all of one type (quantitiatve or qualitative) using different data gathering strategies and/or differing types of data sources. While it is certainly important to plan your approach to data analysis, you should also remember that when you are working through the analysis process, there will be many choices and decisions you will have to make, especially when surprises and anomalies occur. Also, you need to realise that awareness of your audience(s) should influence your data analysis and display choices, at least to some extent. Who you are trying to convince, therefore, becomes an important consideration for analytical and data display decisions. For a master’s or PhD thesis/dissertation, your initial readership will comprise technically and methodologically competent academic researchers (i.e. supervisors and examiners) who will be looking for things such as sensitivity, technical accuracy and perhaps even sophistication in your approach as they make judgements about your competence as a developing researcher. They will look for a tight connection between your research questions/ hypotheses, the data themselves, and the stories that emerge from analyses of those data. They will also be looking for a sufficient level of detail so that they can judge the integrity and depth of your approach. For a professional doctorate dissertation or portfolio, your audiences will likely comprise a broader mixture of academics and professionals and, to some extent, they may have somewhat opposing interests in the level of sophistication and detail you display about your data analyses. A finer balance will have to be struck in such cases and it might be prudent to rely on visual

21.1

Preliminary Thoughts

923

displays a bit more heavily. For a journal article, your readership (e.g. reviewers at first, then a more general readership if the paper is published) will probably be wider but still technically and methodologically sophisticated. However, they will generally have a much wider range of guiding assumption commitments and expectations as well as a view toward reading less rather than more detail about your analytical processes (for journal space reasons if nothing else!). Thus, as part of your analysis processes, you will need to think carefully about how you want to display your results and draw your conclusions, keeping in mind both your research goals/questions/hypotheses as well as your audience(s). These considerations loom large when you are writing up your results but are also well worth thinking about a long time before that. You will have to make strategic decisions as to what you want to present in the text of your results and discussion chapter (or equivalent) and what you want to put in an appendix. We say ‘strategic’ because your choices will affect the convincingness of your overall story, and you want to make the most convincing presentation with minimal side trips and distractions as it is your audience(s) who must be convinced. Your focus should always be on your research questions/hypotheses. Even in your research proposal, you will want to anticipate how you might approach the analysis and display of your data in the context of your research questions/hypotheses. In this regard, you will be looking ahead to the end outcome of your thesis, dissertation or portfolio and making choices about what shape you want it to take. Another aspect of this planning process will involve you making decisions as to whether you will employ a computer support system of some type to help you manage your data analyses. Every support system, even a manual one, will have its own requirements that will constrain and shape how you record, import or input your data, and you will need a system of procedures to help you manage your data within those constraints. If you do decide to use a computer support system, you will need to know which system(s) you will have access to (e.g. SPSS, Stata, eViews, SYSTAT, Excel for quantitative data, and MAXQDA, NVivo, dedoose Decision Explorer, ATLAS.ti, The Ethnograph for qualitative data), and what their associated requirements will be for managing data and carrying out analytical activities. If you don’t know how to use your chosen software package(s), you will have to learn how to use them before you can commence data entry and analysis. It is not ethical, in the context of Honours or postgraduate research, to give your data to someone else to analyse—you must analyse your data yourself. You may not be able to anticipate all the decisions you may need to make about analysing your data, but the more you can anticipate and plan for, the smoother your analytic journey will be. Here are some general considerations to get you thinking: • If you are dealing with quantitative data, don’t ‘brutalise the data’ by over-analysing them, and don’t ‘offend the data’ by under-analysing them. We have seen too many results and discussion chapters where the same data have been analysed three or four different ways (e.g. using a regression model, using an analysis of variance, and using a structural equation model), all to support the

924

21

How Should I Approach Data Analysis and Display of Results?

same story. This is overkill and will suggest to an examiner that you were uncertain as to the best analysis method, and so you took a shotgun approach to the data. We have also seen many results and discussion chapters where the data have been under-analysed, leaving the student vulnerable to the possibility of drawing statistically indefensible conclusions. The most common error here is to adopt a univariate testing approach for data that are inherently multivariate in nature. For example, if you have a questionnaire with 25 job satisfaction items and one of your hypotheses is to show differences in job satisfaction between male and female managers, you could analyse the data using 25 independent ttests (gross underkill; almost guaranteed to produce a false claim of statistical significance), or you could conduct a factor analysis first to condense the 25 items into perhaps 3 or 4 interpretable dimensions, score participants on those dimensions and then compare males and females on those dimensions simultaneously using a multivariate analysis of variance (MANOVA). Many students would prefer to take the multiple t-test approach because it is easier and more mechanical, and perhaps because that is the level their past training has brought them to. Unfortunately, we have to say that there have been PhD theses that have adopted exactly this multiple t-test type of approach. However, this approach is statistically indefensible, and a sharp examiner will bring the hammer down quite vigorously. If you don’t know how to approach the analysis of multivariate data with an appropriate level of sophistication, then you need to learn. Get yourself up to speed through an extra course or through tutorial sessions with an expert (for example, see Ray’s YouTube playlist on statistical methods: https://www.youtube.com/playlist?list=PLiDmwXFBUqUvx9MnJQULJDBgbZl8idcv). This is one reason why planning your analyses at proposal time is so important—you can detect skill deficits that you will need to rectify and have enough time to do this before your data come rolling in. • If you are dealing with qualitative data, don’t just skate over the surface of the data, dig deeper; however, don’t dig too deep too soon, otherwise you may lose sight of the context(s) in which your data are emerging, and you may risk your analyses being influenced by your preconceptions, or worse by you going native. You must hit the right balance between breadth and depth in focus in your analyses and that is something you will have to judge; it is not something that someone from the outside can dictate. If you are coding qualitative data, you need to know that coding is only a first and more surface-level step; interpretations do not reside in the codes themselves, but in the senses you make of those codes in the larger contexts in which your data were gathered and in the patterns and relationships that emerge when you look both within and across your codes and your various data sources. In a sense, qualitative data analysis is like building a scaffold of meanings. Of course, you need to establish a solid and stable base (something that effective coding can help you to do), but you will only reach the pinnacle (i.e., the complete story) by attending closely to building the structure, content and meaning of every layer of interpretation you add to that scaffold. • Be selective in what you report from statistical analyses and qualitative analyses. The right amount of detail (e.g., statistics, graphs, diagrams, quotes) is critical;

21.1

Preliminary Thoughts

925

too little and you will not have presented sufficient evidence to support a conclusion or interpretation; too much and your message will get swamped by the volume of numerical, pictorial or verbal detail. • Following on from the previous point, make sensible choices about the balance between figures and graphs, diagrams, tables, quotes and your discussions of them in the text (i.e., your narrative). This applies, irrespective of whether you are analysing qualitative or quantitative data. We will attack this issue head on in later sections of this chapter. • When you are analysing qualitative data, take concrete steps to ensure that you keep your emerging interpretations and data displays as close to the original data (and their context) as possible. This will help you to tell a more transparent and authentic story and provide ready access to quote material that will be useful for anchoring or exemplifying specific interpretations. Your research journal can be of help in this regard, as can using a software support system such as MAXQDA, NVivo or dedoose. • When you begin writing up your analyses, ask your supervisor(s) or a colleague to read and critique what you have written. Do this in small chunks at first until you find your feet. You don’t want to write your entire results and discussion chapter, then discover that you have got the balance all wrong.

21.1.1 Analyses of Data Inform the Multi-layered Stories that Convey What You Have Learned Different kinds of stories may be pursued through data analysis depending upon your research frame, pattern(s) of guiding assumptions, contextualisation and positionings, MU configuration employed, data gathering strategies, type(s) of data gathered, and sampling approach(es) for choosing data sources. Figure 21.1a provides a visualisation of different analysis pathways you can follow, depending upon your own research needs. The left branch of the figure focuses on quantitative data analysis, which tends to progress in a relatively linear fashion; the right branch focuses on qualitative data, which tends to yield their stories in a more dynamic and iterative fashion. The data types can each be transformed into the other (qualitising quantitative data or quantitising qualitative data), as required, to help support stories emerging from the other type, where the need arises. There are several important lessons in Fig. 21.1a: (1) there are layers of stories (identified down the left side, splitting the diagram up into horizontal regions) that can be assembled and conveyed using each type of data; (2) analyses and data displays influence each other in dynamic ways which will help to shape the layered stories you tell, and (3) under a pluralist approach, these stories emerge from a dynamic mixture of within data type analyses and/or between data type analyses, as appropriate (symbolised by the large rectangle of story-shaping strategies). When focusing between data types, you are working in the space carved out by mixed

DATA = MODEL + ERROR

Typically, linear models used to test models/theories under a specific set of assumptions; goal to facilitate theory testing & generalisation

Triangulate

Test

Create stories that emerge from qualitative & quantitative data patterns

Supplement

Integrate

Look for links/connections between emergent meanings/ themes/concepts/constructs

Convey/display contextualisations and emerging meanings, themes, categories, concepts, constructs

(e.g., Onwuegbuzie & Leech, 2005, p. 287)

• Numbers to profiles, patterns, types (e.g., data mining) • Numbers to factor/cluster names/themes • Graphs/interactions to descriptions/relationships of patterns/flows • Coefficients to relational stories

bQualitising

Create enriched stories that amplify/ qualify/characterise/flesh out qualitative patterns & relationships using other data sources & data gathering strategies

Create stories that build up qualitative theories/accounts emerging from qualitative data patterns

Look for higher-order connections between emergent meanings/themes/concepts/ constructs as well as patterns & relationships; Engage in theory & knowledge building

Create stories that qualitatively amplify/qualify/characterise/ flesh out/follow-up quantitative patterns & relationships

Augment

Seek convergences/commonalities as well as divergences/uniquenesses; Create/co-create new knowledge & understandings

Create stories that test quantitative models emerging from qualitative data patterns

Delimit

Focus between data types

Transform

Data preparation, including setting up document organising systems and implementing data transcription and storage

Focus within data type

Fig. 21.1 a Pathways for pursuing stories in different layers of analysis, both within and between quantitative and qualitative data types as required. b Exploded view of analysis story intentions following on from Fig. 21.1a

(e.g., Onwuegbuzie & Leech, 2005, p. 287) • Categories/themes to counts • Cross-category/theme relationships to co-occurrence counts (e.g., Leximancer, text mining) • Text to diagrams (e.g., systems diagrams, BOT graphs) • Coding of qualitative data/pictures/photos into quantitative measures

aQuantitising

Create enriched stories that amplify/ qualify/characterise/flesh out quantitative patterns & relationships using other data sources & data gathering strategies

Translate

Display patterns, relationships & models [e.g., statistical indicators & tables, graphs, networks, models

Create stories that test/refine quantitative models emerging from quantitative data patterns

Build/test/refine models, theories and/or accounts

Use correlational statistics to display patterns & relationships

Use graphical & numerical statistics to describe/ summarise measurements & contextual features

Data preparation, including setting up coding systems and implementing data entry and storage

Focus within data type

Qualitative Data Themes/constructs/meanings emerge, at least early on

21

Identify & characterise patterns & relationships

Build contextual & data-anchored descriptions & summaries

Data preparation & organisation

Layers of stories to convey

Where constructs/measures are theorised apriori and measurement processes are predetermined

Quantitative Data

Analysis: pathways depend upon research goals/research frame// MU configurations, whether single or multiple data sources are used, whether single or multiple data types are available and the layers/levels of patterns and understanding that are sought

Display meanings/patterns, diagrams & relationships [e.g., narrative, maps, Matrix displays, networks, relational & flow diagrams,

(a)

926 How Should I Approach Data Analysis and Display of Results?

Delimit

Fig. 21.1 (continued)

create stories that explicitly test or evaluate emergent ideas from either type of data; may require gathering of additional data; a highly focused augmentation; primarily enhances breadth

Translate

Test Augment

create stories that fully integrate knowledge/learning from within or between types of data at multiple levels/within multiple layers; enhances depth and breadth

Integrate

create over-arching stories to amplify/extend the dominant or main stories; such stories add layers/levels to understanding; enhances depth and breadth

Triangulate

create sub-stories to back/add value to the dominant or main stories; such stories elaborate within a hierarchical relationship supporting a specific level of understanding; may enhance breadth and/or depth of stories create stand-alone stories within each data type, then cross-compare/contrast within a data type or between data types; stories from each data type have equivalent standing; enhances breadth and depth of stories

Supplement

embellish dominant or main stories, emerging from either type of data, with added details that delimit, qualify, circumscribe, bound, contextualise; such stories produces enriched stories at a specific level of understanding; enhances story character

Exploded view of analysis story intentions

reinterpret/restate patterns observed in one type of data using another type of data or data from other sources or data gathering strategies

(b)

21.1 Preliminary Thoughts 927

928

21

How Should I Approach Data Analysis and Display of Results?

methods researchers but saying this does not demote or devalue the possibilities associated with working entirely within one data type. which, of course, can still be done in a very pluralist and convincing manner as shown by the lighter and darker solid arrow pathways to and from the rectangle. With a pluralist approach, you can achieve several different goals in terms of the kinds of stories you might wish to support through your analyses and these are further unpacked in the exploded view offered in Fig. 21.1b. Note that your pattern of guiding assumptions and your MU configuration can exert a powerful influence on the nature and extent of analyses you can undertake. In general, when we talk about data analysis, we are talking about activities that you undertake to finding meaning in the data you gather. Such activities are simply a means to an end, helping you to build up the stories you want or need to convey; they are not the end in themselves. The analytical approaches you take should be appropriate for the data you have (or will have) to hand. Too often, the choice is made the other way around, where data are forced through a specific analytical approach, perhaps because you are familiar with the approach, good at it or have no experience with other approaches or because it is an approach valued by a specific discipline or audience you want to reach (e.g., supervisors, readership of a journal). However, we would argue that such forcing detracts from convincingness and inhibits making clear arguments about why you adopted a specific analytical approach. Our discussion thus far has tended to be rather high-level and general, important for giving you an overall lay of the land. However, once you start looking deeper at the layers of stories that can be conveyed, there are quite a few types of stories that your analyses could help to support, depending upon your research frame, pattern(s) of guiding assumptions, research questions/hypotheses and specific MU configuration. In the following list, versions of stories typically supported by quantitative data are underscored; versions of stories typically supported by qualitative data are shown in bold italics highlight. • Participant/other data source-based stories focus on identifying/describing and summarising/displaying essential characteristics and backgrounds of the participants and other data sources (e.g., handiworks such as artefacts, documents, performances, secondary databases) in your research; may use analyses of measurements of nominal and/or ordinal scale variables, using descriptive statistical procedures; may extend to assembling, comparing and sharing backgrounds and roles of participants. • Theme/category-based stories focus on identifying/comparing/relating emergent themes/categories of data; may use content analysis of open-ended questions and involve looking for statistical relationships between counts in different categories as well as between category counts and other measurements and/or may involve analyses of handiworks/artefacts, giving rise to themes and categories, which you may then relate to each other. • Time-based stories focus on tracking/explaining patterns/changes/flows of events over time; may use statistical modelling/comparisons of measurements between occasions or time series analyses for longitudinal MU configurations;

21.1

•

•

•

•

Preliminary Thoughts

929

secondary data are popular for such stories, especially for financial, econometric and social policy research; may involve mapping and characterising the time-course of successive events in a research context in conjunction with participants’ perceptions and experiences, as might occur during a series of participant observation sessions. Contextual event-based stories focus on characteristics/perceptions/changes tied to occurrence of a specific event in the lives of participants within the research context; the focal event would typically not have been under your control, as researcher (e.g., organisational change, policy change, natural disaster, advertising campaign, studied using the Non-manipulative experience-focused strategy); may use statistical modelling/comparisons between occasions or time series analyses for intervention time-aligned longitudinal MU configurations where the event serves in the role of the intervention and the stories pursued are how measurements before compare to measurements after; secondary data are also popular for such stories, not only for financial and econometric data, but also for social and behavioural data as well; may involve before-after comparisons of perspectives, but can also pursue in-depth perspectives on the event itself, how participants explain it, how they react to it, what they think caused it, what its consequences were and so on. Role-based stories focus on how perceptions/meanings/understandings compare/contrast between participants occupying different roles in the research context; may make statistical group-based comparisons or use group-defined predictors to assess measurement/construct differences, where groups are defined by role categories (e.g., senior managers/junior managers/line workers, Head teacher/teachers, union members/non-members); may involve in-depth explorations of perspectives held by occupants of different roles (e.g., CEO/member of Executive Board, policy maker/policy implementer, Principal/Vice-Principal/Head teacher). Cognition/knowledge-based stories focus on what participants know/ understand/learn about one or more focal knowledge domains/issues/event/ tasks, including perceptions of causes and effects; especially relevant in the Action, Cross-Cultural, Indigenous and Evaluation research frames; may involve descriptive statistical appraisals and relational comparisons of scores on achievement/ability/competency tests or other measures of knowledge or learning (arising from, for example, Systematic observation-based strategy or Measurement data-shaping strategy); may also involve construct validation analyses (such as factor and reliability analyses); may involve in-depth explorations of participants’ discussions or representations of their knowledge and/ or learning (arising, for example, from participant observations or other Participant-centred or Interaction-based data gathering strategies). Cohort-based stories focus on how perceptions/meanings/understandings compare/contrast between participants from different cohorts (e.g., different school years, different cultures) in the research context; may involve statistical comparisons and modelling between different cohorts of participants, where

930

•

•

•

•

21

How Should I Approach Data Analysis and Display of Results?

cohorts are defined in terms of some important characteristic(s) and tracking of measurements may occur through time (combination with a time-based story); one set of cohorts may be nested within another set other in the case of multi-level statistical models (e.g., teachers nested within classes nested within schools); may involve in-depth characterisations and comparisons of perspectives and meaning across different cohorts, where cohorts may be hierarchically embedded (e.g., teachers within schools). Group-based stories focus on how perceptions/meanings/understandings compare/contrast between participants in different groups in the research context; may make statistical group-based comparisons or use group-defined predictors to assess measurement/construct differences, where groups may be pre-existing (quasi-experimental groups; e.g., defined by demographic categories) or randomly determined/controlled by the researcher (experimental groups or conditions); may involve in-depth explorations and comparisons of perspectives held by occupants of different groups, where the groups typically pre-exist in the research context, but also may emerge as analyses progress and different categories of perceptions/meanings/interpretations are discovered. Associational/relational stories focus on associations and/or relationships between concepts, themes, ideas, measures/variables; may target who, what, where, how and why; may use statistical models of various kinds and levels of sophistication to assess the inter-relationships between different measurements or measurement systems; may also involve construct validation analyses (such as factor and reliability analyses); typically involves higher-level analytical activities to re-organise, detect and visualise linkages and relationships in the data, e.g., between different codes (what Saldaña, 2013, refers to as second-cycle coding of first-cycle codes), between individuals, groups and/or cohorts and/or between codes and participants. Intervention-based stories focus on changes attributable to some researcher-produced or non-researcher produced intervention, and the implications of that intervention; may target who, what, where, how and why; may use intervention time-aligned longitudinal MU configurations and pre-test-post-test repeated measures analysis to test for intervention effects; may also compare different groups defined by different levels of an intervention (like a drug dosage) and which may include one or more control or placebo groups for comparison; may involve in-depth exploration/characterisation of participants’ perceptions/understandings/meanings focusing on a specific intervention, perhaps with a superimposed timeline (e.g., before the intervention, during the intervention, after the intervention); often, you, as researcher, will not have created or imposed the intervention (e.g., as when a school enacts a new policy for managing bullying and cyberbullying on school grounds). Hypothesis-based stories focus on testing research hypotheses proposing relationships between constructs/differences between groups; especially relevant under the positivist pattern of guiding assumptions in the Explanatory research frame; may use statistical models of various kinds and levels of sophistication to evaluate specifically hypothesised models (ranging from simple group difference

21.1

•

•

•

•

Preliminary Thoughts

931

predictions to complex structural equation models); may also involve construct validation analyses (such as exploratory or confirmatory factor and reliability analyses); may involve exploring/characterising participants’ views on hypothesised models and relationships or information/perceptions that may help to amplify or qualify what the hypothesis tests have or have not shown (here, for example, an explanatory sequential MU configuration can be very useful). Pattern-based stories focus on identifying/characterising/explaining emergent patterns of relationships between themes/concepts/constructs; may use statistical models of various kinds and levels of sophistication to assess patterns of inter-relationships between different measurements or measurement systems; data mining analyses, for example, would fall here, relying on large collections of secondary data; typically involves very high-level analytical activities to re-organise, detect and visualise more holistic patterns in linkages and relationships in the data, e.g., between different codes (what Saldaña, 2013, refers to as going beyond second-cycle coding), between individuals, groups and/or cohorts and/or between codes and participants; this is often what grounded theory seeks. Impact-based stories focus on identifying/characterising/assessing the impact and wider implications of a change or innovation process; may target who, what, where, how and why; especially relevant in the Transdisciplinary and Developmental Evaluation research frames, where the participatory inquiry or critical realist pattern of guiding assumptions is used; may use statistical models of various kinds and levels of sophistication to evaluate the impact of a change process or innovation using measurements/indices designed to reveal impact (e.g., failure or adoption rates, attrition, profit/loss, market share, competency levels or grades); may involve in-depth exploration/characterisation/anticipation of participants views of and reactions to a change process or innovation with a view to understanding impacts from multiple angles and points of view. Change-based stories convey a similar type of story to the previous one, but focus more on characterising/understanding how a change process has unfolded and what changes it has produced; may target who, what, where, how and why; may use statistical models of various kinds and levels of sophistication to evaluate the progress of a change process or innovation, often over time, as reflected by various measurements; may involve in-depth exploration/ characterisation/anticipation of participants views of and reactions to a change process or innovation with a view to understanding them from multiple angles and points of view; exploring the potential of future changes/ innovation may also be of interest. Reflective/past oriented stories focus on understanding/explaining perceptions and knowledge looking back into the past, thereby pursuing a historical view; may target who, what, where, how and why; often of interest in the Indigenous research frame; may involve statistical analyses of retrospectively-oriented measures or historical data (perhaps accessed via a secondary database); may involve analyses of reflective and/or biographically-oriented interviews that seek participants’ perspectives and may encompass entire life histories.

932

21

How Should I Approach Data Analysis and Display of Results?

• Prospective/future-oriented stories focus on speculating/anticipating/predicting/ forecasting what might happen in the future because of some change or innovation and exploring their implications; may target who, what, where, how and why; often of interest in the Transdisciplinary and Developmental Evaluation research frames; may involve the fitting, cross-validation and testing of predictive or forecasting models; may involve explorations of participants’ expressed views about the future, about potentials, about their perceived role (s) in the future with respect to the research context and so on. • Descriptive/summary stories focus on the who, what and where facets of a research context; especially useful in the Descriptive, Exploratory and Survey research frames; may involve demographic analyses of samples with a view toward characterising some population of interest; may involve producing narratives that capture participants’ perspectives as well as summaries of essential attributes of participants. • Relationship-based stories focus on the relationships between people, handiworks, artefacts and environments; may target who, what, where, how and why; especially relevant in the Transdisciplinary, Indigenous and Feminist research frames; may involve statistical/graphical analyses of social networks and other relationship-mapping approaches within an identified social or organisational/ institutional context; may involve analytical activities that unpack, characterise, map or otherwise unmask or reveal important relationships, especially between people, but also between people and the handiworks they produce and the environments they inhabit. • Cross-case stories focus on comparisons across case contexts: people, places, times, contexts, tasks, meanings, relationships; especially relevant in the multiple case-based MU configurations; may involve building and testing comparative statistical models as well as analytical activities intended to demonstrate equivalences between constructs measured in different cultural groups; may involve producing data displays and/or narratives that compare and contrast stories and meanings that have emerged from different case studies, often used to support transportability arguments. • Generalising stories focus on making inferences from the researcher’s localised sample to some population of interest as well as, perhaps, to other tasks that people might confront in their lives: people, places, times, contexts, tasks, relationships; especially relevant under the positivist and critical realist patterns of guiding assumptions; may involve creating statistical data displays intended to justify specific types or directions of generalisations or sophisticated statistical analyses (e.g., multi-level modelling). • Model building/testing stories focus on constructing/testing/making statements about meaning and relevance of models of some aspect of the world; especially relevant under the positivist pattern of guiding assumptions in the Explanatory research frame; may involve the use of different statistical modelling technologies (e.g., regression modelling, structural equation modelling; computational modelling and simulation) and the generation of data displays (e.g., structural equation diagrams with statistical outcomes overlaid, summary tables

21.1

•

•

•

•

•

•

Preliminary Thoughts

933

for model building/testing analyses, screenshots of simulations, time series analyses) intended to provide evidence for model-based conclusions. Theory building/testing/revising stories focus on building theory and/or testing and refining a pre-existing theory which causally/relationally connects concepts/ constructs; may involve the use of modelling and other statistical procedures (e.g., hierarchical regression modelling, structural equation modelling) to test specific theoretical and hypothesised propositions relating construct measurements in specific ways; may involve early analyses to identify key or core concepts/ constructs/relationships that can then be further unpacked, examined and refined through further data gathering and analysis(e.g., grounded theory). Power-based stories focus on identifying/describing/critiquing the power-basis of relationships: who, what, where, how and why; especially relevant under critical social science, Indigenous and feminist patterns of guiding assumptions; may involve analyses that focus on power-oriented relationship measurements and observations (e.g., social network analysis); may involve analyses and critiques intended to reveal power and dominance relationships in researcher observations as well as the narratives and perspectives offered by participants and other data sources. Integrative stories focus on bringing stories arising from different data sources, data types, different MUs together; requires more complex cognitive activities and creation of data displays that attempt to bring quantitative and qualitative stories together in a way that produces more convincing conclusions; may involve cognitive activities that attempt to bring various statistical and model testing stories together to produce coherent insights; may involve cognitive activities and creation of data displays that attempt to bring various qualitative stories together to produce coherent insights. Convergence/commonality stories focus on identifying/describing areas of convergence or commonality between different contexts, data sources, data types; may involve analyses and data displays intended to highlight convergence or commonality of perceptions, attitudes and behaviours (e.g., using multidimensional scaling, factoring or clustering approaches); may focus on analyses and data displays intended to highlight convergences or commonalities in meanings, experiences, perspectives across participants or other data sources or across research contexts. Divergence/uniqueness stories focus on identifying/describing areas of divergence or uniqueness between different contexts, data sources, data types; may involve analyses and data displays intended to highlight divergence or uniqueness of perceptions, attitudes and behaviours to particular sub-groups (e.g., using multidimensional scaling, factoring or clustering approaches or discriminant analysis); may focus on analyses and data displays intended to highlight divergences or uniquenesses in meanings, experiences, perspectives among participants or other data sources or amongst research contexts. Learning stories focus on what can be learned from specific data gathering strategies, data sources and data types that has implications for what might be done next in the research, including from pilot tests or trialling; especially

934

21

How Should I Approach Data Analysis and Display of Results?

relevant in sequential MU configurations; may involve some preliminary validation analyses to explore the characteristic of a new or pre-existing construct measure and providing some initial estimates of potential research problems such as participation rates, withdrawal rates, missing data problems and possible adverse reactions to measurements; may involve analytical activities intended to unpack and explore participants’ experiences of and reactions to the data gathering activities they have just experienced and may also incorporate autobiographical analyses of researcher/interviewer experiences of and reactions to data gathering activities and events. • Other stories or hybrids or variants of any above focus on assembling and conveying new and different stories for certain purposes or on more complex stories that synergistically combine learning from two or more of any of the above stories. Many stories will simultaneously convey two or more of the above focal intents. Other stories might emerge as important to convey depending upon the stakeholders who have an interest in your research.

21.2

Analysing Quantitative Data

21.2.1 Understanding Quantitative Analytical Frameworks The analysis of quantitative data can present a bewildering array of choices and considerations. Furthermore, to make defensible choices, you need to possess a minimum level of statistical knowledge, coupled with knowledge of the specific statistical software package you plan to use. By minimum level of statistical knowledge, we mean knowledge sufficient to (1) understand which choices are appropriate given your research questions/hypotheses, (2) understand which choices are appropriate given the nature and quality of your data, (3) understand conceptually (as opposed to computationally) what each procedure you employ is designed to do and what it produces, and (4) know how to interpret the analysis outcomes and how to competently write them up. There is no substitute for undertaking an appropriate unit of study if you do not currently possess the skills set out above. There are good textbooks (e.g. Cooksey, 2014, was written to meet the requirements of points 1 through 4 above) as well as good online resources that you can also tap into. Cooksey (2014, Chap. 3) provides a comparative discussion of a number of statistical packages currently available on the market (e.g., SPSS, Stata, SYSTAT, NCSS, Statistica, R, MPlus, eViews) but in the social and behavioural sciences, SPSS is probably king. There are also self-help books that can assist you in learning your software package—many of these focus on SPSS and are periodically updated as new versions of the software are released (e.g., Allen, Bennett, & Heritage, 2019; Field, 2012; Geiser, 2013; Pallant, 2016). Some resources (e.g. Field, 2018; Hair, Black, Babin, Anderson, & Tatham, 2010; Tabachnick & Fidell, 2013) do a very sound and comprehensive job of combining in-depth statistical training with SPSS training. If you need specialist consultation,

21.2

Analysing Quantitative Data

935

don’t hesitate to seek it out, but remember that the role of that specialist is only to provide you with guidance and advice, and not to perform and interpret the analyses for you. Statistical analysis of quantitative data can provide evidence for each of the layers of stories highlighted down the left-hand branch of Fig. 21.1a. Graphs and numerical indices provide evidence you can use for the “Build contextual and data-anchored descriptions & summaries” stories, relational indicators (e.g., correlations, regression coefficients), various multivariate procedures and graphs provide evidence you can use for the “Identify and characterise patterns and relationships” stories, and various analytical procedures (e.g., regression analysis, analysis of variance, structural equation modelling) can provide evidence you can use for the “Build/test/refine models, theories and/or accounts” stories. However, it is important to realise that these stories inter-twine and feed into each other, usually culminating in the building and testing of some sort of linear model, however simple or complex it is. Cooksey (2014, pp. 209–211) discusses what is referred to as the general linear model for statistical analysis for which Fig. 21.2 provides a schematic representation (Judd, McClelland, & Ryan, 2017, provide an excellent discussion of the DATA = DV

=

MODEL

b 0 + b1IV1 + b2 IV2 + ... + bk IVk +

What remains unknown & cannot be explained comparison assesses MODEL deficits & lack of fit

MODEL behaviour DATA behaviour

ERROR {= Random + Systematic}

MODEL

Comparison evaluates hypotheses via test statistics; can test overall MODEL contribution:

IV 2

IV p

e

DATA

What is known that is built into the model based on theory comparison assesses MODEL quality using an effect size measure, R2

IV 1

+ ERROR

MODEL behaviour ERROR behaviour as well as individual predictor contributions: Predictor behaviour ERROR behaviour

uncontrollable & unpredictable influences potentially controllable alternative plausible causes /extraneous variables not included in the MODEL; effective research control seeks to drive this component toward zero

Fig. 21.2 Conceptualisation of the general linear model as a data analytic framework. Adapted and expanded from Cooksey (2014), Fig. 7.3, p. 209; DATA = MODEL + ERROR representation based on Judd et al. (2017, Chap. 1)

936

21

How Should I Approach Data Analysis and Display of Results?

statistical thinking behind general linear model building and testing). The ‘equation’ at the top of the figure depicts the general linear model (linear because model components are added together) and shows that observed data (the dependent variable) are broken down into two parts: one part for the model, incorporating the various constructs of interest (the independent variables), and one part for error (which itself has two parts: random and systematic). Tests of hypotheses emerge as ratio comparisons between the behaviour of your model (or the behavior of one of its components) and the behaviour of errors (the rounded rectangles). The smaller you can make error (through the exercise of various forms of control to shrink the systematic component), the easier it will be to show your model is significant. Measures of effect size emerge from ratio comparisons between the behavior of your model and the overall behavior of your data (the oval). The larger your effect size, the better quality, more meaningful and practically useful your model will be. It is useful to get a visual impression of what a selection of model building/testing statistical approaches involve so that you can get a feel for the nature and logic of the model being constructed. Regardless of the specific statistical package you use, when you conduct an analysis for a constructing and testing a specific kind of model, you will need to specify all variables involved and appropriate assign them to their respective roles (e.g., dependent variable, independent variable, covariate), choose options for how the analytical process should be conducted and choose what numerical and graphical display outcomes from the analysis you wish to see. The statistical analysis procedure will estimate weights for the relational links, estimate the magnitude of errors, provide statistical tests for the significance of all weights and, where requested, estimates of effect size. Along the way, your choices for controlling how each analysis is performed should be strategic, to provide the evidence you need to address your research questions/hypotheses. Too often, postgraduates are tempted to simply request all possible output from an analytical procedure, which is counterproductive to building an efficient and convincing story. • Correlation—Fig. 21.3 shows a correlation model where IV and DV roles are not differentiated and the model and causal relationship is unspecified or unknown, denoted by r12 (see Cooksey, 2014, Procedure 6.1). • Simple Regression Model—Fig. 21.4 shows an extension of correlation to the case of prediction or causal linkage (only where appropriate); here the roles of IV and DV are clearly and unambiguously assigned (see Cooksey, 2014, Procedure 6.3). Fig. 21.3 Correlation

IV1

Fig. 21.4 Simple linear regression model

IV1

r12

w1

DV

IV2

e

Analysing Quantitative Data

1

IV (Group 3 = 0)

Coded information about categorical IV

IV (Group 2 = 0)

Fig. 21.5 ANOVA model

937

IV (Group 1 = 1)

21.2

w1

Fig. 21.6 Standard multiple regression model

IV1

IV2

IV3

IV (Group 3 = 0)

IV (Group 2 = 1)

2

IV (Group 1 = 0)

DV

e

w2

w1

w2

DV

e

w3

• Analysis of Variance (ANOVA) Model—Fig. 21.5 shows a 3 group comparison; categories (=groups) in a categorical IV are dummy-coded (using only 0s and 1s); 3 groups require 2 IVs (general rule is number of IVs required is one less than the number of categories); multiple categorical variables can be tested as well as their multiplicative combinations (called interactions) (see, Cooksey, 2014, Procedures 7.3, 7.6 and 7.10); effect size is measured by eta-squared. • Standard Multiple Regression Model—Fig. 21.6 shows 3 independent variables all considered at the same time; some IVs may be categorical if coded appropriately as per the ANOVA model; different transformations of IVs and DVs can allow for testing of a diverse range of models (e.g., logistic, logit, power polynomials; probit, tobit; see Cooksey, 2014, Procedure 9.8); effect size is measured by R2. • Hierarchical Multiple Regression Model—Fig. 21.7 shows 4 independent variables entered into the model in 2 sets; the sets of IVs are considered in a specific order, according to logical or theoretical arguments made by you, as the researcher; those sets entered later in the analysis have their relationships with the DV assessed over and above what previously entered sets of IVs explain in the DV; provides a useful theory-testing environment (see Cooksey, 2014,

938

21

Fig. 21.7 Hierarchical multiple regression

How Should I Approach Data Analysis and Display of Results? IV Set I considered 1st IV1

wI,1 IV2

wI,2 DV

IV Set II considered 2nd

e

wII,3

IV3

wII,4 IV4

IV (Group 2 = 0)

IV (Group 3 = 0) IV (Group 3 = 0)

2

IV (Group 2 = 1)

1

IV (Group 1 = 1)

Coded information about categorical IV

IV (Group 1 = 0)

Fig. 21.8 MANOVA model

w1a

DVa

ea

DVb

eb

w1b w2a w2b

Fundamental Concept IV and Procedure 7.13); incremental increases in R2 reflect unique contributions of each added set of IVs. • Multivariate ANOVA (MANOVA) Model—Fig. 21.8 shows a 3 group comparison; same possibilities for assessing IV contributions to explaining multiple DVs; any experimental or quasi-experimental design analysable with ANOVA can be analysed with MANOVA (see Cooksey, 2014, Procedure 7.16); effect size is measured by multivariate eta-squared. • Principal Components Model—sometimes referred to as just factor analysis; analyses patterns of correlations between IVs (e.g., questionnaire items); principal components IVs contribute to one or more statistically identified composite DVs (see Fig. 21.9); direct causation is not inferred, hence the dashed lines; goal to show which IVs work together to measure the same construct and helps to

21.2

Analysing Quantitative Data

Fig. 21.9 Principal components model

939

IV1

IV2

IV3

IV4

IV5

IV6

w1

w2

Composite/ Component 1

w3

w4

w5

Composite/ Component 2

w6

reduce the number of DVs you need to focus on in subsequent analyses; this can be a useful construct validation procedure (see Cooksey, 2014, Procedure 6.5). • Common Factor Analysis (CFA) Model—principal axes latent IVs cause the observed scores on DVs and uniqueness/error is explicitly included in the model (see Fig. 21.10); analyses patterns of correlations and factors explain only common variance shared amonsgt the DVs; this can be a useful construct validation procedure; if you prespecify which DVs are caused by which IVs, then a confirmatory factor model is constructed (see Cooksey, 2014, Procedures 6.5 and 8.6). • Analysis of Covariance (ANCOVA) Model—Fig. 21.11 shows an approach for controlling the influence of one or more extraneous variables (their role is termed ‘covariate’) prior to assessing the impact of any IVs on the DV; analysis proceeds in two stages: Stage 1—assess covariate influence and adjust DV for covariate influence using the resulting model by subtracting the model’s prediction for each participant’s DV score from their observed DV score; Stage 2— test IV contributions to explaining what remains in the DV after adjustment for the covariate(s) (see Cooksey, 2014, Procedure 7.15). • Structural Equation Model—hypothesises causal links between a number of IVs and DVs; some DVs may change roles in the model, serving as a DV in one part of the model and as an IV for another part of the model (note the hybrid shape in the middle of the model); requires a special software package such as AMOS (an add-on to the SPSS package); Fig. 21.12 shows a structural model; it is possible to incorporate a measurement model as well, if constructs in the structural model each have a confirmatory factor model attached to them (see Cooksey, 2014, Procedure 8.7).

940

21

Fig. 21.10 Common factor model

How Should I Approach Data Analysis and Display of Results?

u1

DV1

u2

DV2

u3

DV3

u4

DV4

u5

DV5

u6

DV6

w1 IVConstruct 1

w2 w3

w4 IVconstruct 2

w5 w6

Fig. 21.11 ANCOVA model Covariate

DV

wC

e

IV (Group 3 = 0)

IV (Group 2 = 0)

1

IV (Group 1 = 1)

Coded information about categorical IV

w1

IV (Group 2 = 1)

IV (Group 3 = 0)

2

IV (Group 1 = 0)

DV

e

w2

21.2.2 What Are Some Useful Tips and Considerations for Quantitative Analysis? It is easy to get lost in the maze of statistical possibilities for quantitative analysis, so there are some key strategies that may help you keep your feet on the path: • Focus your analyses on essentials and try to avoid tangential analyses. The easiest way to keep your eye on the ball here is to always focus on your research

Analysing Quantitative Data

IV1

IV2

ea

w1a w2a

IV6

Fig. 21.12 Structural equation model

941

DVa

21.2

w6b

w3a w3b

eb

w4b

IV4

w5b

(Group 1 = 1)

IV5 (Group 2 = 0)

IV3

DVb

questions/hypotheses and to do only the analyses that are necessary to address them—without overkill. By overkill, we mean analysing the same data/ relationships several different ways and then presenting all those different results. • There is a role for preliminary analyses for data screening and assumption checking. You would typically not report on or display such analyses unless you found anomalies, such as non-normality or outliers. If you do find anomalies, you need to describe them and decide what you will do about them. For example, you may find that some participants have extreme scores on a measure (i.e., outliers) and need to decide whether to consider their scores as missing. You may find that one or more variables are non-normal in their distribution and need to decide whether to transform them prior to their use in analyses that require the normality assumption or to use a nonparametric analysis procedure. Finally, if you analyse patterns of missing data, you may need to decide whether the proportion of missing data for one or more specific variables is such that their removal from certain analyses is warranted. • You may also conduct other types of preliminary analyses which you would report outcomes from. For example, you may have a large number of variables targeting different attitudes and wish to reduce the number of variables for your main analyses. In such cases, you might wish to perform a factor analysis followed by a reliability analysis to condense the large number of variables to a smaller number of composites or factors for which you would calculate a new score for each participant. We would call this a preliminary analysis if you did not have a specific hypothesis or research question devoted to it. Instead, such analyses help set the stage for analyses which do target your research questions/ hypotheses. We will discuss this type of analytical trajectory in more detail below. There may also be a role for certain supplemental analyses, because quantitative data usually throw up surprises and trends you may wish to follow up. However, keep in mind these are supplemental analyses and they may best be dealt with by inclusion as appendix material in your thesis/dissertation/portfolio, and then given just a brief narrative discussion in your results section or chapter.

942

21

How Should I Approach Data Analysis and Display of Results?

• Use your research journal to record all your thoughts about your analysis processes and to record all the decisions you have made along the way. You might even make checklists to work through to ensure that you conduct the analyses you want in the most appropriate order. Record any processes you employed for creating or transforming variables as well as your logic for doing so, and carefully note which variables were involved. • Employ all the labelling facilities offered by your chosen software package. For example, SPSS offers facilities to (1) give short labels to every category of nominal, ordinal or Likert-type interval scale variable, (2) attach an extended label (such as actual item content) to each variable, and (3) insert annotations, titles and comments directly into the SPSS output file. Being consistent in your labelling practices throughout your analysis process will provide continuous memory aids to what you have done should you have to leave the analyses untouched for a period of time (not an uncommon occurrence in the postgraduate research journey!). If you create new variables (e.g., by averaging several variables together to calculate scores for factors), make sure you remember to fully label those new variables as well. • Keep a hard copy, appropriately labelled and dated, of all statistical output you generate. To save paper, you might wish to edit the output to delete tables and graphs you know you will not need, and then print the edited output by printing several output pages per page of paper (most printers allow for printing multiple pages per page of paper; four SPSS output pages, in landscape format, will fit on a single A4 page and still remain comfortably readable). This practice will not only provide a backup system in case of hard drive crashes, it will also allow you to record your thoughts and interpretations directly on the hard copy. A useful practice is to use a highlighter to emphasise those portions of the output and key numbers and statistics you will want to report when you write up your results. You might consider creating a separate research notebook to contain all these annotated hardcopies. • Make a clear decision early on as to what criterion you will use to establish statistical significance (i.e. the alpha level or p-value for judging significance) and employ this criterion throughout. The conventional standard is, of course, p = 0.05, which expresses a willingness to be wrong in your conclusion of ‘a significant result’ 5 times out of 100. However, you should know that this is just a convention; you have the capacity to set this criterion at any level you and your supervisor(s) are willing to live with (what you anticipate you could successfully argue for when writing for your target audience(s)). If you do deviate from the p = 0.05 convention, you will need to be very clear as to your logic for doing so. A reason to set a stricter criterion for significance (say, p < 0.01 or p < 0.001) might be to partially account for conducting many tests of hypotheses using the same set of data or to acknowledge that decisions with potentially significant consequences (such as associated high costs or policy change implications) may hinge upon your claims of significance. A reason to set a more relaxed criterion for significance (say, p < 0.10) might be if you are

21.2

Analysing Quantitative Data

943

doing exploratory research in a new area where not much is known and you are looking for possible avenues to pursue in further research. • Some social and behavioural researchers will say that your decision about significance is a dichotomous one; that is, you decide if a specific result is significant or non-significant relative to some specified level of significance such as p = 0.05. Other researchers are willing to consider the decision about significance to be a trichotomous one; that is, you decide if a specific result is significant, marginally significant (or suggestive), or non-significant relative to some specified level of significance such as p = 0.05 or a specified band of marginal significance such as p falling between 0.05 and 0.10. The trichotomous criterion is generally more acceptable in exploratory research where not a lot of previous research has been done and you are looking for future possibilities and trends to investigate. You will need to decide early on, which decision criterion framework you are willing to implement, and then make all subsequent statistical decisions consistent with the stance you adopt. • A fair proportion of social and behavioural research is multivariate in nature (i.e. involving many measures such as questionnaire items or financial indicators) and is conducted without a strong apriori idea of the structure of the questionnaire or other instruments. Such research can be deemed exploratory because at least part of your analytical process is devoted to finding structure within the multivariate data set in order to reduce the dimensionality of the data (i.e. variable ‘condensation’), simplifying the conduct of statistical tests, and reducing the chances of making false claims of significance (i.e. reduce the chances of making a Type I or alpha error). This is often the pathway that you must employ if you design your questionnaire or other data gathering instrumentation from scratch. The general sequence of steps for a typical exploratory multivariate study are set out more fully in Cooksey (2014, Appendix B, pp. 582–583) but are summarised briefly here: – undertake data preparation to appropriately code and move data from raw recorded form to electronic spreadsheet form within your chosen statistical software package; – conduct data screening, assumption checking and missing value analysis to get a ‘feel’ for your variables, check their distributions to see how well they satisfy any required assumptions, and look for any patterns of missing data that need to be addressed; – conduct exploratory factor analysis to condense a large group of variables into a smaller number of ‘components’ or to recover the underlying structure, i.e. ‘factors’, in a set of variables; – establish reliability of scales to demonstrate the internal consistency reliability of each factor identified in the previous step, using Cronbach’s alpha – create scale scores, preferably by averaging (especially if you have missing data) scores on the individual defining items for each scale (a generic term for a component or factor when used as a measurement process), identified in the previous two steps

944

21

How Should I Approach Data Analysis and Display of Results?

– (a) conduct, where appropriate, multivariate tests of hypotheses using any new scale scores in combination with other relevant dependent, independent and covariate variables, conduct relevant multivariate group comparisons, ensuring that you request effect sizes be produced by the statistical software procedure (this step may not be required for simpler investigations yielding just a few key variables); (b) build and test, where appropriate, correlation/regression/general linear models, using any new scale scores in combination with other relevant independent variables (or predictors where categorical predictors have been suitably coded) and dependent variables; such predictive or explanatory models may be constructed using simultaneous or hierarchical variable entry; ensure that you request that appropriate effect sizes be produced by the statistical software procedure; – (a) conduct, where appropriate, univariate tests, conditional on achieving significant multivariate group comparison outcomes, using univariate group comparison tests on the scale scores or other dependent variables and ensuring that you request that appropriate effect sizes be produced by the statistical software procedure; (b) evaluate individual predictors or bivariate relationships, testing for the contributions of individual predictors in a model and computing effect sizes or conducting tests of correlational hypotheses; and – conduct, where appropriate, posthoc tests to isolate group differences, conditional on achieving significant univariate comparison outcomes for independent variables involving three or more groups, using posthoc multiple comparison tests. • Some multivariate research is undertaken where you do have a strong apriori idea of the structure of your questionnaire or other instruments, based on theoretical grounds or prior research. Such research can be deemed confirmatory because your main goal is to verify the theorised or intended structure within a specific research context. Sometimes, confirmatory research may be undertaken to test an instrument’s theoretical structure in different sampling contexts (e.g., cross-cultural settings) in order to test the generality of the constructs embedded in the instrument. Any research that proposes a causal model to be tested, using a structural equation modelling approach, is essentially a confirmatory study. The sequence of general steps for a typical confirmatory model testing multivariate study are also set out more fully in Cooksey (2014, Appendix B, pp. 583–584) but are summarised briefly here: – develop your conceptual/theoretical model, which is often done in diagrammatic form (e.g. a ‘path diagram’) showing all constructs and their hypothesised linkages and you will need to decide whether you will take a covariance-based (e.g., using AMOS or other structural equation modelling software, see, e.g., Byrne, 2016) or variance-based partial least squares analysis approach (e.g., using the SmartPLS software, see Hair, Hult, Ringle, & Sarstedt, 2014);

21.2

Analysing Quantitative Data

945

– undertake data preparation to appropriately code and move data from raw recorded form to electronic spreadsheet form, within your chosen statistical software package; – conduct data screening, assumption checking and missing value analysis to get a ‘feel’ for your variables, check their distributions to see how well they satisfy any required assumptions, and, importantly, look for any patterns of missing data that need to be addressed (structural equation modelling estimation procedures are notoriously sensitive to missing data patterns); – specify your conceptual/theoretical model using the appropriate software interface (e.g. AMOS, SmartPLS), ensuring that all variables, causal and/or covariation linkages and error terms are correctly represented; – test your conceptual/theoretical model using the appropriate software package (e.g. AMOS, SmartPLS) and specify the analysis options you need to carry out the necessary model-fitting estimation processes (request bootstrap errors for all parameter estimates, where possible, as this can offset problems created by non-normal data); – make, where appropriate, any necessary model modifications using evaluation results and diagnostic indicators from the previous step and revise your model specifications; – obtain and prepare data measurements from a new sample for testing your refined model (this avoids capitalising on chance which occurs if the same sample is used to both fit and refine a model); and – confirm your final model refinements using the new sample, i.e. test the refined model using the data from the new sample. [Note that with a large enough original sample, you could accomplish the last two steps by randomly splitting your original sample into two sub-samples of equal size, fit and evaluate the original model using one sub-sample, then test the refined model using the second sub-sample—a process called cross-validation].

21.2.3 What Are Some Useful Tips on Reporting Quantitative Outcomes? Many statistical analyses reported by software packages such as SPSS will produce far more tables, numbers and figures than you will ever want to include in your thesis, dissertation or portfolio. An important part of writing up results from quantitative analyses involves finding the right balance between textual discussion, presentation of tables, and presentation of graphs and figures. The right balance provides the reader with a clear running narrative description and preliminary interpretations of the findings, reporting only those numbers, tables and graphs that are necessary to get your story across. Many postgraduates (and academics as well!) tend to err on the side of reporting overkill, cutting and pasting every table and figure produced by SPSS into the chapter they are writing, which means that the story gets

946

21

How Should I Approach Data Analysis and Display of Results?

lost in the volume of irrelevant detail. Generally speaking, most types of statistical analyses will require, at most, one, perhaps two, summary tables or figures. Many analyses, however, may not need any table or figure to clearly convey your story. For example, you can summarise the findings of a t-test, a simple correlation analysis, or simpler types of multiple regression analysis by selecting and parenthetically reporting just a few key numbers from the computer output as part of your narrative discussion of your analysis. This means that the reader spends less time jumping back and forth between tables, figures and your narrative, allowing reading to flow much easier and making your story more concise and interesting. There is a second important consideration as well. Good ‘statistical’ practice in the social and behavioural sciences now dictates that, wherever possible, each statistical hypothesis test reported needs to be accompanied by a specific measure of effect size (see Cooksey, 2014, Procedure 7.7, for a conceptual review; Rosenthal & Rosnow, 2007 provide one of the more comprehensive discussions of measures of effect size and have tables showing how to compute them from common test statistic values like t and F). This is because it is no longer sufficient to report a statistical test with its associated p-value (which you use to decide on significance). With large enough sample sizes, many quantitative studies have sufficient power to find very small differences and relationships significant. Whether such differences and relationships are practically meaningful and useful is another matter. This is where a measure of effect size comes into play because it quantifies just how large (i.e. practically important) a specific difference or relationship is. Knowing this, you need to plan, up front, to ensure that your computer package (e.g., SPSS) produces these effect size measures, as they are often not produced by default settings. For example, in a multiple regression analysis using SPSS, a global measure of effect size is produced by default and that measure is R-squared. However, if you want to report partial or semi-partial correlations, you need to explicitly instruct SPSS to produce these. In an Analysis of Variance (ANOVA) or Multivariate Analysis of Variance (MANOVA) analysis using SPSS, you need to explicitly ask for the appropriate measure of effect size, namely eta-squared, to be produced. For a t-test, SPSS does compute a measure of effect size. However, Rosenthal and Rosnow (2007, inside back cover) show how with a formula that uses the value of the t-statistic and the value for degrees of freedom, both of which SPSS does produce by default. To achieve the right balance and content in the reporting of quantitative results, you need to remember a few key points and strategies: • Many descriptive analyses (e.g. frequency distributions, Exploratory Data Analysis (EDA) graphs) are never reported other than to generally indicate that they have been examined and informing the reader as to what, if any, decisions might have followed from these preliminary explorations. That is, you use them to familiarise you with your data, to inform any decisions about how well specific variables satisfy assumptions and what you will do about emerging problems (transformation of non-normal variables, missing data patterns, outliers). The first part of your presentation of outcomes from your statistical analyses will typically focus on understanding and telling the story about your

21.2

Analysing Quantitative Data

947

sample and its relevant characteristics. Here, some descriptive statistical results may be displayed, usually in table format. • Every table or figure you choose to include must be discussed as to the meaning (s) it is conveying. Nothing is more disruptive for a reader than to be reading along and suddenly be referred to Table A or Figure B and then when they return to the text, you have simply moved on to the next point you want to make. Instead, make sure you connect the reader with the interpretations you want them to take from the table or figure. Therefore, the choice of tables and figures to include is more strategic than mechanical and you want to make those choices that will help you tell your story in the simplest and smoothest way while maintaining an appropriate level of analytical integrity. • Unless you are reporting on an analysis that produces several related statistical hypothesis tests or relationships all at once (e.g. standard or hierarchical multiple regression with many predictors, correlation matrix, factor analysis, multi-way ANOVA or MANOVA), you can more easily summarise the results of specific statistical tests in your written narrative. For example, a t-test for comparing male and female senior managers in terms of their overall job satisfaction ratings (on a 7-point Likert-type attitude scale) on a questionnaire could be completely summarised in the following single sentence: “An independent-groups t-test showed that male senior managers reported a significantly higher level of overall job satisfaction (mean = 6.25; s.d. = 1.19) compared to female senior managers (mean = 5.48; s.d. = 2.02) (t(46) = +2.21, p = 0.015, r2 = 0.36).” The descriptive statistics (means and standard deviations (s.d.)—always report both) for each group are displayed in parentheses linked, by proximity in the sentence, to their relevant group, and the relevant test statistics are in a parenthetical inclusion at the end of the sentence. The general format and ordering for the parenthetical reporting of statistical outcomes from a hypothesis test is: name of test statistic (e.g. t), the degrees of freedom associated with that statistic set off by their own parentheses (e.g. 46), the p-value (probability) associated with that statistical outcome (e.g., p = 0.015), and finally, the measure of effect size (e.g., r2). One important convention to remember for this reporting style is that the p-value you include should be the exact number produced by SPSS or any other software package you are using. However, if the p-value reported by SPSS as something like 0.000, then you display the p-value in your parenthetical list as p < 0.001 (i.e. less than the smallest number that could have been shown by SPSS). • Tables are generally most useful for displaying multi-group or multi-variable descriptive data (e.g. counts, percentages, group sizes, means and standard deviations for different demographic categories). In most cases, cutting and pasting tables straight from an SPSS output into your Microsoft Word document for your chapter or paper, without any editing, is a poor strategy, simply because most tables will contain more information than you need to tell your story. For example, when SPSS reports a MANOVA, it displays several different significance tests for the same effect (Wilks’ Lambda, Pillai’s Trace, Hotelling’s Trace and Roy’s Largest Root), where only one (Wilks’ Lambda is generally preferred,

948

21

How Should I Approach Data Analysis and Display of Results?

see Cooksey, 2014, p. 348) is necessary to report; the superfluous ones can simply be deleted. If you do want to include a table from an SPSS output, edit it to delete the information that is not essential for your interpretation. You can edit tables in SPSS for this purpose, but this is often clumsy. A better strategy is to either cut and paste the table into your Microsoft Word document as ‘Formatted Text (RTF)’, and then edit the resulting inserted table like you would edit any other table in Microsoft Word. Alternatively, SPSS does have the facility to export all or selected portions of an output file in Microsoft Word or Rich Text Format. • Figures are generally most useful for displaying a specific relationship or set of relationships that you will then discuss (e.g. a graph of an ANOVA interaction, a histogram, a multidimensional scaling coordinate plot, a structural equation model path diagram). You would generally not include graphs you have produced during your initial explorations of your data (i.e. to look at distributions, to check assumptions, etc.) unless they reflect something important that you will be attending to (e.g., showing the plot of a highly positively skewed variable to set the stage for arguing for a logarithmic transformation to normalise that distribution). Nor would you generally include a whole slew of graphs to display the demographic characteristics of your sample, unless you only have one or two demographic variables. If you have many variables to briefly and descriptively summarise, a tabular presentation would work better. • There are three broad classes of graphs you should be aware of (see also Cooksey, 2014; Jacoby, 1997, 1998): – Univariate graphs (e.g. histograms, pie charts, bar charts) summarise the behaviour of a single measure over participants or over a sub-group of participants. (Generally, you would not include many, if any, of these in your thesis, dissertation or portfolio; they are mostly useful for data familiarisation). – Bivariate graphs (e.g. scatterplots) summarise the behaviour of a pair of measures over participants or over a sub-group of participants, depicting especially their correlation and, perhaps, the fit of a regression line (multi-function graphs can combine univariate displays with a bivariate display—the SYSTAT software package is especially good for this). Most scatterplots would not be included in your thesis, dissertation or portfolio as they are most useful for getting familiar with possible relationships amongst variables. However, if a predictive relationship is a specific target of your research question or hypothesis, then such a graph may be useful in helping you to tell the story. – Multivariate graphs (e.g. multi-group line graphs, scatterplot matrices, Chernoff’s faces, multidimensional scaling plots) summarise the behaviour of a number of measures over participants or over a sub-group of participants (the SYSTAT software package is especially good for this as well; SPSS has adequate capabilities for most purposes). Certain types of multivariate graphs may be useful in your thesis, dissertation or portfolio, depending upon your research questions/hypotheses. For example, in any ANOVA or MANOVA design involving two or more independent variables, you would expect a significant interaction to be displayed using a multi-group line graph.

21.2

Analysing Quantitative Data

949

• Graphs can seriously mislead if they are inappropriately drawn, scaled or proportioned. Fortunately, most statistical packages will make sensible scaling choices—just be careful when you edit graphs not to create distortions. Graphs that employ fancy devices such as 3D formatting, shading and colours can often create confusion—Excel is particularly notorious in this respect. Graphs, in black and white or greyscale (a thesis isn’t typically printed in colour, but it can be) work best with white backgrounds and black or darker grey text, symbols and lines. You can preset SPSS options to produce all graphs using a specific colour, line style and symbol marker scheme that you design (in which case that will become the default style for all graphs SPSS produces), or you can choose to only edit the graphs you want. For example, if your graph is produced with lines of several colours, this may not translate well into black and white, if this will be the final format of your thesis, dissertation or portfolio. You need to either force SPSS to produce the graphs using different line styles (solid lines, dashed lines, dotted lines, etc.) or edit the line styles yourself. If your graph is produced using all the same type of symbol (varying only in colour), you will need to edit not only the line and symbol colours but also the symbol shapes and fills (e.g. solid or hollow circles, squares and triangles) to make the trends for different groups or variables distinctive. While it is possible to cut and paste a graph produced by SPSS directly from the software output window into your Microsoft Word document, you may find that SPSS has made choices for the presentation of your graph that you may not want or that you may wish to change. In this case, edit the graph in SPSS using the range of tools that it provides (e.g. to change colours, line styles, background, fonts and font sizes of labels, the scaling of each axis, etc.), then cut and paste it into your Microsoft Word document. Whether or not you edit an SPSS graph or use it as it is, remember to use the ‘Paste Special …’ function in Microsoft Word to paste the graph into your document. Choose the option that lets you paste the graph either as a ‘Picture (Extended Metafile)’ format or as a ‘Bitmap’. (Alternatively, you can have SPSS export the selected chart or charts to a Microsoft Word or Rich Text Format file for you.) When you have done that, you can then scale the picture to the size you want and crop any extra blank spaces if needed. If you realise that you want to change something else after pasting, you cannot edit the graph in Microsoft Word; you must go back to SPSS, re-edit the graph and then repeat the cut-and-paste or export process. • If you are not sure whether you have the balance and content of your quantitative analysis presentation quite right, discuss the possibilities with your supervisor(s). Also, try to find a previous thesis, dissertation or portfolio or published article which may illustrate how to report that same type of analysis (there is no shame and certainly no crime in modelling your presentation style after one that has already been proven!). • Table 21.1 provides some insights into specific stories you might wish to convey about quantitative data analyses in your thesis, dissertation or portfolio. The table briefly describes the type of story you might want to convey, suggests

Some story specifics [potential story content]

Story about who provided the measurements to be analysed, in terms of key demographic characteristics and general patterns of response and non-response [a narrative backed up by relevant numbers and tables and/or graphs as needed] Comments, where appropriate, about representativeness of your sample relative to your intended population—helps to shape the reader’s expectations about the extensional reach of your generalisations [a narrative backed up by key numbers and/or a table] See Fig. 21.13 for an illustration

Story about what preliminary explorations of the data revealed about distributional patterns and meeting of key assumptions (e.g. normality, presence of outliers, etc.) for the planned analyses [generally, a narrative backed up with numbers and perhaps a strategically chosen graph] Story about patterns of non-response and comments on possible biases [a narrative with numbers and perhaps a key table or graph] See Fig. 21.14 for an illustration

Short story about the approach to data condensation/construct validation and the analytical choices made and decision criteria used [a narrative] Story about how measurement items were condensed into smaller number of composites/components/factors including interpretation and clear labelling of each emerging composite/component/factor and incorporating any outcomes of refining analyses [a narrative backed up with numbers and a key table] Story about the reliability of each composite/component/factor and an indication of how participants were given a score on each composite/ component/factor [a narrative backed up with numbers] See Fig. 21.15 for an illustration

Story about how measurements or variables relate to each other where the variables do not necessarily have theorised roles as independent

Type of statistical analysis story

Demographic stories about research participants

Stories summarising preliminary explorations and screening of the data

Where relevant, stories about data condensation and/or construct validation

Stories focusing on patterns and relationships relevant to research questions/hypotheses

(continued)

Correlation analysis (for relating pairs of variables; partial and part correlations allow you to control for a third set of variables)

Principal components or another form of factor analysis (to identify combinations of measures (scales) that measure or reflect the same construct) Reliability analysis (applied to any scales you identify using factor analysis; preferred index to report is Cronbach’s alpha measure of internal consistency reliability) Confirmatory factor analysis (only in situations where you are formally testing for a hypothesised structure in your measurements)

21

Graphs/histograms of distributions (shapes and distortions—normal versus non-normal) Missing Values Analysis (looking for patterns and biases in ‘missingness’) Indications of outliers (exploratory data analysis displays like stem-and-leaf or box plots, general or analysis-specific statistical indicators)

• Frequency tables (counts and percentages) • Graphs of distributions (histograms, pie charts, bar charts, line graphs) • Summary tables of statistics (means and standard deviations)

Statistical evidence you can use

Table 21.1 Finding meaning in quantitative data—fleshing out relevant stories, with illustrations from postgraduate theses

950 How Should I Approach Data Analysis and Display of Results?

Story that summarises and interprets the analyses for targeting specific research questions/hypotheses using relevant descriptive statistics and/ or relational statistics, accompanied by hypothesis test results to signal statistical significance and effect size to indicate practical significance; focus should be on a basic interpretation of what the analysis results meant, what the important patterns were and what they indicate with respect to your research question/hypothesis [a narrative backed up with numbers and, where strategic to do so, a key table or graph or two] See Figs. 21.17, 21.18 and 21.19 for illustrations

Story that summarises and interprets the analyses for targeting specific research questions using relevant descriptive statistics and/or relational statistics, accompanied by hypothesis test results to signal statistical significance and effect sizes to indicate practical significance; focus should be on a basic interpretation of what the analysis results meant, what the important patterns were and why the supplemental analyses were important to conduct [a narrative backed up with numbers and, where strategic to do so, key tables or graphs (perhaps reported in an appendix), but with less emphasis than for the research question/ hypothesis analyses]

Stories emerging from any supplementary analyses

Use any statistical procedure that helps you to anchor the supplemental story

Correlation and regression analyses (to assess simple or complex predictive or explanatory models and relationships) Analysis of variance (to compare group means in an experimental design; use multivariate version if you have several dependent variables) Structural Equation Modelling (to assess simple or complex causal models) Time series analysis (to build series models and tests for trends over time; may focus on testing intervention effects, where appropriate) Multi-level or hierarchical linear model analysis (for assessing models estimated at different nested levels of analysis within a sample, such as for students (level 1) nested in classes (level 2) nested in schools (level 3))

Statistical evidence you can use Canonical correlation analysis/Set correlation analysis (a multivariate analysis to relate sets of variables to each other and, in the case of set correlation, perhaps controlling a third and/or fourth set of variables) Cluster analysis (for discovering and forming a typology or groups of participants or other data sources within your sample using a set of variables, resulting in profiles for the various groups found)

Some story specifics [potential story content]

(i.e., causes) and dependent (i.e., effects) variables [a narrative backed up with correlations (or a matrix of correlations)] Story about discovering previously unknown groups of participants or other data sources using information on a set of measurements and then perhaps relating members in those discovered groups (or clusters) to other variables not used in their formation (what is called cluster validation) [a narrative accompanied by a multiple line graph depicting the profile of average scores on each measurement used to form the clusters as well as statistics comparing the clusters on other variables] See Fig. 21.16 for an illustration

Stories focusing on model building and testing to address research questions/hypotheses

Type of statistical analysis story

Table 21.1 (continued)

21.2 Analysing Quantitative Data 951

Fig. 21.13 Illustrative demographic story

Approximately 89% (420) of the respondents received their last degree in Saudi Arabia, and 10.1% (47) of respondents received their last degree overseas in countries such as Egypt, Australia, US and Canada; 13.7% (64) of respondents were from the Northern Region of KSA, 12.8% (60) were from the Southern Region, 13.3% (62) were from the Eastern Region, 14.8% (69) were from the Western Region, and 45.4% (212) were from the Central Region; approximately 25 % (117) of respondents were relatively New to the Current University, 55% (257) were Experienced at the Current University, and 19% (93) had Longer Career at Current University; approximately 26% (126) were decision-makers (Dean and Managers), 49 % (231) were Administrative Officers and 23% (110) were Academic Staff; and the percentage of those in a Staffing Decision-Maker Role with Respect to Administrative Officers was 10% (50) with an average of 4 to 14 years of Experience with Administrative Officers Staffing.” (Alfawaz, 2015, pp. 164-166)

As explained earlier, respondents were selected from five different regions in KSA. The characteristics of respondents who participated in the survey are shown in Table 6.2: 69% (326) were females and 30% (141) males; 30.8% (144) were unmarried, and 69.2% (323) were married; 34.3% (160) were conservative and 65.7% (307) were moderate; 13.7% (64) had less than a bachelor’s degree, 56.5% (264) had a bachelor's degree and 29.8% (139) had a postgraduate degree.

21

“Of the 900 questionnaires sent out, 574 responses were returned, representing a response rate of 63.77%, of which 498 were fully completed (55.33%) and 467 were from those with a Saudi background (51.89%). Table 6.1 provides details of the response rate statistics.

“This study was undertaken to explore the recruitment and selection (R&S) activities experienced by Saudi female administrative officers in Saudi higher education institutions (HEIs). The major aim of the study was to investigate how administrative staff in the public sector of HEIs in the Kingdom of Saudi Arabia are recruited, and what the selection practices and principles of recruitment processes are, as well as the associated social, economic and global influences on these R&S functions.” (Alfawaz, 2015, Abstract, p. ii)

Alfawaz (2015) PhD thesis: Cross-cultural/Survey research frame; Exploratory Sequential MU configuration (Phase 1 – interpretivist/constructivist guiding assumptions - face-to-face interviews; Phase 2 – positivist guiding assumptions – mail questionnaire)

952 How Should I Approach Data Analysis and Display of Results?

As shown in Table 4.1, female principals had a relatively higher percentage of missing values in the entire indicator variables compared to male principals. In terms of their current post, vice /assistant principals had substantially a higher percentage of missing values than the principals. Missing values in terms of school levels were also substantially higher for secondary level principals than for primary level principals. Data from urban schools revealed a slightly higher percentage of missing values than semi-urban and rural schools. Going by the school system, private school principals had a slightly higher percentage of missing values than the principals from the government schools. In terms of type of school, day-school principals had a relatively higher percentage of missing values than the boarding school principals. So findings related to these support systems items (which are mainly related to the perceptions of principals support provided by Paro workshop and external agents such as colleges of education, education monitoring division, district education office and curriculum department) needs to be cautiously interpreted. (adapted from Sherab, 2013, pp. 73-76)

The missing values analysis also computed cross tabulations of categorical variables versus indicator (missing/non-missing) variables, which helped to determine whether there were differences in missing values distributions across various categories. For the principal questionnaire, items SSGNHE65, SSGNHE66, SSGNHE67, SSGNHE68, SSGNHE69, SSGNHE70, SSGNHE72 and SSGNHE74 formed the indicator variables, based on their percentage of missing cases exceeding 5%. An examination of the cross-tabulation tables for all the categorical variables, the percentage of missing values in the indicator variables did not appear to vary except for the gender, school level, current post, school location and school system (see Table 4.1).

Exploratory t-tests (comparing groups defined by whether or not they were missing a score on a specific item) were computed to identify variables whose pattern of missing values was likely to affect positioning on other quantitative variables. Examination of the separatevariance t-tests indicated that ‘missingness’ associated with the support system items SSGNHE65, SSGNHE66 and SSGNHE67 (each related to the Paro GNH Education workshop) predicted significant differences in the means of several of the principal self-efficacy belief (PSEB) items (PSEB1, PSEB2, PSEB3, PSEB4 & PSEB8) (see Appendix 3.1, part A for the wording of items). The means for the missing data group (SSGNHE65, M = 3.86, SSGNHE66, M = 3.92 and SSGNHE67, M = 3.93 respectively) were significantly lower than the means of these self-efficacy belief items (PSEB1, M = 4.00, PSEB2, M = 3.93, PSEB3, M = 3.89, PSEB4, M = 4.01 & PSEB8, = M = 4.10 respectively). The t-values for these self -efficacy belief items were large (above 2.1 to -3.1) compared to other items. This finding helped to explain why the data were not missing completely at random. On the other hand, the ‘missingness’ of the support system items (SSGNHE68, SSGNHE69, SSGNHE70, SSGNHE72, 75 and SSGNHE74, each related to external support) predicted differences in means for relatively few quantitative variables (ImpGNHE40, ImpGNHE41, ImpGNHE42,ImpGNHE43 & ImpGNHE44) that were related to principal perceptions of importance of GNH Education, signalling that their influence was at least localised.

The high rates of ‘Not Applicable’ responses likely reflect principals’ inability or unwillingness to access these support systems. This observation was an indication that there was some pattern in the missing data and that, at least for these specific items, data were not missing completely at random. The implication of this was that some caution was needed when interpreting the outcomes involving support systems forGNH Education. Further, ‘Not Applicable’ responses were treated as missing data in all subsequent analyses. Respondents who gave such responses, as well as who omitted responses, were automatically deleted (by SPSS) from consideration in analyses as appropriate. The remaining items in the principal questionnaire showed relatively low frequencies of missing data (less than 5% in all cases).

Little’s MCAR test with marginal significance value between .01 and 0.001 indicated that to a certain extent principal data were not missing completely at random (χ² (7493) = 7793.503, p = .008). In order to assess why this might have been the case, inspection of univariate statistics for the principal questionnaire was undertaken and showed that several items targeting Support Systems for GNH Education (SSGNHE) hadthe greatest number of cases with missing values (items SSGNHE65, 74 SSGNHE66, SSGNHE67, SSGNHE68, SSGNHE69, SSGNHE70, SSGNHE72, & SSGNHE74, see Appendix 3.1, Part D). The missing value percentage for these items ranged from five percent to nineteen percent of the principal’s sample. These missing items were all largely related to respondent ratings of ‘Not Applicable’ category in terms of support provided by the Paro GNH Education workshop (items SSGNHE65, SSGNHE66, & SSGNHE67) and other external support (items SSGNHE68, SSGNHE69, SSGNHE70, SSGNHE72, & SSGNHE74) such as support from Curriculum Department, Education Monitoring, Support and Services Division, District Education Office, Principals from other schools and Teacher Education Colleges) in implementing GNH Education.

4.3.1 Missing value analysis for principal data

It was imperative for this research to get a clear understanding of the behaviour of each data set as missing data patterns could potentially have an impact on the quality of data analysis (Cooksey, 2007). Were the data missing completely at random or not for both the principal and teacher questionnaires? The databases were also examined for any anomalous patterns and relationships associated with missing data.

4.3 Missing value analysis

Data from 248 principals and 1649 teachers who responded to the survey from 155 schools were initially entered into the SPSS database . A thorough data screening process was undertaken to confirm that data from the questionnaires had been entered correctly into the SPSS database and to check the distribution characteristics of the questionnaire items. Frequency distributions with the normal curve superimposed and summary skewness and kurtosis statistics were produced for all Likert-type items in each questionnaire to check for any irregularities. These descriptive analyses showed no substantive non -normality in terms of skewness, kurtosis or outliers in the items. However, some typographic errors were sortedout through crosschecking with the original questionnaires. As a part of the preparation for the statistical analysis, a discrete missing value of ‘9’ was assigned to the ‘Not Applicable’ category to distinguish these responses from those coded as SPSS system missing (no response to an item) (i. e.,omitted responses). As a part of data screening process, a total of twenty respondents (four principals and 16 teachers) were deleted from the database as each of these respondents left more than half of their questionnaire incomplete. Following the data screening process, a total of 244 principal and 1633 teacher respondents remained in the final samples for data analysis. Missing value analyses for these sets of data were carried out as presented in the next section.

4.2 Data screening

Analysing Quantitative Data

Fig. 21.14 Illustrative data screening story

“Since 2010, the Bhutanese education system has emphasised values education through its own unique approach known as ‘Educating for Gross National Happiness’ (referred to as GNH Education). This policy outcome emerged from the concern, shared by some Bhutanese leaders and educators, over the apparent deterioration of human values among youth in Bhutan. GNH Education was regarded by the Government of the day as critical for the sustained development of Bhutan. With GNH Education in place, the intention was that GNH values and principles would eventually be deeply embedded in the consciousness of every youth in Bhutan through implementation of an holistic approach to student development led by principals and teachers as key change agents. This study investigated the nature of principals’ and teachers’ self- and collective efficacy beliefs with respect to capabilities for GNH Education. Also of interest were their lived experiences, as they were involved in implementing GNH Education in its early stages and their relationships to different facets of GNH Education school contexts. Knowledge generated from this study was intended to contribute to an understanding of how schools (principals and teachers) have responded to the GNH Education challenge in local and national contexts.” (Sherab, 2013, Abstract, p. iv)

Sherab (2013) PhD thesis: Cross-Cultural/Survey Research/Case Study Research frame; Explanatory MU configuration (Phase 1 – positivist guiding assumptions - quantitative questionnaire; Phase 2 – interpretivist/constructivist guiding assumptions – Four on-site participant observation case studies)

21.2 953

1.000 0.427

Factor 1

Fig. 21.15 Illustrative construct validation story

Factor 1 Factor 2

Factor Correlation Matrix 0.427 1.000

Factor 2

2.069

3.932

Eigenvalues

2

0.84 0.79 0.76

1 0.71 0.70 0.70 0.68 0.67 0.65 0.53 0.51 0.46 0.41 0.45

Factors

LTD3 Internet banking allows me to manage my finances more efficiently LTD2 Internet banking gives me greater control over my finances LTD6 Internet banking is compatible with my lifestyle LTD7 Using internet banking fits well with the way I like to manage my finances LTD4 Internet banking is a convenient way to manage my finances LTD8 Using the internet to conduct banking transactions fits into my working style LTD10 Internet banking is useful for managing my financial resources LTD1 Internet banking makes it easier for me to conduct my banking transactions LTD16 I believe I could communicate to others the consequences of using internet banking LTD17 I would have no difficulty explaining to others why using internet banking may be beneficial LTD9 Internet banking does not require a lot of mental effort LTD12 Prior to the actual adoption, internet banking is available for me to use on a trial basis LTD14 I have a great deal of opportunity to try internet banking before the actual adoption LTD13 I am able to use internet banking to see what it can do for me prior to the actual adoption

Scale items

Number of items 11 3

Table 4.16: Reliability Statistics Sub-scale Perceived usability Perceived trialability (adapted from Adapa, 2010, pp. 145-147 and p. 152)

Variable Technology factors

Cronbach’s alpha 0.81 0.75

Reliability is the correlation of an item, scale, or instrument with a hypothetical one which truly measures what it is supposed to. Cronbach’s alpha is the most common form of internal consistency reliability coefficient. By convention, a lenient cut-off of 0.60 is common in exploratory research; alpha should be at least 0.70 or higher to retain an item in an ‘adequate scale’, and a cut-off of 0.80 is required for a ‘good scale’ (Graham 2006). In the present study, an approach guided by a lenient cut-off of 0.60 is considered to be appropriate. The reliability statistics for the scales identified via principal components analysis are presented in Table 4.16. In all cases, Cronbach’s alpha exceeded at least 0.60. For all subsequent analyses, respondents were given scores on each identified component or scale by averaging the scores on the items that defined that scale.

4.5.6 Reliability Analysis

...

This component relates to the extent to which customers can actually try internet banking or any add on features with regard to internet banking on a limited basis before they actually consume the service. Customers are often comfortable with the trialability nature of internet banking as it can remove any technology apprehensions that persist with the customers. The component was thus labell ed perceived trialability.

Component 2: Perceived Trialability

This component consists of items relating to the consumer’s extent of ease and convenience with internet banking, compatibility with their personal and professional lifestyles and the extent to which they could communicate the benefits of internet banking us age to others. Therefore, the component represents the perceived usability of internet banking.

Component 1: Perceived Usability

The two technology factors components were labelled based on a review of the thematic content of the different defining items.

21

Table 4.11: Summary of Principal Component Analysis for Technology Factors Measure (N = 372)

The principal component analysis method and a promax rotation of the technology determinants constituting fourteen scale item s was conducted on a random sample (N = 372) of Australian internet banking users and is presented in Table 4.11. The KMO measure of sampling adequacy (0.819) was satisfied indicating that the present data were suitable for principal component analysis. Similarly, Bartlett’s test of Sphericity was significant (p < 0.001), indicating sufficient correlations between the variables to proceed with the analysis. A two-component solution provided the clearest extraction, using the Kaiser -Guttman retention criterion of eigenvalues of more than one. The component correlation matrix indicates that the two components emerged from the technology factors to be positively correlated (0.427) at a moderate level and are fairly-well discriminated. Table 4.11 presents the fourteen scale items, pattern loadings and eigenvalues.

4.5.1 Principal Component Analysis Output for the Technology Factors

“The goal of this study was to investigate the factors that influence how consumers continue to use, and how frequently they use, internet banking in Australia. Patterns of continued use and frequency of use of internet banking have been neglected as most of the existing studies focus on either consumer adoption or acceptance of internet banking. However, in comparison to new customer acquisition, measures of continued and frequent use of internet banking are related to a cost-effective marketing strategy aimed at retaining customers. The research in this thesis is a response to a gap in existing literature which requires the application of more integrated theory testing and the identification of factors that influence the continued and frequent use of internet banking in order of importance to consumers.” (Adapa, 2010, Abstract, p. iii)

Adapa (2010) PhD thesis: Explanatory/Survey Research frame; Hierarchical Embedded MU configuration (positivist guiding assumptions; quantitative questionnaire which included several open-ended qualitative items)

954 How Should I Approach Data Analysis and Display of Results?

Fig. 21.16 Illustrative pattern/relationship (i.e., classification) story

Analysing Quantitative Data

(adapted from Kaine, 2008, pp. 132-138)

The classification was conducted using a monothetic divisive clustering algorithm available in CLUSTAN (Wishart 1987) which is specifically designed for use with dichotomous data. The similarity coefficient used was squared Euclidean distance. The algorithm operates by placing all respondents in one segment and then dividing respondents into successively smaller and smaller segments depending on the distribution of their characteristics.

24

The sample of respondents was classified into different farm contexts based on their responses to questions regarding the timing and severity of waterlogging and the suitability of their soil for sub-surface drainage …24. Respondents who had installed sub-surface drainage were excluded from the classification procedure. These respondents were excluded to allow testing of the hypothesis that the frequency of adoption of sub-surface drainage would differ across the different farm contexts. This was tested using the following threestep procedure. First, farm contexts for management of waterlogging were identified using responses from respondents that had not installed sub-surface drainage. Second, respondents that had installed sub-surface drainage were then allocated to one of these farm contexts based on their responses to questions regarding the timing and severity of waterlogging and the suitability of their soil for subsurface drainage prior to the installation of sub-surface drainage. Third, the hypothesis that the frequency of adoption of sub-surface drainage would differ across the different farm contexts was tested by testing for significant differences in the percentage of respondents in each segment that had installed sub-surface drainage. A solution of six clusters was chosen and the formation of these is illustrated in the classification tree in Figure 5.1. Each branch in the tree indicates whether an element is present or not in a farm system. The resulting clusters represent six different farm contexts for management of waterlogging. The profiles of each farm context, in terms of the variables used to form the context, are presented in Table 5.5. Waterlogging should be a serious problem for respondents in contexts one (severe spring and winter waterlogging), two (serious spring and winter waterlogging) and three (moderate spring and winter waterlogging) because they are unable to take full advantage of high rates of pasture growth in spring. Note that waterlogging prevented these respondents from fully utilising their pastures in winter as well as spring. This meant that milk production from the farms in these contexts was substantially affected by waterlogging and suggested that, where feasible, investment in sub-surface drainage may have been worthwhile for most of the respondents in these contexts. Since respondents in context three could graze waterlogged pastures for a few hours each day, on-off grazing over extended periods may be a viable alternative to subsurface drainage for some in this context. Waterlogging should be less of a problem for respondents in contexts four (serious winter waterlogging), five (moderate winter waterlogging) and six (light winter waterlogging). Although waterlogging prevented these respondents from fully utilising pastures in winter, they were able to take advantage of high rates of pasture growth in spring. This means that, although farm output is affected by waterlogging, investment in sub-surface drainage is less likely to be worthwhile for these respondents. These respondents may counter waterlogging by following appropriate grazing management strategies such as on-off grazing. In Table 5.6 the profiles of respondents in each context are reported in terms of timing and severity of waterlogging. The profiles confirm that waterlogging on farms unable to utilise spring pasture (contexts one, two and three) was much more severe than on farms that could (contexts four, five and six). The majority of respondents in these contexts experienced waterlogging during most of spring as well as winter, many on more than half the area of their farms. These respondents were likely to experience waterlogging in most years or every year, and waterlogging tended to occur continuously for a month or more. Respondents in these contexts were also significantly more likely than respondents in contexts four, five and six to experience problems with pasture growth and utilisation in winter because of waterlogging. They were also significantly more likely to report that waterlogging had an unfavourable impact on pasture composition. In contrast, the majority of respondents in contexts four, five and six tended to experience waterlogging on less than half the area of their farms. Also, these respondents were more likely to only experience waterlogging in winter. Waterlogging on these farms generally lasted for less than a month and was unlikely to occur every year.

“The aim in this thesis was to describe a framework for discovering how agricultural innovations contribute to satisfying the needs of primary producers as managers of agricultural enterprises. Meeting this objective required describing a method for properly specifying the population of potential adopters of agricultural innovations. Drawing on consumer behaviour theory and farming systems theory a method was described that was based on the assumption that the adoption of agricultural innovations is a highly involving decision for producers and the hypothesis that the benefits to be had from adopting an agricultural innovation are influenced by particular elements in a farming system that are specific to each innovation. These elements were termed the farm context for an innovation. The method allowed the population of potential adopters to be classified into segments on the basis that producers with different farm contexts obtained different benefits from an agricultural innovation.” (Kaine, 2008, Abstract, p. iv-v)

Kaine (2008) PhD thesis: Case Study/Explanatory Research frame; Exploratory Sequential MU configuration (Phase 1 – interpretivist/constructivist assumptions – unstructured and semi-structured interviews; Phase 2 – positivist guiding assumptions, quantitative mail questionnaire)

21.2 955

Fig. 21.17 Illustrative hypothesis testing story involving experimental design group comparisons

21

Hypothesis 4d: There will be a two-way interaction between complaint handling time and complaint outcome as they influence global SRE. (adapted from Valenzuela, 2007, pp. 135-136)

c) Complaint Handling Time and Complaint Outcome. The multivariate main effect for the interaction between complaint handling time and complaint outcome was significant at p