The Researcher's Toolkit: The Complete Guide to Practitioner Research [2 ed.] 0415215668, 9780415215664

Table of contents :
Cover
Half Title
Title Page
Copyright Page
Table of Contents
List of figures
Preface
Chapter 1: Why research?: What to look out for and what to think of
The purpose of research
The focus of research
To expand knowledge in a discipline or area
To create impact
Traditions and methods
Underlying elements: Ontology, epistemology, methodology
Some examples of traditions: positivism, constructivism, pragmatism
Positivism
Constructivism
Pragmatism
Approaches to collecting data – methods and tools
Quantitative methods
Qualitative methods
Mixing quantitative and qualitative methods
Action research
Ethics of research
Consenting to participate (informed consent)
Voluntary participation
Anonymity
Confidentiality
Potential for harm
Accurate reporting
Summary
References
Chapter 2: Planning the research
Planning your research and the messy research journey
Stages in the process
Things to think about before you start
Resources
Amount of time available
Accessibility of research sample (your participants or respondents)
Framing your questions
Being realistic about what you can do
The hypothesis
Exploring the literature
Developing a strategy
An example research plan
A visual plan – the GANTT chart
Timing and planning
Collecting data
Analysing data
Drawing conclusions
Writing and submitting your report
Methodology
Ethics as part of your planning
Informed consent
Summary
References
Chapter 3: Collecting your data: Literature and other forms of data
The importance of data – finding out what is already out there
Collecting data through a literature review
Types of literature review
Literature review as a list
Literature review as a search
Literature review as a survey or scan
Literature review as a knowledge enhancer
Literature review as a steering instrument
Literature review as a report
Locating the literature
Developing a review strategy: search terms and lists
Selecting appropriate literature and maintaining literature notes
Aggregating literature material
Critically analysing the literature
Being aware of bias
Collecting other research data
Fundamental types – quantitative and qualitative
Which tools or instruments?
Experiment-based research
Survey-based research
Research interviews
Before the interview
Setting up the interview ‘space’
Conducting the interview
Focus group interviews
The focus group checklist
Observation as a data collection tool
Types of data collected: passive and contextualised
A note on using social media as data collection tools
Summary
References
Chapter 4: Analysing your data
Dealing with data
Coding and classifying data
Analysing qualitative data
Content analysis
Concept-driven and data-driven coding
Guides to help you code
Qualitative data analysis software
Narrative analysis
Digital narrative – digital storytelling
Discourse analysis
Critical Discourse Analysis
Grounded theory
Analysing quantitative data
Quantitative data analysis software
Descriptive tools and techniques
Describing data
The mode, the median, and the mean
Standard deviation
Associating data
Formula for correlating data
A note on inferential analysis
Statistical significance
Testing for significance
Presenting your data
Dealing with ever-increasing amounts of data – the role of data visualisation
Summary
References
Chapter 5: Writing it all up
Getting started: preparing your writing
Tips for getting started
An example for organising the write-up
Time management
Formatting and structuring your writing: an example
Preliminary part
Title
Abstract
List of contents
Lists of figures and tables
Preface
Acknowledgements
The main text
Introduction
Literature review
Methods
Results
Analysis and discussion
Conclusions and recommendations
The end matter
Fallacies (mistakes) to avoid when writing
Thinking about your audience when writing
Presenting your work to others
A note on ethics when writing up
Submitting your research report to others
Styling your report
Social media and publication
Storytelling
Visualising your data
Tone and voice
Summary
References
Chapter 6: Research impact
What is impact?
The development of impact
Examples of impact
Academia
Industry
Public sector
Society
Sharing impact (and research) findings with others
Different interpretations of impact
Co-production as impact
Example of co-production across the sectors
Assessing impact
Limitations of assessing impact
Dissemination
A dissemination identifier
Limitations to dissemination
Contextualising research impact (by using Merton’s norms)
Summary
References
Index

Citation preview

THE RESEARCHER’S TOOLKIT

Designed for those undertaking research for the first time, this fully updated edition of The Researcher’s Toolkit is a practical and accessible guide for all those partaking in small-scale research. Jargon-free and assuming no prior knowledge, it covers the entire research process, from defining a research topic or question through to its completion. This second edition has been fully revised by a collaborating team with a wealth of knowledge and practical experience in research project work. Including activity boxes to highlight key concepts and short summary boxes to indicate fundamental elements of various research areas, the chapters cover: ●

●

●

●

●

●

The importance of research and framing your research question and research ethics Practical elements associated with planning and executing your research activity The application of survey-based research methods and the value provided by social media as data collection devices Deploying both quantitative and qualitative tools and techniques to analyse research data Writing up your research work and preparing it for wider access and consumption Examining the effect of your research work through assessing or measuring its impact

The Researcher’s Toolkit is a must-read guide for students and budding researchers as well as educators seeking to explain academic research and writing to their pupils. It will benefit anyone looking to complete a research project whether inside academia or beyond.

David Wilkinson is based in Nexus at the University of Leeds where he runs his consultancy – Research Toolkit. He edited the first edition of this textbook and has been a researcher and evaluator for over twenty years. Dennis Dokter is the Manager of Data & Insights and Smart Cities lead at Nexus, the University of Leeds’ innovation hub and community. He possesses wide-ranging experience in writing research proposals as well as coordinating, managing, and leading them.

THE RESEARCHER’S TOOLKIT The Complete Guide to Practitioner Research Second Edition

David Wilkinson and Dennis Dokter

Designed cover image: © Getty Images Second edition published 2023 by Routledge 4 Park Square, Milton Park, Abingdon, Oxon OX14 4RN and by Routledge 605 Third Avenue, New York, NY 10158 Routledge is an imprint of the Taylor & Francis Group, an informa business © 2023 David Wilkinson and Dennis Dokter The right of David Wilkinson and Dennis Dokter to be identified as authors of this work has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. First edition published by Routledge 2000 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library ISBN: 978-1-032-01809-6 (hbk) ISBN: 978-1-032-01810-2 (pbk) ISBN: 978-1-003-18015-9 (ebk) DOI: 10.4324/9781003180159 Typeset in Mixage by SPi Technologies India Pvt Ltd (Straive)

CONTENTS



vii ix

23 David Wilkinson 3

Collecting your data: literature and other forms of data David Wilkinson

45

4

Analysing your data

82 116

139  161

FIGURES

igure 1.1 F Figure 1.2 Figure 1.3 Figure 2.1 Figure 2.2 Figure 2.3 Figure 3.1 Figure 3.2 Figure 3.3

Pure and applied research comparison 4 Traditions in research 10 Quantitative and qualitative research compared 13 The research project process 25 An example research plan – Gantt chart 37 Example informed consent protocol 42 Examples of open-ended questions 63 Examples of multiple-choice questions 64 Advantages and disadvantages of social media platforms as data collection devices 77 Figure 4.1 Sample categories that may emerge from the data when exploring important memories linked to music 84 Figure 4.2 Analysing content: stages in developing a coding frame 85 Figure 4.3 Charts showing admissions to Paperfield Hospital (year and gender) 95 Figure 4.4 Admissions to Barnswell Hospital (2022 and 2023 compared) 96 Figure 4.5 Age of admissions (female) to Department A in 2022 97 Figure 4.6 Tally chart of female admissions to Department A in 2022 97 Figure 4.7 Grouping of ages – female admissions to Department A in 2022 98 Figure 4.8 Frequency of ages of female admissions during 2022 100 Figure 4.9 Distribution of exam results 101 Figure 4.10 Working out standard deviations 102 Figure 4.11 Distribution of exam results showing standard deviation 103

viii

Figures

Figure 4.12

Positive relationship (correlation) between two data sets 104 Figure 4.13 Negative relationship (correlation) between two data sets 105 Figure 4.14 No relationship (correlation) between two data sets 105 Figure 4.15 An example using Pearson’s Product Moment Correlation Coefficient approach (R) 106 Figure 6.1 Exploring impact and examples 144 igure 6.2 A co-produced view of a research topic in health F 147 Figure 6.3 Dissemination identifier 152

PREFACE

The first edition of this book was written over twenty years ago to support those engaging in research work for the first time. It was developed to help demystify the research project process and make it accessible and (hopefully) enjoyable. A new element of this edition is to assess the value of the content specific to four user-groups or stakeholders who may have a voice in the research work: academics (including undergraduate or college students), industry or business representatives, those who work in the public sector, and society or wider community groups. Throughout the content we make regular reference to these groups and discuss elements of the research process and its application and relevance to them. Our aim is to contextualise the content and make it meaningful, applicable and relevant. Whilst this updated edition recasts and updates the content, it still follows the typical research project structure and timeline. We begin (in Chapter 1) by exploring why research work is undertaken and the purposes it satisfies. This includes an assessment of the defining characteristics of different attitudes towards research work and its associated definitions. From this foundation we then quickly move (Chapter 2) to practical elements associated with planning and executing your research activity. In this section we spend some time on the importance of specifying an appropriate research question or hypothesis and linking this to relevant published material. Developing a research strategy and considering the best or most effective research methods are also covered in this chapter. Following the planning stage, we then move on (Chapter 3) to consider the forms and types of data you might collect or collate as part of your research project work. This section provides an overview of the typical research methods used in small-scale research project work (experimental methods, survey-based approaches, research interviews, focus groups, observation techniques, and the value provided by social media as data

x

Preface

collection devices). Dealing with the data you collect (Chapter 4) is an important part of the research process and will require you to become skilled in data analysis techniques. In this chapter we spend some time working through a few approaches that allow researchers to deal with both quantitative and qualitative tools and techniques. Preparing your research findings for wider consumption (Chapter 5) includes preparing your written work and carefully considering the format and structing of your reporting. In this chapter we spend some time reflecting on the styles of research reporting as relevant for different readership or audience groups. This includes considering the value of tools and techniques such as storytelling and data visualisation. Finally, we explore (Chapter 6) the value of research work by examining material which seeks to assess effect and impact. This includes coverage of the interpretation of impact and how it can be enhanced through mechanisms such as co-production and dissemination. Although the content of the Researcher’s Toolkit has been completely updated for this edition, the driving force remains the same – to support you as a researcher engaged in small-scale project work, and to help you successfully negotiate your own way through the process. David Wilkinson Dennis Dokter January 2023

WHY RESEARCH? What to look out for and what to think of

1

David Wilkinson

IN THIS CHAPTER WE EXPLORE: ●

The purpose of research

●

Traditions and methods

●

Ethics of research

The purpose of research In the social world, research is going on all around us, every day. Most of us have been asked to provide feedback on services we’ve received or answer a survey seeking our views on particular topics. Market researchers ask us about our preferred brand of toothpaste, chocolate, smartphone, or other consumer good. Pollsters ask us about our views on the latest political topics, environmental issues, world events. Software applications (apps) and websites will ask for your feedback or to give a satisfaction score. Academic researchers may seek our views on medicines, public policy issues, and new technological developments. All of this activity is carried out under the banner of ‘research’. In all of these examples research is carried out with a common purpose: To increase knowledge and understanding of what is known or understood and to take action based on that increased knowledge and understanding. Research results may also have indirect effects on our lives. Policymakers in central or local government may, for example, make decisions based on the outcomes of research work they have been involved with or DOI: 10.4324/9781003180159-1

2

Why research?

commissioned. Each year government departments employ universities or research organisations to carry out enquiries on their behalf. Other organisations, such as charities, healthcare organisations, hospitals, community groups, and unions, also employ research organisations to undertake research work on their behalf. Research work is also carried out by, or on behalf of, commercial and industrial organisations to explore the potential value and development of new products or services, to gain customer feedback, and to monitor competitors. Having a research and innovation strategy is a big part of the successes made by many innovative corporations and small and medium-sized enterprises (SMEs). Research activity, therefore, informs policy and action; it can help guide or steer organisational improvement and development and can also lead to new products and services. Throughout this textbook we return to a core set of stakeholders and discuss the relevance of our content to them. We are interested in exploring ‘what this means’ to them and want to further emphasise that research is not only relevant when pursuing a career in academia, but that this skillset is valuable within any career path. Our four stakeholder groups are: Academia, society, industry, and the public sector.

THE PURPOSE OF RESEARCH Academia Engaging in research activity is one of the most important parts of the work of academia (university or higher education). It provides a base from which to develop effective and relevant teaching practice and knowledge creation. It is here that most state-of-the-art knowledge is developed and disseminated.

Society Research into the effectiveness treatments and vaccines helps to protect health and save lives. A powerful example of this were the huge research programmes developed to evaluate the effectiveness of Covid-19 vaccines during the global pandemic in 2020–2022. Research that has a societal impact or purpose provides us with insights on the value of research and how it affects groups within society.

Why research?

Industry Research that reveals changing consumer practices enable industry to refine products and services to better meet their needs. It is at the core of their innovation strategy and provides insights into future endeavours.

Public sector Research for the public sector helps establish policy that is datadriven and evidence-based. This allows for a more transparent and democratic process by which decisions within the public sector are made and evaluated. Transport habit research and the collection and analysis of population growth data for instance allow public sector bodies to make changes to infrastructure and services to support the population.

The focus of research Broadly speaking, research can be categorised as expanding knowledge in a discipline or subject area, and/or to create impact. There is some degree of flexibility and overlap between these two categories. For example, policy research may also contribute to disciplinary knowledge. They represent different points on a continuum rather than being completely separate. Each of these objectives is examined below. However, because this book is aimed at practitioners, the role of research in informing practice is explored in greater detail. In socially driven research work there are other classifiers for research work that are equally valid (Robson and McCartan, 2016). For example (Leavy, 2017, p. 5) articulates that research can be a mixture of exploration, explanation, and description.

To expand knowledge in a discipline or area Research can be seen as enquiry designed to contribute to discipline-based knowledge. Much of what we learn in school, college, or university is derived from some form of research. The social, behavioural, and natural sciences, in particular, are research-based disciplines, but all subjects rely on continuous enquiry and new ideas. For example, some

3

4

Why research?

people may think that history is a given set of facts that do not change, but it is likely that the version of history that we were taught in school or university is quite different from the one which our grandparents were taught. This is partly because history is continually being revised, based on new evidence or on a reconsideration of the existing evidence. Research moves disciplines forward and is central to the life of the university. Research that is primarily aimed at expanding theory and knowledge in a particular discipline is sometimes called ‘pure’ research because it is often without context or direct application (Robson and McCartan, 2016, p. 397). Pure research can be described as a search for knowledge and understanding for its own sake. Historically, most scientific and academic research work would be described as pure research work in that its primary focus was on expanding knowledge and understanding. The alternative to research work that is ‘pure’ is research work that is more immediately transferrable, contextualised, or applied. There are differences between these two classifying types: Academic ‘pure’ research Contextualised ‘applied’ research Seeks to expand “body of knowledge” in Seeks to find solutions to instant ‘reala given subject area. world’ (Robson, 2017) problems and issues. Questions tend to be more conceptual and theory-driven.

Problems tend to be more practical and focused around a given topic or concern.

Findings are generally made public.

Findings are often made public but sometimes are kept private (for example when linked to the development of new products).

Results generally spur ideas and questions for future research. Assessed through peer review by means of academic discipline standards.

Results are generally used internally to make decisions and set up strategy. Assessed by client-organisation and/or industry standards.

Shared primarily through academic writings (doctoral dissertation, thesis, dissertation research, scholarly journals, academic conferences & presentations, academic articles and other publications (e.g., books).

Shared mainly through internal reports to reveal results; may also be shared more widely through professional conferences and industry/trade publications (e.g., articles, case studies, etc.).

Figure 1.1  Pure and applied research comparison

To create impact The measurable effect or impact of research work is central in commercially framed work. Examples might include the attractiveness amongst

Why research?

consumer groups of remote working and the likelihood of purchasing technology solutions to support this. In the academic world, although research may contribute to the knowledge base of a discipline, findings are often accessible and meaningful to only a small group of fellow researchers or academics. Results from such research projects have historically been published in academic journals that were not easily accessible for the non-expert. Greater understanding of a research topic or issue was more important than the tangible (and measurable) effect or impact of the research. However, this is now changing. In most academic research work funded in UK universities there are measures in place to explore effect and impact. This includes how the work is disseminated inside and outside academia. This was not always the case. Historically, research in academia (as well as other areas) was often referred to (unkindly) as ‘research by academics for academics’. Some commentators argued that historically research in the social and behavioural sciences had little influence on the day-to-day lives and practices of most people. They suggested that practitioners (e.g., psychologists, teachers, nurses, business managers) did not read research findings, or that, if they did, they didn’t necessarily act upon them in their work. More recently, things have changed in the perception of the rationale and value of research work. It is now driven by a focus on effect and measurable impact. There are numerous ways to explore value and impact in research work. Some of the key ones have been identified by Mark Reed, a transdisciplinary researcher specialising in the identification and measurement of research impact Reed et al., (2021) identifies that to demonstrate meaningful impact, research should be capable of classification within one or more of the following ten categories: •



• •

•



•



Understanding and awareness – meaning your research helped people understand an issue better than they had before. Attitudinal – your research helped lead to a change in attitudes. Economic – your research contributed to cost savings, or costs avoided; or increases in revenue, profits or funding. Environmental – benefits arising from your research aid genetic diversity, habitat conservation and ecosystems. Health and well-being – your research led to better outcomes for individuals or groups.

5

6

Why research?

•



•



• •

•



Policy – your research contributed to new or amended guidelines or laws. Decision-making – supporting the development of this and other behavioural impacts. Cultural – changes in prevailing values, attitudes and beliefs. Other social impacts – such as access to education or improvement in human rights. Capacity or preparedness – research that helps individuals and groups better cope with changes that might otherwise have a negative impact. (Reed, 2016)

One of the ways in which research can provide impact is by it being accessible and contextualised. When it reaches this level of understanding it can be effectively used to inform policy, decision-making, and practice. Those who fund large-scale research recognise this and now demand a commitment to demonstrating impact in all of the research project work they support. Examples here include requirements issued by the largest academic research funder in the UK (UK Research and Innovation (UKRI)), and one of the largest healthcare research funders (National Institute for Health Research (NIHR)).

UK RESEARCH AND INNOVATION (UKRI) Launched in April 2018, UKRI is a non-departmental public body sponsored by the Department for Business, Energy and Industrial Strategy (BEIS). The organisation brings together the seven disciplinary research councils, Research England, which is responsible for supporting research and knowledge exchange at higher education institutions in England, and the UK’s innovation agency, Innovate UK. They provide funding to researchers, ­businesses, universities, NHS bodies, charities, non-­governmental organisations (NGOs) and other institutions.

The UKRI provide two substantive strands which should be used to demonstrate impact, and both of which could be present in impactful research work. Academic impact is the demonstrable contribution that excellent social and economic research makes in shifting understanding and advancing scientific method, theory, and application across and within disciplines. Economic and societal impact is the demonstrable

Why research?

contribution that excellent social and economic research has on society and the economy, and its benefits to individuals, organisations, or nations (UKRI, 2022). This will include the development of new innovative products that will help a company and its business models, or/and support endusers with the solution/support for societal challenges.

NATIONAL INSTITUTE FOR HEALTH RESEARCH (NIHR) The National Institute for Health Research (NIHR) has a mission to improve the health and wealth of the nation through research. A substantial part of this mission involves funding health, public health, and social care research that leads to improved outcomes for patients and the public and makes the health and social care system more efficient, effective, and safe. They work closely with a range of different organisations and stakeholders from across the healthcare ecosystem to ensure that the research they fund addresses the health and wealth challenges the nation faces.

The NIHR provide a short summary of their focused interest in supporting impactful work as part of their core guidelines to applicants. It articulates that applicants for funding should plan their ‘pathway to impact’ within their funding applications. They usefully define how they interpret impact as the demonstrable contribution that research makes to society and the economy. It should be of benefit to individuals, organisations, and nations. Generating impact from research is highly context-dependent, takes time, involves serendipity, and, often, comprises a series of small incremental changes carried out collaboratively (NIHR, 2022).

Traditions and methods Research work relies on an agreed understanding of how we come to know something is true or valid. In most research work carried out in the social sciences our ‘truth’ is defined by our acceptance of certain traditions in research. When embarking on research work (i.e., at the planning stage) we usually decide on our underpinning tradition as this will then steer us towards (or away from) particular devices or research tools to help us on our research journey.

7

8

Why research?

Underlying elements: Ontology, epistemology, methodology Guba and Lincoln (1994, p. 12) have explored the important characteristics of research traditions (sometimes also called paradigms). They state that all substantive ways of seeing the truth through research rely on three underlying elements: What is the form and nature of reality and what is there that can be known about it? This is often referred to as ‘Ontology’. What is the nature of the relationship between the knower or would-be knower and what can be known? This is often called ‘Epistemology’. How can the inquirer (would-be knower) go about finding out whatever he or she believes can be known? This is often termed the ‘Methodology’ and includes the identification of the research tools or instruments the researcher will use to conduct the work. Any given research tradition is expected to have coherent responses to these questions as a paradigmatic trait (Gallifa, 2018, p. 12).

RESEARCH TRADITION A research tradition or paradigm is a philosophical framework that your research is based on. It sets out a framework of understandings and beliefs and from which the theories and practices of your research project operate.

There are numerous traditions or paradigms through which to frame your research work. Popular ones include positivism, constructivism, pragmatism.

Some examples of traditions: positivism, constructivism, pragmatism Positivism Positivist research is often linked to scientific research and is reliant on quantitative data (usually lots of it). Positivism uses well-established statistical and analytical techniques to interpret research work. For this reason,

Why research?

positivism typically uses quantitative research methods or approaches to collecting and managing data. Within positivist-framed research, measurement and comparisons are common in order to evaluate effect and impact. Positivist researchers are comfortable utilising language that sets out a hypothesis or research question which can then be proved or disproved by the methods applied. Post-positivist researchers make efforts to utilise the core underlying principles of positivism, but are generally less rigid in their use of only quantitative research tools or instruments. However, not all writers on research traditions and methodologies agree with this interpretation of post-positivists. Some believe that positivism/post-positivism only draws upon quantitative approaches and summarise it in this way (Mertens, 2019), whereas others identify that post-positivist approaches have similarities to constructivism (detailed below), in that they are more inclusive of different approaches to determining truth and validity (Marvasti, 2004).

Constructivism Constructivist research traditions are fundamentally based on the belief that reality is socially constructed. A major element of constructivism is the interpretive understanding of meaning (often referred to as hermeneutics). The essence of constructivism identifies that human beings do not find or discover knowledge so much as construct or make it (Schwandt, 2000, p. 197). These constructions, interpretations, and understandings of meaning are carried out by researchers in a number of settings. For example, historians attempt to include contextual and environmental factors to assess and interpret documents written in a particular time period. Constructivist researchers therefore utilise hermeneutics (the interpretation of spoken and written language) as a way to interpret the meaning of something from a certain standpoint or situation (Guba and Lincoln, 1989).

Pragmatism The pragmatist paradigm is underpinned by its acceptance of a mixed-methods approach to research work (Tashakkori and Teddlie, 2010). In essence, the pragmatic tradition is one that is developmental in its embrace and acceptance of newer and more innovative ways of

9

10

Why research?

conducting research work. A pragmatic approach to research work is more practically focused upon the core scope and remit of the research work. As such, pragmatism allows the researchers to choose the methods (or combination of methods) that work best for answering their research questions (Onwuegbuzie and Johnson, 2006).

Approaches to collecting data – methods and tools Methods and tools commonly used within particular paradigms or traditions vary to fit the philosophical underpinning or positioning of each. Some are rigidly framed and applied and tend to be exclusively used by particular traditions; others are more flexibly utilised and are therefore applied across multiple traditions. A useful shorthand for broadly classifying research tools or instruments can be achieved by exploring their central data source. Research tools can therefore be described as either essentially quantitative (numericallybased) or essentially qualitative (non-numeric) in nature. Performing a classification in this way helps us to identify some of the fundamental strengths and limitations of the specific tools and instruments we might use to realise our research work. In order to understand the nature of data collection and analysis two broad categories have been used to describe different approaches. These are: Quantitative and qualitative research. Tradition or paradigm

Quantitative or qualitative

Examples

Positivism.

Mainly quantitative.

Experiments. Quasi-experiments. Tests. Scales.

Constructivism.

Mainly qualitative.

Interviews. Focus groups. Observations. Document reviews. Visual data analysis.

Pragmatism

Mixture of both quantitative and qualitative.

Includes tools or instruments that are quantitative and qualitative. Focus groups. Interviews. Observations. Testing. Scale measurement. Experiments.

Figure 1.2  Traditions in research

Why research?

Quantitative methods Surveys, tests, structured interviews, laboratory experiments, and non-participant observation are usually categorised as quantitative data collection methods. One of the important features of quantitative research is that it is highly structured and produces data which are amenable to statistical analysis. For example, structured questionnaires usually ask respondents to select an appropriate response in order to answer questions – respondents are not usually asked to say anything in their own words. They simply have to agree or disagree with statements the researcher has devised. This approach makes it easier for the researcher to quantify the data and calculate how many people made a particular point.

QUANTITATIVE DATA Quantitative data are those types of data that can usually be reduced to numerical form. The analysis of these data types involves manipulating them in some way and/or applying some form of statistical test.

The results of quantitative research are presented in the form of descriptive or complex statistics, like tests of significance, correlation, regression analysis. As the name suggests, quantitative research is concerned with presenting findings in a numerical form. The values underlying quantitative research include neutrality, objectivity, and the acquisition of a sizeable scope of knowledge (e.g., a statistical overview from a large sample) (Leavy, 2017, p. 9). Some authors articulate that quantitative tools and techniques are of far more importance in terms of presenting truth about a research topic or issue than their qualitative counterparts. The canons of reliability for quantitative research may be simply unworkable for qualitative research (LeCompte and Preissle, 1993). However, the complexities of researched realities have caused some academics to question the rigid absolutes provided through the use of only quantitative approaches to research investigation (Brunsdon, 2016).

11

12

Why research?

Qualitative methods Participant observation, focus groups, unstructured interviews, or life histories are all types of qualitative research methods (Dawson, 2019). The resulting data are often presented in the form of quotations or descriptions, though some basic statistics may also be presented. The development and application of qualitative tools and techniques are a relatively recent phenomena in social research. Up until around the 1960s social sciences researchers modelled themselves on the natural sciences, focusing on the need for objective, quantifiable information. Much of the research in psychology, for example, was based on an experimental design and carried out in laboratories or similar controlled conditions. Another important aspect of psychological research was (and still is) the use of various tests, for example, of intelligence, personality, attitude, and academic achievement. Although sociological research was not usually experimental in character, it used measurement techniques (e.g., pupils’ ability tests) and forms of statistical analyses similar to those used in psychology. In the social sciences, surveys, tests, and observation were commonly accepted as objective methods of producing ‘hard’ data.

QUALITATIVE DATA Qualitative data include observations, interviews, and life history accounts. They enable the voices of those being researched to be heard. Qualitative data are usually analysed by subjecting it to some form of coding process.

This approach to research began to be challenged during the 1960s and 1970s when it was argued that the application of a ‘scientific’ quantitative approach – in the form of surveys and experiments – failed to take into account the differences between people and the objects of the natural sciences. There were concerns that the experimental method, in particular, was so artificial and removed from everyday life that the findings might not be valid; they might not represent accurately what they

Why research?

claim to represent. Quantitative research just seemed to lack imagination (Shipman, 1985). These criticisms of quantitative research have led many researchers to adopt qualitative approaches that are more flexible and adaptable to different research topics or scenarios. For example, a qualitative research strategy, in which participant observation and unstructured interviewing are the data collection methods, allows researchers to get closer to the people they are investigating (Mannay, 2015). Quantitative research

Qualitative research

Quantitative research tools include surveys with closed questions.

Qualitative research tools include surveys with open-ended questions.

Uses categorical questions such as male/female, yes/no, checkbox or multiple choice responses.

Asks respondents to specify views and reactions to questions by providing written or verbal responses.

Examples of specific tools and instruments include: content analysis, frequency counts, inferential analysis tools and techniques.

Examples of specific tools include: focus groups, interviews, observations, video analysis.

Tend to be narrowly focused around specific research parameters or aims.

Tend to be broad in focus, embracing multiple perspectives and accounts.

Heavily reliant on facts and numbers.

Heavily reliant on impressions, opinions, views and perspectives.

Figure 1.3  Quantitative and qualitative research compared

Mixing quantitative and qualitative methods There are numerous examples of how quantitative and qualitative methods have been combined in research projects, though one method is usually dominant. Furthermore, the distinction between quantitative and qualitative approaches can be rather artificial and misleading as quantitative methods, such as surveys, can produce qualitative data if open-ended questions are included. Qualitative data can also be quantified (Brannen, 1995). In the following section we explore a popular approach to research that draws upon a mixture of methods, although there is an emphasis on interpretative qualitative techniques. Action research has its roots in constructivist or pragmatic research traditions (given our overview descriptions above). Developing understanding or context around issues and relating research work to the ‘real world’ would also afford it the label of ‘applied research’.

13

14

Why research?

Action research Action research is about diagnosing a specific problem (e.g., pupil absence) in a specific setting (a school) and attempting to solve it. The ultimate objective is to improve practice in some way. Action research is usually, but not always, collaborative. Teams of researchers and practitioners work together on a project. Alternatively, a project can be undertaken by the practitioners themselves, without any involvement from outside researchers. In action research, practitioners play an active role in designing the project, collecting data, and implementing change. This is quite different from the forms of research described earlier where an outside researcher conducts a study (e.g., in a hospital) and the role of the practitioners is usually to fill out questionnaires or participate in interviews. Action research can take place in a diverse range of settings, e.g., hospitals, companies or schools. Observation and interviews are the two methods of data collection most often associated with action research, though a whole range of other methods, including questionnaires, tests, or documentary evidence, can also be used. The conditions imposed on other forms of research are often relaxed with action research, it interprets the scientific method much more loosely and flexibly. The claim is made that action research is strongly empowering and emancipatory in that it gives practitioners a ‘voice’ (Cohen, Manion, and Morrison, 2017, p. 31). Not surprisingly, action research has been criticised by those who subscribe to a more traditional scientific approach to research. Critics suggest that it is too subjective; it overlooks the need for systematic methods and lacks scientific rigour. Findings are not generalisable; in other words, they do not apply solely to the environment in which the research was carried out. Nevertheless, the champions of action research are encouraged by the thoughts of one of its founders, who indicated that research which produced nothing but books was inadequate; to be effective and meaningful research required action to be taken once its findings were revealed (Lewin, 1968).

Ethics of research If research work is carried out to expand knowledge and understanding and/or provide impact it should be conducted in a transparent and fair way. If it is not, we would question its value or worth (Farrimond, 2012).

Why research?

All commissioned or sponsored research work will document and identify the scope and remit of what is being investigated, examined, or explored. It will also identify how it will collect data, where it will collect it from, and how it will be analysed. This process documentation (sometimes called a research protocol) includes the ethical issues considered by the researcher or research group. Ethical considerations in research are a set of principles that provide clear structure and a ‘moral compass’ for research designs and practices. Within academia and public sector research work there are clearly defined codes of practice and guidance materials articulating how research projects must be conducted and how data must be held and analysed (Brooks, Te Riele, and Maguire, 2014). Research ethics matters because your actions in this area help to ensure research work is accepted and trusted by sponsors, peer groups, and society. Without clear and robust ethical guidance, our trust of research work would erode.

MERTON’S NORMS AND COUNTER-NORMS Robert Merton made his reputation as a key social science researcher and academic. He published prolifically in the 1950s, 1960s, and 1970s on various sociological issues and he brought visibility and legitimacy to the specialty of the social study of science (Cole, 2004). Very early in his academic career, Merton devised a framework of norms for good scientific research work. These ‘norms’ he identified as the core standards of good academic research work (Merton and Sztompka, 1996). They should be in the minds of all researchers seeking to produce good and effective research work. Although devised to support ‘scientific’ research work, they have commonly been applied to all research work as meaningful standards of good academic practice. His four norms are: Communalism (or communism) This identifies that research discoveries should have shared ownership if they are to have the maximum societal effect or impact. This links to notions of open science and discoveries belonging in the public domain. Universalism removes the protective barriers that preclude some from conducting research work. This norm celebrates

15

16

Why research?

the idea that everyone can do science, regardless of traditional limitations such as race, nationality, gender, or any other differences. Within the universalism norm, everyone should be judged equally, on the merits of their research work, rather than their prestige or position. Disinterestedness articulates that research should be pure and free from influences. Merton believed that scientists should work only for the benefit of science. Organised scepticism espouses that research should be robustly and transparently assessed and examined. The acceptance of any research work should be conditional on assessments of its scientific contribution, objectivity, and rigour. Whilst these norms are important yardsticks by which to measure research work, some other authors have presented counter-norms which seek to undermine these worthy standards. In work published in 1974 Ian Mitroff presented his analysis of these counter-norms that are diametrically opposed to Merton’s idealised standards. Mitroff’s counter-norms focused instead on research work that is kept secret, is particular to specific research approaches or ‘ways of doing things’, and is heavily influenced by external factors (including sponsors and ­policy-makers), and dogmatic in that its inflexible to what the truth might be. (Mitroff, 1974)

Of paramount ethical importance in research work is the fair and reasoned treatment of participants in a study. Participants may include, for example, patients, students, service users, clients, or colleagues. Ethical considerations should always be driven by transparency and openness in how we coordinate the research work and deal with the data we collect, especially if it is collected from individuals or participants. Professional research processes that are ethical will ensure scientific integrity, protect human rights and dignity, and encourage meaningful collaboration between science and society.

Why research?

If you are conducting research work within an academic or public sector environment, there are clear guidelines or frameworks directing you to deal with your research subjects and the data collected from them in very specific ways. Generally, you should have consideration of the following standard areas, and this should be clearly communicated as part of your research design to sponsors, academic reviewers, and clients. These should also be communicated to participants (usually as information sheets or guidance notes) as part of the recruitment process to your research project.

Consenting to participate (informed consent) Those participating in your research project should be informed about the purpose and benefits of your study. They should also be made aware of any risks associated with their participation in the research work. Risks are usually associated with healthcare-related research work but also apply to more socially driven work. For example, if participants are exposed to differing viewpoints to their own this may be uncomfortable for some to hear so a robust ethics policy will ensure that you inform them of this possibility. Armed with this information, prior to the research beginning, some participants may decide not to support your work. Whilst this may be disappointing, it shows empathy and consideration of your participant group. It is good ethical practice as a researcher to do this.

Voluntary participation Your participants should not be coerced into participating in your research work. They should participate voluntarily. You should communicate to them that they are free to opt in or out of the study at any point in time. Participants who contribute to your research work voluntarily are more likely to provide authentic and realistic data to support your work as a result.

Anonymity Protecting the identity of participants to your research work should be of central importance. Most research protocols or ethics guidance have clear mechanisms and processes that show how respondent information

17

18

Why research?

is dealt with anonymously. Sometimes projects may need to collect identifying information from participants in order to ensure data are being collected from the correct profile of respondents, but these data are usually removed before reporting and analysis. Generally, personally identifiable data are not collected.

Confidentiality Where identifiable information is collected from participants to a research project, you must provide clear information on how this is dealt with in the research work. Examples of what you might do to ensure confidentiality are actions you take to keep that information hidden from everyone else. You anonymise personally identifiable data so that it can’t be linked to other data by anyone else.

Potential for harm In almost all social science-based research work participants will not usually be exposed to activities or actions that would cause physical, social, or psychological harm. In some circumstances, you may be asking participants to consider views or perspectives that are different from their own. This in itself could be a potential cause of distress or harm for some participants, especially if the viewpoint of others is very different from their own. Examples here might include attitudes and perspectives related to socially divisive or controversial issues. Where there is a potential to cause harm to participants researchers should make clear their strategies for mitigating or removing this harm.

Accurate reporting Professionally produced research work should be your own and clearly acknowledge the work of others i.e., it should not be plagiarised. It should objectively and accurately present research data and make clear the types of data collected and analytical tools or techniques applied.

Why research?

WHEN RESEARCH GOES WRONG Unfortunately, not all research work includes robust mechanisms for ensuring clear ethical processes have been applied. Robson and McCartan (2016, p. 211) have identified ten questionable practices in social research. These are: 1. Involving people without their knowledge or consent. 2. Coercing them to participate. 3. Withholding information about the true nature of the research. 4. Otherwise deceiving the participant. 5. Inducing them to commit acts diminishing their self-esteem. 6. Violating rights of self-determination (e.g., in studies seeking to promote individual change). 7. Exposing participants to physical or mental stress. 8. Invading their privacy. 9. Withholding benefits from some participants (e.g., in comparison groups). 10. Not treating participants fairly, or with consideration, or with respect.

Summary This introductory chapter has explored the role of research in our society. Research can contribute to disciplinary knowledge, inform policy, develop new products and materials, or address specific problems. Some of the limitations of research approaches, traditions, or paradigms have also been discussed. These include the fact that methods of interpretation can have an effect on the perceived value of the research (scientific or positivist approaches may be ‘valued’ more than constructivist or mixed-methods approaches), or policy-makers may ignore information that does not fit their agenda. Debates about the nature of social research have also been covered. We have seen that the scientific approach has historically influenced social

19

20

Why research?

research methods but in recent decades innovative and more qualitative methods have become more popular, particularly in sociology and education. Ethical considerations in research work direct the approach undertaken by researchers and provide important markers to allow sponsors and other research consumers to value results and outputs produced from it. The problems typically encountered by weak commitments to ethical principles have been outlined.

References Brannen, J. (1995) Mixing methods: Qualitative and quantitative research/ edited by Julia Brannen. Aldershot, Brookfield, USA: Avebury. Brooks, R., Te Riele, K. and Maguire, M. (2014) Ethics and education research. Los Angeles: SAGE. Brunsdon, C. (2016) ‘Quantitative methods I: Reproducible research and quantitative geography’, Progress in Human Geography, 40(5), pp. 687–696. Cohen, L., Manion, L. and Morrison, K. (2017) Research methods in education. 8th edn. London: Routledge. Cole, S. (2004) ‘Merton’s contribution to the sociology of science’, Social Studies of Science, 34(6), pp. 829–844. Dawson, C. (2019) Introduction to research methods: A practical guide for anyone undertaking a research project. 5th edn. London: Robinson. Farrimond, H. (2012) Doing ethical research. London: Bloomsbury Academic. Gallifa, J. (2018) ‘Research traditions in social sciences and their methodological rationales’, Aloma: Revista de Psicologia, Ciències de l’Educació i de l’Esport, 36, pp. 9–20. Guba, E. G. and Lincoln, Y. S. (1989) Fourth generation evaluation. Newbury Park, CA: SAGE Publications. Guba, E. G. and Lincoln, Y. S. (1994) ‘Competing paradigms in qualitative research’, in N. K. Denzin and Y. S. Lincoln (eds.) Handbook of qualitative research. Thousand Oaks, CA: SAGE Publications, Inc., pp. 105–117.

Why research?

Leavy, P. (2017) Research design: Quantitative, qualitative, mixed methods, arts-based, and community-based participatory research approaches. New York: The Guilford Press. LeCompte, M. and Preissle, J. (1993) Ethnography and qualitative design in educational research. 2nd/Margaret D. LeCompte, Judith Preissle with Renata Tesch. edn. London: Academic Press Limited. Lewin, K. (1968) Resolving social conflicts. [S.l.]: Harper & Brothers. Mannay, D.(2015) Visual, narrative and creative research methods: Application, reflection and ethics. London: Routledge. Marvasti, A. B. (2004) Qualitative research in sociology: An introduction. London: SAGE. Mertens, D. M. (2019) Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods. 5th edn. Thousand Oaks: SAGE Publications, Inc. Merton, R. K. and Sztompka, P. (1996) On social structure and science. Heritage of Sociology. Chicago: University of Chicago Press. Mitroff, I. I. (1974) ‘Norms and counter-norms in a select group of the apollo moon scientists: A case study of the ambivalence of scientists’, American Sociological Review, 39(4), pp. 579–595. NIHR (2022) Plan Your Pathway to Impact: NIHR. Available at: https:// www.nihr.ac.uk/researchers/apply-for-funding/how-to-apply-forproject-funding/plan-for-impact.htm (Accessed: 29 July 2022). Onwuegbuzie, A. and Johnson, R. (2006) ‘The validity issues in mixed research’, Research in the Schools Mid-South Educational Research Association Research in the Schools, 13, pp. 48–63. Reed, M. S. (2016) The research impact handbook. Huntly, Aberdeenshire: Fast Track Impact. Reed, M. S., Ferré, M., Martin-Ortega, J., Blanche, R., Lawford-Rolfe, R., Dallimer, M. and Holden, J. (2021) ‘Evaluating impact from research: A methodological framework’, Research Policy, 50(4), pp. 104147. Robson, C. and McCartan, K. (2016) Real world research: A resource for users of social research methods in applied settings. 4th edn. Hoboken, NJ: John Wiley & Sons. Schwandt, T. A. (2000) ‘Three epistemological stances for qualitative inquiry: Interpretivism, hermeneutics, and social constructionism’, in N. K. Denzin and Y. S. Lincoln (eds.), Handbook of Qualitative Research. 2nd edn. Thousand Oaks, CA: SAGE Publishing, pp. 189–213.

21

22

Why research?

Shipman, M. D. (1985) Educational research: Principles, policies and practices. London: Falmer. Tashakkori, A. and Teddlie, C. (2010) Sage handbook of mixed methods in social & behavioral research. 2nd edn. Los Angeles: SAGE Publications. UKRI (2022) Defining Impact: UK Research and Innovation. Available at: https://www.ukri.org/councils/esrc/impact-toolkit-for-economicand-social-sciences/defining-impact/ (Accessed: 29 July 2022).

PLANNING THE RESEARCH David Wilkinson

2

IN THIS CHAPTER WE EXPLORE: ●

Planning your research and the messy research journey.

●

Things to think about before you start.

●

Framing your questions.

●

Being realistic about what you can do.

●

The hypothesis.

●

Exploring the literature.

●

Developing a strategy.

●

Timing and planning.

●

Methodology.

●

Ethics as part of your planning.

Planning your research and the messy research journey Most research begins with the development of a research question, topic, or theme. In some cases, such as a personal piece of work, you may decide on your own questions or theme; in others, they may be given to

DOI: 10.4324/9781003180159-2

24

Planning the research

you. A great deal of research project work is now commissioned by others to support organisational development, inform policy, and develop new services, for example, with explicit aims and outcomes. Whoever decides on the topic or theme of the research, there are a number of fundamental clarifying questions that can be applied to it in order to help steer and direct its direction and focus. These include: What is it that we wish to learn more about? Why is it of interest or importance to research this area? What other work has been carried out in this area? This clarifying process should enable you to develop, refine, and frame the questions you’d like to set out to answer in your research. Initially you will, no doubt, have generated many questions. Some of these will be related, so you should aim to limit the questions tackled in your research to a few which are clearly formulated and distinctive. Many research projects fail due to the sheer number of questions posed and the lack of clarity they display.

Stages in the process Research in the social sciences typically has a number of fundamental stages. These include framing your questions, exploring the literature, developing a strategy, collecting data, analysing data, and writing/ submitting your report. This might suggest that research is a neat, linear undertaking but in real-world research, this is rarely the case (Robson and McCartan, 2016). However, most research project work cycles around these fundamental stages (VanderStoep and Johnson, 2009), and is iterative, reflective, and developmental. Setting your research topic or question can be time-consuming and requires careful thought in order to frame it appropriately around a given focus or topic (Denscombe, 2017). Planning research work can sometimes feel overwhelming when you have limited experience or are unsure about the process. It can be helpful in these situations to draw strength from your personal characteristics in order to make the experience as pain-free as possible.

Planning the research

Figure 2.1  The research project process

WHAT KIND OF RESEARCHER ARE YOU? Factors you might think about to help you plan and structure your research include: ●

Are you good with people?

●

Do you prefer written communication or face-to-face interaction?

●

Do you love or loath mathematics and statistics?

●

Do people feel at ease with you and are they willing to confide in you?

●

Do you like to number-crunch?

●

Do you like to conduct web-based research work? (Dawson, 2019, p. 3)

In a real-world setting the stages of research are not distinct and as separate as they might at first appear; they are flexible and organic in their form, adapting to the research work as it continues and develops. As an example, your research project may have reached the data collection stage and you may become aware of other published work that raises important

25

26

Planning the research

additional questions you might like to ask your research subjects about. In a real-world setting you might be required to edit your interview or survey questions to include the newly published work you have discovered. Similarly, you may start to analyse some elements of the data you have collected (for example, reading through your interview transcripts) whilst still collecting other forms of data (such as observations of participants in their work setting). If this is the case (and it happens to the best of us), you may need to revise some aspect of your plan. The researcher often has to move backwards and forwards between different stages in their work. Social research is, by its very nature, a messy process, something that many research textbooks disguise from their readers. However, some well-respected social science researchers do acknowledge that social research is often a lot less smooth than the accounts of research you may read about in textbooks (Bryman, 2016, p. 14).

ACADEMIC Students are required (on most college and degree programmes) to carry out a piece of original research work as part of their programme of study. Such work would need to ensure that it met the standard required.

Industry A small or medium-sized enterprise (SME) might need to carry out research work to explore the appetite for a new product or service. This would require planning in terms of (for example) which markets/sectors to concentrate on in terms of survey or interview work.

Public sector A local authority may be charged with providing adequate levels of service in relation to refuse collection. They may need to plan for research work examining the best collection times and days for certain postcode areas.

Planning the research

Society Charities or third sector organisations might be interested in the rights afforded to certain groups represented by them. They may need to plan for effective research work to establish how to service these groups.

Things to think about before you start There are a number of factors that shape the way in which a project is carried out, especially the availability and amount of time, money and other resources (for example, software and other appliances, and instruments or tools you might use to help shape and report on your research work) (Robson, 2017, pp. 150–157). Accessibility of the research subjects or participants, and ethical issues also need to be kept in mind.

Resources One of the key resource elements in research work is the amount of funding resource (money) that is available to undertake the work. Very often in small-scale research work, you will be acting as a lone researcher and will be required to undertake all elements of the work yourself. But this is not without a financial cost; your time and experience are of some financial value, and it is useful (for planning and assignment purposes) to think of it in this way. Research projects that are based primarily on observation and face-to-face interviews are labour-intensive, so the ‘cost’ in terms of researcher time will be the greatest. If the project data are being collected using qualitative methods, such as open-ended interviews, then the analysis can also be very time-consuming as interviews may need to be transcribed and transcriptions analysed and coded in some way as part of the reporting process. Some of the main costs (in terms of time and effort) in survey research include: piloting the draft questions; constructing the survey instrument

27

28

Planning the research

(usually an online hosted survey), coding the responses, and analysing the data. There are many software packages (such as SPSS (Statistical Package for the Social Sciences) and Nvivo for dealing with interview, observation and other qualitative data). Using analysis packages such as these can drastically reduce the time you might need to dedicate to collating, coding, and analysing your research data.

NEGOTIATING ACCESS CHECKLIST – EXAMPLE FROM EDUCATION Gaining access within school or other education settings requires thought and consideration of a number of issues. Cohen, Manion, and Morrison (2017) identify ten key elements that are of value: 1. Formally request permission to carry out your research work as soon as you have an agreed project outline. 2. Speak to the people who will be asked to participate in your work. 3. Submit the project outline to the headteacher, if you are carrying out a study in your or another educational institution. 4. Decide what you mean by anonymity and confidentiality and communicate this to your research participants. 5. Decide whether participants will receive a copy of the report and/or see drafts or interview transcripts. 6. Inform participants what is to be done with the information they provide. 7. Prepare an outline of intentions and conditions under which the study will be carried out to hand to the participants. 8. Be honest about the purpose of the study and about the conditions of the research. 9. Remember that people who agree to help are doing you a favour. 10. Never assume ‘it will be all right’. Negotiating access is an important stage in your investigation. (adapted from Cohen, Manion, and Morrison, 2017, p. 57)

Planning the research

Amount of time available The timeframe in which you need to complete the research is central to your planning. Setting realistic goals is important. One trap that researchers can fall into is that whilst they allow sufficient time to carry out the research and to analyse the data, they do not allocate enough time for writing up the final report. Report writing is where you bring all of the various elements of your project together into a coherent piece of work. It is where you provide clear answers or responses to your research questions. You need to allow yourself time and space in order to reflect upon the data you have collected, literature consulted, and the feedback received from peers/colleagues or line managers. Managing and committing time to the stages of your project (during your planning stage) will ensure you are more likely to produce good-quality work, on time and within the budget or resourcing envelope.

Accessibility of research sample (your participants or respondents) Some groups are more accessible than others. Teachers, university students and school pupils are among the easier targets, whereas the homeless, those with drug dependencies or the super-rich may be more difficult to access. In some cases you may need to negotiate access with a ‘gatekeeper’ before you are able to reach the people you would like to reach. For example, if you want to carry out research on hospital patients, you will probably have to get the approval of the hospital management. Similarly, it will be impossible to interview school pupils without first obtaining permission from the headteacher, and also from the children’s parents. Therefore, in any one project you may have to negotiate several access ‘hurdles’ before you finally reach your respondent group. Guaranteeing confidentiality, arranging visits well in advance, and impressing upon people the value of the research are all useful tactics in negotiating access (Clark, 2011).

RESEARCH ‘GATEKEEPERS’ Gatekeepers are key or influential figures associated with the community or participant group you wish to work with or

29

30

Planning the research

research. These individuals can help you to better understand the community or participant group and can help you get to know other people. Working with them, you will be able to build trust and generate acceptance within their community. (Dawson, 2019, p. 106)

Negotiating or managing successful access to your research subjects or participants is fundamental to all research projects. Without their support your research project will undoubtably stumble, if not fail. A typical, negotiated access approach undertaken by researchers seeking to collect interview data from a respondent group might include: 1. Introducing the research work to the participant group – through an email, letter, tweet or other social media release sent directly to participants or via a gatekeeper (someone who acts as a link person between you and the participants). 2. Negotiating access to the participant group – achieved by asking those interested to come forward and discuss the specifics of their involvement (time for interviews, detailed purposes of the research, etc.). 3. Securing consent from participants – achieved through discussing the need and use of consent forms with participants. 4. Interacting with participants – speaking with participants and securing their responses to your questions. Part of this negotiation stage would also include summarising data and reporting back to participants (for sense-checking and accuracy) what they have said. (adapted from Fobosi, 2019, p. 507)

STAGES IN A RESEARCH PROJECT Framing your questions Before you start you need a subject. If you work as a researcher or have been asked to carry out a specific project as part of your job, then your focus or topics may already have been

Planning the research

selected for you. This stage can be one of the most challenging parts of the research process, as it involves translating the overall theme of the research into a viable research question or hypothesis.

Exploring the literature Acknowledging what has been published is an important part of all research projects. In this stage you establish and identify what has been produced and published elsewhere and how this influences your own research work.

Develop a strategy This development stage is where you identify the tradition or paradigm you will draw upon to support delivery of your research project. This is the stage where decisions are made regarding which methods of data collection and analysis are to be used. You will also identify the types of participants or stakeholders involved in your research work.

Collecting data This is the stage in which you carry out the interviews, send out your surveys, and so on (these are your primary data sources). You may also utilise data collected elsewhere (such as census data or market research agency data). These are often referred to as secondary data sources and may require specific permissions to be sought from the data holders prior to using them for your research work.

Data analysis The data analysis stage is where you bring together the various strands of your research project work in order to develop a narrative within your write-up stage. Data analysis is where you would code your data and identify its key themes or topics. This stage usually takes place towards the project when most or all of the data have been collected.

31

32

Planning the research

Writing up the results The stage in the project where you bring together your research data and present it in an accessible form for your audience. It is a good idea to have the framework for your report or write-up developed during your planning stage.

Framing your questions All research work is based around a topic or theme which is then narrowed down to a specific research topic or question. These elements of your research project are usually detailed in the introductory section of your write-up or reporting. They are where you set the scene for your research work by determining its scope and remit. Introducing the research and setting it in context is an important part of a research proposal and any subsequent write-up or reporting. This is where you would define what you wanted to find out by setting your research in context and linking it to other related work. Setting the context of your own research includes identifying data/information, published materials that have acted as guides to the development of your research topic or questions. Your introduction should also seek to establish the need for your research work. This could include, for example, a need driven by the limited previous work that has been undertaken in this area. A need could also reasonably be defined if your work was developmental for practitioners who could apply the results of your work in their own practice. Framing your questions allows you to be specific about what your research will cover and want it will not. It provides markers or areas of interest so that you can explore and follow these up, in-depth, in relevant literature sources. The linear process for developing research questions is usually: 1. Establish your research topic or theme so that you can focus on the body of literature you need to include and review in your work. 2. Carry out an extensive review of relevant literature so that you are in a position to form appropriate, researchable questions. (O’Leary, 2014, p. 35)

Planning the research

However, this neat and logical interpretation presents an issue around practicality. Which do you focus on first – your topic or question, or the literature? You may find that, following an exhaustive review of the literature, your research question has already been answered by others. Equally, you may become aware of issues, in this particular research context, linked to the use of particular research and analytical tools or instruments. These types of dilemmas emerge in many more research projects than are probably reported. As a professional researcher you may, through your own iterative and developmental research journey, need to rationalise changes or edits to research question, and adaptations to methods, data sources and forms of reporting. Effective research activity should work through these stages of organic developmental planning and refinement. Your goal should be to articulate in your reporting or write up the decisions you have made (to edit or adapt your research questions, or to abandon the use of specific data collection approaches, for example), and to provide robust arguments for those decisions.

THE PURPOSE OF RESEARCH QUESTIONS Research questions should help you to define the limits of your study. ●

Research questions help you to clarify your research study.

●

Research questions allow you to develop knowledge and understanding based on practical experience rather than theory.

When defining research questions care must be taken to develop them objectively. As professional researchers we must always make clear the theoretical perspectives or models we are employing, and the related strategies we are about to employ. We should similarly provide clear reasons for collecting data and presenting information in the way that we do. This makes clear, to anyone who reads our work, our philosophical positions and identifies any biases that might exist (Creswell and Creswell, 2018).

Being realistic about what you can do It is worth spending some time thinking carefully about the research question you will seek to address in your work. The questions below might help and guide you:

33

34

Planning the research

●

●

●

● ● ●

Will the research question be of interest to me and be able to sustain my interest for the duration of the project work? Am I able to answer the question in an objective way, or are there any biases I have towards the research work that I might need to address? Will the research work make a valuable and meaningful contribution to the work that is already out there? In terms of my question, is it clearly and unambiguously set? Is the question ‘do-able’? Can I complete it within the timeframe available? Do I have the requisite skills needed to complete the research work? Will I need to be able to use specific data collection tools or analytical techniques?

The hypothesis Research topics or themes are usually generally set. Research questions are more specific, and a research hypothesis is a refinement of a research question. In scientific research work (belonging to the positivist tradition or paradigm) the hypothesis is a central part of the research process. For those who operate within other research traditions, such as constructivists or pragmatists, there is a common view that research which focuses on the research hypothesis loses all of its colour and value. Research terms such as ‘hypothesis’ are usually linked to scientific, quantitative, research projects rather than qualitative ones (Creswell and Creswell, 2018). The following definition aligns with this view: A research hypothesis is a specific, clear and testable proposition or predictive statement about the possible outcome of a scientific research study based on a particular property of a population, such as presumed differences between groups on a particular variable or relationships between variables. Specifying the research hypotheses is one of the most important steps in planning a scientific quantitative research study. (Kalaian and Kasim, 2008)

Exploring the literature In many proposals, the research plan will make reference to key literature to emphasise points and provide authority to the work being undertaken. Therefore, early consultation of the literature in a research project is important.

Planning the research

The next chapter details the processes involved in comprehensively reviewing the literature and this should be carefully timetabled into the project. A quick literature search or scan through relevant journal abstracts in the early stages of the research should, however, provide assistance in establishing the key concerns or issues arising in your subject area. Following a comprehensive literature review, you may find that your original research questions are no longer appropriate or require reshaping (for example, you may discover that very similar work has been carried out elsewhere). In this case you would need to refine your questions to concentrate on an area not explored fully in the other work or concentrate on questions which add to the research already conducted. This re-forming of the research is quite common, and you should not feel obliged to stick rigidly to your original questions. Many substantial research projects change their focus once it has emerged that similar work has either been carried out or is being carried out elsewhere.

Developing a strategy Once you have framed (and perhaps reshaped) your questions, how will you actually go about answering them? You will need to develop a strategy for your research. Your research strategy should focus on the specified question or questions and explore the most effective and efficient ways of answering them. For example, your strategy should detail which research instruments you will use and how you will collect the data (through documentary analysis, via telephone/face-to-face interviews, through questionnaires, by using case studies, etc.). The strategy forms a major part of the research, and it is useful to develop a visual plan as part of the strategy indicating key milestones in the research. We’ve produced a simple example of a research plan below.

An example research plan

Week 1 to 2 ●

Develop questions

●

Consult colleagues

●

Refer to key journals

●

Review current research in the area

●

Write proposal

35

36

Planning the research

Week 3 to 10 ●

Develop strategy for research

●

Design instruments

●

Pilot instruments

●

Refine instruments

●

Select sample group

●

Administer instruments

Week 10 to 13 ●

Collate data

●

Analyse data

●

Submit draft report to colleagues for comment

Week 14 ●

Submit report

A visual plan – the GANTT chart Whilst the outline plan above shows some of the component parts of the research and provides a broad indication of when they will occur, it is linear and does not take into account the inter-related nature of some elements of the work. Gantt charts do just that. They are a visual representation of a project, with clear indications of when each activity will take place and how it is linked, or related to, other activities. A well-prepared Gantt chart, in a research plan, shows that you have dedicated some time and effort to thinking in detail about your research work. They show your logic and timetable for your research work and, as such, are very useful for sharing with those involved with the research work, such as sponsors, line managers, peers, and supervisors.

GANTT CHART A Gantt chart is a visual chart that illustrates a project schedule. This chart lists the tasks to be performed on the vertical axis, and time intervals on the horizontal axis. Typically, the width of the horizontal bars in the graph shows the duration of each activity.

Planning the research

Develop questions Consult colleagues Refer to key journals Review current research Write proposal Develop strategy for research Design instruments Pilot instruments Refine instruments Select sample group Administer instruments Collate data Analyse data Submit draft report to colleagues Submit report

W 1

W 2

W 3

W 4

W 5

W 6

W 7

W 8

W 9

Figure 2.2  An example research plan – Gantt chart

Of course, even the most carefully organised research may not go according to plan. For example, it might prove difficult to interview teachers during the summer holiday when the school is closed! Therefore, you need to build into your plan some ‘leeway’ and you may need to change the order of things slightly. Attempt to view your plan as a template: all the necessary ingredients for the research should be there, but the ordering of them may change. This shouldn’t alter greatly the eventual ‘dish’ that you serve.

Timing and planning Collecting data Collecting data very often takes longer than researchers anticipate. Arranging interview venues, conducting interviews, distributing online surveys, monitoring returns, and issuing reminders all take time and effort. It is easy to fall into the trap of thinking data collection will be easy and problem-free. You should carefully plan and manage your research time, unmanaged research time has a funny way of slipping away from you (O’Leary, 2014, p. 22). Although it is important not to underestimate the length of time it will take to collect research data; you should also not overestimate the

37

W 10

W 11

W 12

W 13

W 14

38

Planning the research

amount of data you are likely to be able to collect. Many research projects have spectacularly unravelled when the ambitiously planned-for survey response rates have not been realised. In our experience, response rates of between 30% and 60% are reasonable. These rates would apply to work that has succinct and well-formed questions, requires limited time and effort from the respondent group, and is of interest to your participant or respondent group. You should not be too disheartened if, after all your work, you only receive 30 responses out of 100 surveys distributed (that’s actually a reasonable response rate). Rather than being disheartened, however, it might be useful to anticipate in your planning stage that this might occur. You could then aim to distribute many more surveys (subject to resource considerations such as your time in circulating to a larger respondent group) than you expect to be returned completed.

SURVEY RESPONSE RATE – WHAT IS IT? A survey response rate relies on two pieces of information in order to provide you with a response rate. You need to divide the total number of survey responses received by people you have sent the survey to. So, for example 40 people responding to a survey that was initially sent out to 250 would equal a response rate of 16% (that is 40/250 = 0.16).

Analysing data The ease of analysing your data will depend on how well structured your instruments for collecting the data are. One of the most common instruments used by less experienced researchers is the research survey or questionnaire. Many researchers are quick to use this instrument as they often view it as an inexpensive and easy way to collect large amounts of research data. In many cases this is true, but the structuring, planning and layout of a survey all require careful consideration. Doing this well requires thought and effort and is often a time-consuming process. Whilst there are numerous support tools and software packages designed to assist with the analysis of your research data, you will need to carefully consider the analysis frameworks you deploy to theorise and make meaning from your results (O’Leary, 2014). The process of analysing data can also take time; it may even produce results you did not expect to find. Again, be

Planning the research

prepared for this and apportion time to consider the implications of the data being different to how you expected them to be. Can you explain this? Does it necessitate further analysis or data collection?

Drawing conclusions Drawing conclusions from your data is often the most difficult part of a research project. You may have considered your conclusions when designing or framing your research questions. Once you have collected your data you must ask yourself how the data answers your original questions. Does it provide evidence (in your findings) upon which to make conclusions? Do you consider alternative explanations for your conclusions? In other words, is your research topic subject to other factors perhaps not considered in your work? It is not a major failing if you indicate that other work or external factors beyond the remit of your research affect your conclusions. However, it would be a failing if you didn’t mention them. In addition, do you indicate the strengths and weaknesses of your research (or methodological) approach? These are the types of question you should seek to address in the conclusions of your research report. They show that you have evaluated the approach you have taken in the work.

Writing and submitting your report In PhD research, which takes an average period of three years of full-time study, it is usual for six months to be given to the write-up of the work. This is often in addition to notes and draft chapters written throughout the period of study. Many of those new to research don’t allow enough time for writing. The process often involves drafting and re-drafting. In, say a ten-week project you should aim to leave perhaps two weeks for writing the report. Chapter 5 provides further guidance on the research write-up process, including how to prepare, draft, and submit your work.

Methodology At the planning stage of your research work you will need to make clear decisions about the approach to research you are going to draw upon. Which tools or instruments will you use? Which theories or design

39

40

Planning the research

elements will you base your work on in order to carry out the research project work? In Chapter 1 we briefly covered some of the research traditions typically used in social and behavioural sciences research work. As part of your planning, you should consider which of the traditions or paradigms will inform or influence your research work and you should consider the implications this will have on planning the research. In choosing your methods it is useful to consider a number of core factors:

●

●

●

Is it a suitable approach? For example, will it produce the kinds of data that are required? Is it feasible? For example, can it be done in the time available and with the resources at hand? What are the ethical implications? For example, will the approach chosen allow you as a researcher to be honest and ethical in your dealings with participants? (Denscombe, 2017, p. 6)

For many researchers (particularly those engaged in socially-framed research work), core methodological considerations usually reduce down to utilising and deploying quantitative, qualitative, or a mixed-methods approach. Your decisions related to tools, techniques, and traditions will all impact upon the planning requirements for your research work. Open, organic, and iterative approaches, favoured more by constructivist or pragmatist philosophies, may require more flexible and reactive plans than a research project which draws upon positivist traditions or approaches. Qualitative and quantitative data rely on different approaches to their collection and analysis. They are different in terms of the way they can be reduced or analysed, and the way they can be presented or reported. Some funders often place greater emphasis and weight on one data type over the other. For example, in scientific work (drawing upon positivist research traditions), data sources are usually quantitative when presenting messages about effect and impact. In social science research funders are often interested in a mixture of both quantitative and qualitative data types to provide evidence of effect or impact.

Planning the research

Ethics as part of your planning All professional researchers must follow ethical principles, particularly when research work involves collecting or harvesting data from participants or organisations. Your research plan should clearly outline the processes and procedures you will adopt to ensure any data you collect is stored safely, securely and protects the identify of those who provided it (Mager and Galandini, 2020). As researchers we generally recognise that, when working with human subjects, certain steps must be taken to protect the dignity and safety of the research participants (Marvasti, 2004, p. 134). There are numerous set frameworks or protocols you can refer to as you develop the ethical framework for your research work. One such framework is that of UK Research and Innovation (UKRI) (a nondepartmental public body of the government of the United Kingdom that directs research and innovation funding). This ethics framework helps researchers to consider ethics issues during the complete lifecycle of a project and includes information and guidelines on good research conduct and governance. The framework covers six substantive areas or principles: ●

● ●

● ● ●

research should aim to maximise benefit for individuals and society and minimise risk and harm the rights and dignity of individuals and groups should be respected wherever possible, participation should be voluntary and appropriately informed research should be conducted with integrity and transparency lines of responsibility and accountability should be clearly defined independence of research should be maintained and where conflicts of interest cannot be avoided they should be made explicit. (UKRI, 2022; UKRIO, 2021)

Informed consent Appropriately and adequately informing participants or research subjects about the scope and remit of research work forms part of informed consent. This is the process you as a researcher would follow to enable research subjects to consent to participate. Informed consent is a fundamental element

41

42

Planning the research

Add your initials next to the statement if you agree I confirm that I have read and understand the information sheet dated 12/08/2023 explaining the above research project and I have had the opportunity to ask questions about the project. I understand that my participation is voluntary and that I am free to withdraw at any time without giving any reason and without there being any negative consequences. In addition, should I not wish to answer any particular question or questions, I am free to decline. I give permission for members of the research team to have access to my anonymised responses. I understand that my name will not be linked with the research materials, and I will not be identified or identifiable in the report or reports that result from the research. I understand that my responses will be kept strictly confidential. I agree for the data collected from me to be stored and used in relevant future research [in an anonymised form]. I understand that other genuine researchers may use my words in publications, reports, web pages, and other research outputs, only if they agree to preserve the confidentiality of the information as requested in this form. I agree to take part in the above research project and will inform the lead researcher should my contact details change. I confirm that I have read and understand the information sheet dated 12/08/2023 explaining the above research project and I have had the opportunity to ask questions about the project. Name of participant Participant’s signature Date Name of lead researcher Signature Date* *To be signed and dated in the presence of the participant.

Figure 2.3  Example informed consent protocol

of ethical frameworks as this ensures participants give information and data freely and are fully aware of how you will use and analyse and present their data. Often, information is provided to participants at the beginning of the research process. This usually details the research project, and why participant data are sought. The information sheet is usually accompanied by an ‘informed consent protocol’ which seeks a signature from the participant indicating they understand the process and agree to their data being used. A typical ‘informed consent protocol’ might look like Figure 2.3.

Summary This chapter has introduced the important role that effective project planning has on the research process. Initial considerations – such as resourcing, time available and access to your participant or respondent

Planning the research

group – have also been discussed. Recognising the difficulties of gaining access to research subjects or settings should also be considered in the planning stage of any research project. Of central importance is the development of a clear research topic or theme through which to set realistic research questions. Guidance has been provided in this chapter to help you to determine workable and engaging questions to explore as you progress on your research journey. Planning tools and techniques, such as the project plan and Gantt chart, have been briefly covered and their value specified.

References Bryman, A. (2016) Social research methods. 5th edn. Oxford, New York: Oxford University Press. Clark, T. (2011) ‘Gaining and maintaining access: Exploring the mechanisms that support and challenge the relationship between gatekeepers and researchers’, Qualitative Social Work, 10(4), pp. 485–502. Cohen, L., Manion, L. and Morrison, K. (2017) Research methods in education. 8th edn. London: Routledge. Creswell, J. W. and Creswell, J. D. (2018) Research design: Qualitative, quantitative, and mixed method approaches. 5th edn. Los Angeles: SAGE. Dawson, C. (2019) Introduction to research methods: A practical guide for anyone undertaking a research project. 5th edn. London: Robinson. Denscombe, M. (2017) The good research guide: For small-scale social research projects. 6th edn. London: McGraw Hill Education/Open University Press. Fobosi, S. (2019) ‘Experience of negotiating access in the “Field”: Lessons for future research’, World Journal of Social Science Research, 6, p. 503. Kalaian, S. A. and Kasim, R. M. (2008) ‘Encyclopedia of Survey Research Methods’, [Online]. Version. Available at: https://methods.sagepub. com/reference/encyclopedia-of-survey-research-methods Mager, F. and Galandini, S. (2020). Research ethics: A practical guide. Oxford: Oxfam GB. Marvasti, A. B. (2004) Qualitative research in sociology: An introduction. London: SAGE. O’Leary, Z. (2014) The essential guide to doing your research project. 2nd edn. Los Angeles, CA: SAGE.

43

44

Planning the research

Robson, C. (2017) Small-scale evaluation: Principles and practice. 2nd edn. London: SAGE. Robson, C. and McCartan, K. (2016) Real world research: A resource for users of social research methods in applied settings. 4th edn. Hoboken, NJ: John Wiley & Sons. UKRI (2022) Research Ethics Guidance: UKRI. Available at: https:// www.ukri.org/councils/esrc/guidance-for-applicants/researchethics-guidance/ (Accessed: 7 August 2022). UKRIO (2021) UKRIO Recommended Checklist for Researchers: UK Research Integrity Office. Available at: https://ukrio.org/wp-content/ uploads/UKRIO-Recommended-Checklist-for-Researchers.pdf (Accessed: 29 July 2022). VanderStoep, S. W. and Johnson, D. D. (2009) Research methods for everyday life: Blending qualitative and quantitative approaches. San Francisco, CA: Jossey-Bass.

COLLECTING YOUR DATA

3

Literature and other forms of data David Wilkinson

IN THIS CHAPTER WE EXPLORE: ●

The importance of data – finding out what is already out there.

●

Collecting data through a literature review.

●

Collecting other research data.

●

Experiment-­based research.

●

Survey-­based research.

●

Research interviews.

●

Focus group interviews.

●

Observation as a data collection tool.

●

A note on using social media as data collection tools.

The importance of data – finding out what is already out there Everyone who decides to undertake a piece of research should feel confident and knowledgeable about the topic they are studying and the questions they are asking. A researcher who fails to invest sufficient time and effort into investigating others’ previous, related work in their chosen area of study will be unable to make much progress (Ridley, 2012). How can you refine and build upon the previous work of others if you are not fully aware of the efforts they have made and the conclusions they reached? Likewise, once you have decided to undertake a piece of research you

DOI: 10.4324/9781003180159-3

46

Collecting your data

should want your finished work to be both valued and valuable. It will be most highly valued if it is apparent that it has been informed by, and has expanded upon, a rigorous and thorough attention to similar work undertaken in the past (Boote and Beile, 2005, p. 3). It will be valuable if, precisely because of the efforts devoted to its formative stages, the research reveals the final pieces of a complex puzzle, or indeed introduces more puzzles to the debate.

LITERATURE REVIEW A research literature review is a systematic, explicit, and reproducible method for identifying, evaluating, and synthesising the existing body of completed and recorded work produced by researchers, scholars, and practitioners (Fink, 2013, p. 3). The purpose of a research literature review is to identify the previous work that has been carried out in your area and to show how it relates to your work and influences it.

To put it simply, you cannot advance knowledge in your field without first learning what has been achieved by others and what still remains to be achieved (Denney and Tewksbury, 2013). The literature review is to a research project what the foundations are to a house. Without solid foundations, the house is likely to fall down, and without a detailed look at the literature, your project is likely to be simplistic, naïve, and an inferior repetition of work already completed by someone else. Learning to review the literature has never really received the attention it undoubtedly deserves. Despite the long-­standing tradition of literature reviews featuring in the early stages of research, there has been a significant lack of attention paid to just how a researcher ought to go about searching for, collecting, evaluating, and using past research in his or her current project. In fact, it seems to be a lot more difficult to provide a definition of a literature review that we can all have confidence in than it is to recognise one on paper when we turn to the first few pages of a research report. Researchers’ opinions of, and attitudes to, the nature, process, and purpose of a literature review vary enormously. There are, however, common elements that all researchers ought to take on board.

Collecting your data

Collecting data through a literature review A literature review enables a researcher to accomplish a number of more specific aims. It is likely, for example, that in the early stages of your research you may have only a vague idea of the area you would like to explore more fully. You may have only a tentative outline of your research problem. This should not give you cause for concern. A review of the literature will help you to focus your tentative problem by both limiting, and defining more clearly, the topic you are interested in researching (Fink, 2013). Look out for recommendations made by researchers for those intent on continuing with research in a particular field. You may be provided with advance warnings of possible pitfalls, or research questions that have been thus far neglected. Reading around the subject will help you to distil the issues you wish to concentrate upon and leave you with a concise, detailed, and distinct plan of action. The existing literature relating to the topic you wish to study is just as important for what it omits as it is for what it contains. Do not be overwhelmed by the work others have done before you. You may have experienced something related to your area of interest that others have not; an experience that allows you to approach the problem from a unique and novel perspective. In examining the available literature, it is tempting to look first (or only) at the results and conclusions the authors have drawn. It is advisable to employ a little scrutiny. Rather than focus on results alone, look at the methods, measurements, and subjects that the researcher has used. In tackling a particular research problem, the use of certain methodologies and sampling procedures will prove more fruitful than the use of other, less appropriate strategies, and a good researcher will justify his or her choice from a range of possible options. Do not disregard whole studies because you may not be convinced by their results. If you throw out the baby with the bath water, you may miss out on insights into how best to design a piece of your own research that produces findings which stand up to the criticism and scrutiny of others.

Types of literature review Having discussed why a literature review is a vitally important element of any research, it is appropriate to consider just what such a review might entail. It seems that there are as many approaches to undertaking

47

48

Collecting your data

a literature review as there are research methods textbooks available to new researchers keen to conduct their own inquiries. It is fair to say that experienced and novice researchers alike differ in their understandings of literature reviews because of this. But just what are these understandings? Broadly speaking, there are six fundamental types of literature review of value for research work (Bruce, 1994). These are:

Literature review as a list A list or collection of descriptions and key words from journal articles, books, newspapers, etc. that represent the available literature on the research topic. The emphasis is on the listing of the literature rather than its content.

Literature review as a search The act of searching for and identifying information of relevance to the research topic. Again, the content of the literature does not receive priority. Instead, the literature’s ability to steer the researcher in the direction of other, relevant existing literature is the prime motivation.

Literature review as a survey or scan A survey or a scan of past, present, and possible future writing or research related to the research topic. The focus is very much on the content of the literature, especially what is known about a particular topic.

Literature review as a knowledge enhancer A means by which a researcher can increase his or her knowledge of a particular research topic and test his or her own thoughts or hypotheses. The focus here is on the literature’s potential to influence the researcher, in terms of his or her personal development, but not to influence the research.

Collecting your data

49

Literature review as a steering instrument An instrument capable of supporting, influencing, steering, shaping, or changing the research to be undertaken. In this conception, the literature influences both the researcher and the research.

Literature review as a report A report in its own right, or as a discrete section of a larger report, in which the researcher frames and thematically organises the literature. It is a final representation of the ways in which the literature has impacted upon both the researcher and the research project. There are clear links and relationships between the fundamental types of literature review. They are related in that they are progressively more encompassing. It is a good idea to view the types as six rungs of a ladder. It is not possible to climb on to a higher rung without first being familiar with, and actually using, the lower rungs. Similarly, you continue to appreciate the lower rungs in assisting you to climb higher, long after you have reached the top of the ladder.

Locating the literature Wherever you go or to whomever you speak in order to gather your information, the first thing you ought to do is identify and list as many key words as you can that relate to the topic of your research. This is because, as a general rule, all resources open to you – apart from people with experience or knowledge related to your topic you might be lucky enough to access – are organised by subject. It is by means of these key words that you will be able to find information connected with the topic you intend to study. They provide you with a starting point. Of course, in these very early stages of your research you will not want to overlook related studies completed by others before you, especially those which may prove to be important or relevant. You should therefore ‘cast your nets’ as widely as possible when you come to making a note of key words.

Six fundamental types of literature review that can help and shape a research project 1.

A list

2.

A search

3.

A survey

4.

A knowledge enhancer

5.

A supporting/directing tool

6.

A report. (Adapted from Bruce, 1994)

50

Collecting your data

Developing a review strategy: search terms and lists Try thinking of a way to turn the topic of your research into a question. What is it you may wish to find out? If, rather than doing the research yourself, you were able to approach someone in your imagination and ask them to give you the answers you seek, how would you phrase the question? For example, if you were a nurse interested in investigating the ways in which elderly patients on geriatric wards were viewed by your colleagues, you may ask: ‘What are the attitudes of medical professionals towards geriatric patients in hospital?’ Alternatively, as a teacher (say, of history) you may be interested in looking at how a new computer program may impact upon a lesson. In this case you might ask: ‘What effect do computers have on the teaching of history?’ In both cases, you can see that some of the words within your question would be appropriate key words with which to start your search: ATTITUDES, MEDICAL PROFESSION, GERIATRIC, PATIENT, HOSPITAL, and COMPUTERS, TEACHING, HISTORY. This initial list of key words may be very short – as in the second example above, which contains only three words – but do not be put off by this. The next stage is to take each of the words from your list in turn and think of related words or phrases. You might use a thesaurus to help you find these synonyms. Terms related to COMPUTERS, TEACHING, and HISTORY which you think appropriate to add might include ONLINE LEARNING, TABLETS, SMARTPHONES, INTERNET, PEDAGOGY, CLASSROOMS, TEACHER TRAINING, HUMANITIES, and so on. Soon your list will have grown, along with your chances of finding as much of the literature as possible when you begin your search. The information you do find might be located in a range of different sources – two of the most common being books and journals. Your first port of call should therefore be an online library catalogue or bibliographic database to search for relevant source material in these two formats. There are many online resource portals you can consult in order to begin your research literature review. Some of the more expansive ones require paid-­for subscriptions so it might be useful to check with your institution or organisation to find out if you already have a license to access these. Others require that you are linked to an educational institution in order to secure access.

Collecting your data

USEFUL DATABASES AND ONLINE REFERENCE RESOURCES EBSCO (https://essentials.ebsco.com/) EBSCO provides free research databases covering a variety of subjects for students, researchers, and librarians. Researchers are able to use EBSCO Essentials to search for free, reliable articles and connect to their library to access additional EBSCO content.

ResearchGate (https://www.researchgate.net/) ResearchGate is an online portal that allows you to connect with researchers across a wide range of discipline and subject areas. Materials written by other members (academics, researchers, policy-­makers etc.) can be downloaded and archived to your local account.

ScienceDirect (https://www.sciencedirect.com/) ScienceDirect is a website which provides access to a large bibliographic database of scientific and medical publications of the Dutch publisher Elsevier. The full site provides access to over 4,000 academic journals and around 30,000 e-­books.

JSTOR (https://www.jstor.org/) JSTOR is an online library that provides access to more than 12 million journal articles, books, images, and primary sources across 75 distinct disciplinary groups. The service was established in 1994 and is used by researchers worldwide to support literature searches.

Google Scholar (https://scholar.google.com/) Google Scholar has been referred to as the more academic version of Google. Rather than searching all of the indexed information on the web, Google Scholar searches repositories of publishers, universities, or scholarly websites. Searches using the platform therefore tend to be more focused and relevant than a general search using Google.

51

52

Collecting your data

Web of Science (https://www.webofscience.com/) The Web of Science is a paid-­access platform that provides access to multiple databases that provide reference and citation data from academic journals, conference proceedings, and other documents in various academic disciplines. It is one of the largest, if not the largest, bibliographic databases in the world and spans a diverse range of disciplines in its coverage.

Wikipedia (https://en.wikipedia.org/) Wikipedia is a free online encyclopaedia written and maintained by a community of volunteers. Anyone can log into the online resource and edit/update or amend entries. This makes its content accessible and up to date, but has also encouraged users to create distorted or inaccurate entries. As a reference resource it is useful, but researchers have generally been guided away from it as an authoritative source. This is despite the fact that some studies have found that its content is less inaccurate than has historically been perceived (Giles, 2005).

The range of materials which you may be able to access will no doubt vary according to your own particular circumstances, and you should make efforts to find out what facilities are available to you (Knopf, 2006). As we have already indicated, your institution or organisation may have specific licensing arrangements with particular online reference resources or databases. The more resources you can consult and the wider and more specialist the content held in them, the more chance you have of your literature review being thorough and exhaustive. As you consult these resources, you may see patterns emerge. Perhaps a small number of authors appear again and again, or you may notice alternative synonyms to those you listed crop up repeatedly. In either case, cross-­referencing the bibliographies with the actual library catalogue may help you to find more books of relevance to your topic of study than you thought were available after consulting just the library catalogue.

Collecting your data

It is a good idea to use indexes, categorisations, and abstracts provided by online databases and reference resources. These supporting functions are designed to help you to identify and locate research articles and other information relevant to your own project. Finding precisely what you want is by no means a straightforward task, but to discover a reference closely centred on exactly the nature of your own inquiries has the potential to be the most valuable single find in your entire investigation, so persevere!

Selecting appropriate literature and maintaining literature notes As you begin your investigations in earnest you will begin to get a feeling for the amount of information that has been written by others about the topic you wish to research. You might find yourself with tens, or possibly hundreds, of pieces of relevant information, each of a different length, prospective audience or focus of study. At this point you need to consider ways in which you might manage and organise all this information to prevent it getting out of hand. There are a number of reference management software packages available to support researchers in logging, managing, and coordinating your information sources. One of the most popular ones is Endnote, which helps you collect and store all the references you have found from different sources. You can use your Endnote library to insert in-­text citations and create bibliographies within Microsoft Word documents, reformatting them into your chosen referencing style. As you collect your research literature and log its detail within Endnote, the software helps you to categorise and keep track of the books, policy papers, websites etc. that you have consulted.

TOOLS TO HELP YOU MANAGE YOUR LITERATURE: ENDNOTE Endnote (https://endnote.com/) is a commercial software program that helps you to create, store, and manage the literature you consult as part of your research project. It can save

53

54

Collecting your data

references and citations you import from other sources (such as online library catalogues and databases). You can use Endnote with Microsoft Word to produce reference lists and bibliographies to support your research report writing.

When you are selecting material for consideration and potential inclusion in your literature review, you should start with the most recently published materials. They are likely to be more valuable to you as their authors should have used earlier research as a foundation. Apart from noting accurately where you found the material, a good summary of a piece of research literature you uncover should include details about: ● ● ●

● ● ●

The problem the material is attempting to address. The purpose(s) of the study or studies related in the material. Brief information about the population(s) studied – comprising whom? How many subjects? Methods and techniques used by the researcher(s). The results of the study or studies. Any conclusions.

Introductions to articles and other published materials usually contain details on the first two of the above points, while more information on methods and results is usually reported in the middle and the end of the material respectively. Of course, after reading through the abstract at the beginning or the summary at the end, you may decide that a particular reference contains insufficient information relevant to your study to justify reading the whole thing from end to end. One thing is certain: The contents of a literature source will remain fresh in your mind only for the time you are reading it. With so much to familiarise yourself with you will begin to confuse the contents and conclusions of others’ research very early on in your reading. With each new piece of literature you consult you will find yourself relying increasingly on the summaries you make, so it is worth investing some time and effort in collating thorough and consistent notes at this stage. Apart from listing all the pertinent points in each of the studies you consider to be important, you might like to record your own evaluation of the study at the same time. If you are utilising reference management

Collecting your data

software such as Endnote to support this, you can make additional notes within the reference record in the software. These additional notes can provide your assessment of the arguments in the reference as well as your other thoughts on how it might link with other materials you have consulted. One rapid way to familiarise yourself quickly with any type of written material consulted to inform your research project work would be to scan what has been written by reading the first one or two sentences of each paragraph. This ought to provide you with enough information to help you to decide whether an article or chapter is worth reading thoroughly from start to finish. In the case of those research papers, policy papers, journal articles etc. that you choose not to discard you should still be able to summarise the problem being addressed in the material, and the conclusions drawn by the author. In such cases, your focus should turn to summarising and evaluating the theme or themes of the material: What is the author saying? What reasoning, logic, or arguments does he or she use to say it? On what is the author’s reasoning and logic based? Can you see any particular strengths or weaknesses in the author’s arguments? Occasionally, you may find an author has written something in a particularly skilful way, e.g. managing to phrase a complex idea, argument, or conclusion in concise and clear terms. Similarly, you may sometimes come across a couple of lines in a report which sum up the essence of the whole article. If you find yourself in this situation it would do you no harm to copy this down carefully somewhere in your summary, enclosing the extract in quotation marks and noting down the relevant page number.

Aggregating literature material As you continue with this process you will begin to find yourself surrounded by dozens of literature sources containing a summary of a piece of work related to, or important for, your own intended research. Just as it is necessary to impose some kind of order upon your choice of key words in the earliest stages of your literature search if it is to be focused and well-­ defined, it is equally necessary to organise and group all your literature notes in order both to maximise their value for your research and to minimise your workload. One way of organising your summaries is to code or group each one according to the characteristics of the information it contains. You may find that your original key words can, on the whole, act

55

56

Collecting your data

as ‘pegs’ on which to ‘hang’ your summaries. For example, the author of a review of the literature written about the use of online learning technologies in classrooms might code each summary according to one or more of the many issues which arise when bringing new technology into schools: L

Articles dealing with pupils’ abilities and how pupils Learn.

U

Articles about the User-­friendliness of software, its scope, its aims and the way it ought to look.

M

Articles relating to the structure and content of additional Materials,

T

Articles dealing with Technical issues surrounding the software, such as

E

Articles about Educating pupils using software, for example, when to

such as users’ guides and teachers’ notes to supplement the software. its compatibility with a school’s existing hardware. use it or when not to use it. R

Articles about the technology’s impact on teacher and pupil Roles, including the interactions between pupils, teachers, and computers.

These codes, categories, or groups can be used to identify and quickly locate all your notes relating to one area within your topic of study. In addition, perhaps most important for the novice researcher overwhelmed by information, it should ease the burden of writing up your review by dividing it into easy-­to-­handle, bite-­sized pieces. Again, if you are utilising reference management software to support your review process, you will be able to apply multiple organising groups or categories to your reference sources.

Critically analysing the literature How much trust should you place in the research you find? Even after you have discarded those studies that appeared to be pertinent to your own research but, on closer inspection, proved not to be, does each and every piece of research you are left with deserve to be a part of your own inquiry? If not, on what basis should you either include or exclude material in the section of your own work which deals with related, previous research? In short, you may have to make some evaluative decisions. In order to do that, you have to establish your own set of criteria for judging the adequacy of the material in front of you. As authors base the conclusions of their studies (at least in theory!) on the outcomes of analyses of the data they have collected, any critical evaluation ought to include as its

Collecting your data

focus a thorough scrutinisation of a study’s methodology and results sections. You may find instances where data have been collected or recorded unreliably or erroneously. You may also discover that results or analyses have been calculated incorrectly, and that conclusions have been made on the basis of those miscalculations. Such errors do occur – researchers are not an infallible breed. Obviously, the extent to which you feel confident in your own ability to make calculations and recalculations of others’ data to check their reliability and authenticity will depend on your background and experience of these situations, but even the least confident of novice researchers should be able to spot intuitively any reported values or measurements that seem spurious, or at least a little odd. Can the results be trusted in your opinion? Do you think the study was carried out in a sufficiently careful manner? After considering just these few basic questions you ought to be able to do one of three things:

● ● ●

Include the study in the literature review section of your own work. Exclude the study because it ‘fell at the first fence’. Reserve judgement as to the trustworthiness of the study until you are able to make a more informed decision, perhaps based on a wider range of criteria for judging research quality.

Being aware of bias Even at this stage you should be aware of something called confirmatory bias (Peters, 2022). Researchers – as well as being fallible creatures – do not live in a vacuum. We have been known to have biases and predispositions towards certain points of view and certain outcomes of studies rather than others. This, if not sufficiently borne in mind, could lead your evaluation of another’s research to be coloured or distorted by its premise, outcomes, or conclusions. Experiments have shown that a reviewer’s predispositions towards a review’s results can influence his or her judgement about the quality of a piece of research. In the past, reviewers have accepted material which, despite containing questionable and dubious methodology, has supported their own intuitions, while rejecting sound and well-­grounded research that advances counter-­intuitive conclusions (Button et al., 2016). To help you to critically assess the material you are collecting, you should consider the following questions:

57

58

Collecting your data

Does the material go into sufficient depth? Does it provide you with enough information about, for example, the reasons why the research was conducted; the context or location in which the research took place; the methods employed; and the results obtained? Do you get the impression the author has included as much detail in the write-­up as he or she was capable of, or do you feel that there have been certain omissions? Are there any inconsistencies? For example, are there inconsistencies in the way the author refers to, or provides information about, the size(s) of the population(s) studied? Are references to the initial sample and the achieved sample made consistently? Confusing the number of people originally contacted to partake in the research and the number who actually responded and took part will produce inaccurate results, occasionally skewed in favour of the researcher! Does the researcher inform you to your satisfaction of details about response rates, the sizes of sub-­samples, the number of drop-­outs and those unable to be contacted (sometimes referred to as the attrition rate), and the total on which any percentages are based? Where did the author obtain this information? Is it clear how and from whom information presented to the reader as fact was obtained? Which questions were asked? Of whom? Were there any attempts to obtain corroborative evidence from another source? Do you sense any assumptions made by the author? In other words, has he or she accepted some aspects of the research blindly which you would have liked to have been investigated further? Are the author’s claims reasonable? Do you feel that too much is being claimed on the basis of the evidence in front of you? Have the author’s analyses been adequate? When something is referred to as ‘significant’, is it? How has the author measured significance? Do you consider there to be equally plausible explanations for the results of the research that the author has failed (or worse, refused) to consider?

ACADEMIC In terms of bias, academics might be interested in ensuring the depth and scope of the research work. Referenced or cited content might be one of the most important elements for them to ensure the content is representative and free from bias.

Collecting your data

Industry From a commercial perspective, industry might be concerned that the research reporting includes an advanced critical analysis of the research data. If a new product or service is to be launched following the research work, an industrial sponsor might need to be confident that background work has objectively summarised the work of competitors.

Public sector From a public sector perspective, research work might need to be assessed and evaluated to ensure that this is free from political influence.

Society Third sector organisations, charities and groups are usually concerned that research work is objective and free from discriminatory biases.

Collecting other research data Fundamental types – quantitative and qualitative We have already, in Chapter 1, outlined the key traditions or paradigms that inform the development of a great deal of research work undertaken in the social world. These traditions also have some influence over the research tools or instruments deployed to explore research questions. In turn, these tools or instruments control the types or forms of data collected. Broadly speaking, the research data you collect will be either quantitative or qualitative in nature. Both of these fundamental data types or forms have value (and limitations); they provide support in providing evidence to answer specific research questions. Quantitative data focus on recording and understanding truth typically through the use of explanatory models or theories. It is often reductive in nature and relies on hypothesis testing. Qualitative data, on the other hand, is more broadly focused. It is centrally interested in meaning and exploring this in context and practice. One core aim of qualitative data is to support contextualised and situated knowledge (Braun and Clarke, 2021, p. 6).

59

60

Collecting your data

Which tools or instruments? There are a wide range of potential data collection devices or instruments you can utilise for your research work – each of these are either quantitatively or qualitatively framed in terms of the data they collect and present for analysis. Some tools also have the potential to collect data that are both quantitative and qualitative (such as a survey that captures ratings or grade-­type data as well as open-­ended comments or reflections). We identify some of the most popular data collection devices below and discuss their utility as data collection tools and instruments to help support your research project work:

Experiment-based research Experimental research is the core approach for positivist researchers interested in testing and control of the research space or environment. As such, it is commonly viewed as one of the most rigorous of all research designs and approaches to collecting research data. In the purest experimental research design, one or more independent variables are manipulated by the researcher; subjects or participants are then randomly assigned or attached to different variables, and the results of these assignments or attachments are observed.

EXPERIMENT-BASED RESEARCH AND RANDOMISED CONTROL TRIALS (RCTS) Randomised Control Trials (RCTs) originated in clinical settings and are known as the ‘gold standard’ of medical and health research. They are a form of experiment-­based research with two or more randomly selected groups (an experimental group and a control group) in which the researcher controls or introduces an intervention (such as a new training or development programme) and measures its impact on the dependent variable on at least two occasions (usually termed pre-­and post-­test measurements). RCTs are often used for addressing evaluative research questions, which seek to assess the effectiveness of programmatic and policy interventions in developmental settings.

Collecting your data

One identified strength of experiment-­ based research is its recognised ability to link cause and effect through variable manipulation. The technique also adapts variables, through modelling and statistical calculations, to be able to control for (and limit) the effects of spurious or unrelated variables. Experimental research is applicable in social research when you might wish to determine a causal relationship between variables (Dunning, 2012). To do this you would select the experimental group of participants or subjects (where you can apply a specific treatment or variable) and you would compare these participants to another group of participants who had not been exposed to the same treatment or variable. Sometimes it is not practically possible to conduct research work that is experimental-­based in its purest form. This happens when, for example, it proves difficult to randomly assign participants to either a participant (treatment) group or a non-­participant (control group) (White and Sabarwal, 2014). Indeed, it is often seen as unethical to arbitrarily determine who should receive an intervention and who should not (Cook and Thigpen, 2019). Where this happens, researchers who wish to utilise some form of experimental design in their work turn to a less-­rigid definition and practical application. A quasi-­experimental research design allows the researcher to identify a comparison group that is as similar as is possible to the participant or treatment group. This flexibility in approach makes the technique much more attractive to social scientists who have labelled the approach ‘field experiment’ research (McIntosh-­Scott et al., 2014). To be categorised as an experimental data collection design, research must include: (a) An intervention or treatment that can be observed – this could include an education or training programme, undertaking an exercise regime, agreeing to carry out a range of tasks or a test. (b) A control mechanism – included here would be a suitable comparator group of participants (who are not part of the intervention or treatment). These act as a controlling device and help to reveal and eliminate supplementary variables that are unimportant for the purposes of the research work. (c) Randomisation – this involves the researcher randomly (where possible) assigning participants to either the treatment or intervention

61

62

Collecting your data

group or the control group. Doing this removes the issue of selection bias and supports the development of balanced and comparable groups.

Survey-based research Surveys are perhaps one of the most popular research instruments through which to collect research data. They are an accessible option for researchers who are eager to understand what customers, service users, and clients think about products or services. They help businesses make better business decisions and help policy-­makers to shape and adapt policies. This type of research has a long history of supporting diverse types of research investigations. Surveys have traditionally been viewed as essentially quantitative research tools because of their reliance on closed, pre-­formatted question types. However, the use of open-­ended elements and free-­text response options enables qualitative content to be captured by them.

Open-ended and closed-ended questions Open-­ended questions are questions that allow someone to give a free-­form answer. One of the advantages of using open-­ended questions in survey-­based research work is that they do not restrict the respondent to a predefined set of responses. Closed-­ ended questions are those where all responses are predefined and categorised to enable the respondent to provide a response (Hyman and Sierra, 2016).

Multiple-choice and scale questions Many surveys include questions which provide a number of pre-­defined responses. This allows the researcher to hold some control over the responses given. However, the construction and

Collecting your data

piloting of multiple-­ choice questions usually requires careful thought to ensure that all or most responses possible are covered (Johnson and Morgan, 2016).

Figure 3.1  Examples of open-ended questions

63

64

Collecting your data

Figure 3.2  Examples of multiple-choice questions

There are many commercial providers who can support the online development of your survey questions, the collection of the data, and even its synthesis and analysis. An online survey is a method for extracting information about your research topic from an individual or a group of individuals. It consists of structured survey questions that are, usually, well presented to guide and encourage your participants to respond. Because of this, survey-­based research is extremely useful when you are required to collect data from a large number of respondents. Research surveys consist of a number of standard elements. These include: An introduction, a statement identifying how data will be used, an indication of the number of questions (or how long the survey might take to complete), and any incentives that are being offered to respondents to encourage them to complete the survey.

IMPROVING RESPONSE RATES IN ONLINE SURVEYS – SOME TOP TIPS ●

The length of time its takes to complete the survey has a direct impact on response rates and drop-­out rates, so keeping the questionnaire below 15 minutes (and ideally nearer to 5 minutes) is a key aim.

Collecting your data

●

However, with online questionnaires – whatever the length – you need to actively consider the respondent experience of completing the survey to manage their motivation to give considered answers.

●

Keep the words to a minimum to encourage respondents to read them.

●

Edit out superfluous language that is a carry-­over from the text-­heavy style needed in interview-­based surveys.

●

Drop key parts of the question into the response lists. This helps to maintain the focus and attention of the respondent.

●

Explore question approaches that will draw the respondents in by, for example, creating situations they can relate to or associate themselves with. (Brace and Bolton, 2022)

The introduction section of the research survey is where the main purpose of the research work is usually summarised, in accessible terms relevant to the respondent. Usually, the main aims and objectives of the research work are covered here. Usually, an introduction also includes a data protection statement outlining how long respondent data will be held for, how it will be analysed, and how personal data will be treated. The collection and protection of identifiable information, which includes a respondent’s name, job role/title, or another identifier should be a central part of your research ethics processes. If you are conducting your project work on behalf of a company, college, or university you will need to ensure you comply with any guidance issued centrally on this. Of particular relevance, when collecting and storing personal data, are the rules and guidance presented in GDPR legislation.

GDPR On 25 May 2018, the General Data Protection Regulation (GDPR) became law across the European Union. This legislation requires that any personal data collected as part of research survey work (or any other research activity) should be held securely.

65

66

Collecting your data

In addition, the legislation demands that respondents should be told how their data are to be used, for how long it will be held, and how they can request access to it . (Foulsham, Hitchen and Denley, 2019)

It is good practice to provide an indication, up front, to potential respondents identifying how long a survey might reasonably take to complete. Often, this is presented as something similar to: ‘This survey should take around 10–15 minutes to complete’, or ‘There are four sections to the survey, each section comprising between 3 and 4 questions’.

ROUTING AND PIPING Routing allows you to direct a respondent through your survey based on the answers that they give. If a respondent provides a particular answer, then they are directed to a particular page in the survey. They are very useful devices for directing respondents to relevant sections in a survey, allowing them to skip questions they don’t need to answer. Sometimes routing in an online survey is called ‘skip logic’ or ‘branching’. This technique is used extensively in large-­scale survey work. Piping is a survey technique whereby a previous response is included in a subsequent question. An example of this might be: ‘Can you please tell us why you thought our service was poor?’ – where ‘poor’ was the response to the previous question: ‘Tell us about our service’.

Offering incentives to respondents is a way of acknowledging, and valuing, the contribution that respondents are making to your research work. They can include gift vouchers for each respondent who completes the survey (for research projects with large budgets), but often include placing a respondent in a prize draw following completion of your survey.

Collecting your data

ADVANTAGES OF USING A SURVEY FOR YOUR RESEARCH WORK ●

Low cost to develop and deploy.

●

Easy to get information from a lot of people very quickly.

●

Respondents can usually complete the questionnaire when it suits them.

● The analysis of answers to closed questions is fairly straightforward. ●

Less pressure for an immediate response.

●

Respondents are usually anonymous (and therefore more honest?).

Research interviews Interviews can vary in their structure and formality. They are flexible in that they can be carried out over the phone, online (via Teams, Zoom, Skype etc) or face-­to-­face. They can be focused on a given set of pre-­defined questions that are covered in turn (known as a structured interview), or they can focus on a pre-­defined theme or area and allow a discussion to take place between researcher and interviewee on that theme (known as an unstructured interview). In many interview situations, a mixture of the two approaches is used, where some structured questions are asked followed by the exploration of general themes related to those questions. It is important when planning your interview to consider the information the interviewee might reasonably need to know, the location of the interview, whether you would like to record the interview, how the interview is going to be used in your write-­up or transcription and analysis.

Before the interview Prior to the interview, you should have informed your interviewee on the area of research, either by telephone or by letter or email, and given a guideline on the anticipated length of interview. If the interview is to be audio-­recorded, then consent should be sought. The respondent should

67

68

Collecting your data

be advised of the potential audience of the research. They should also be given appropriate assurances about anonymity and confidentiality. Sometimes, when collecting interview data, your interviewees may wish to verify and check their responses (by viewing any transcript or notes you produce). This allows interviewees the option to clarify what they said, and also to provide greater detail on their responses. However, it should be carefully managed as a research device as data may become sanitised or ‘cleaned’ in some way by interviewees (Caretta and Pérez, 2019).

MEMBER CHECKING INTERVIEW DATA Member checking, also known as participant or respondent validation, is a technique for exploring the credibility of results. Data or results are returned to participants to check for accuracy and resonance with their experiences. (Birt et al., 2016, p. 1802)

Setting up the interview ‘space’ The interview setting should be carefully chosen, with minimum outside or distracting noise. This is easier to achieve if you are carrying out the interview in a face-­to-­face setting, but can be more challenging if you are asking someone to participate in a Teams or Zoom call. In a face-­to-­face setting, try to ensure that no interruptions are likely to take place and that chairs are carefully placed in the room at a comfortable distance apart. If an audio-­recorder is being used, you should familiarise yourself with its functions. It is often helpful at the planning stage to pilot your interview with a friend or colleague who can provide constructive feedback on your questions and interviewing performance.

Conducting the interview Your interviewee should be made to feel comfortable, so begin the interview by thanking them for cooperating and assisting with your work. This helps to establish a relaxed attitude.

Collecting your data

Any encounter between two people involves conscious and unconscious elements. Besides the verbal aspects of the interaction, non-­ verbal communication also affects the encounter in both obvious and more subtle ways. Both body language and eye contact are important, and you may wish to record or note an interviewee’s body language or eye contact when you cover a particularly important question or theme. For example, they may adopt an aggressive pose when answering questions covering subjects they feel strongly about. It may also be useful to occasionally mirror the interviewee’s body language to encourage rapport. If audio-­recording an interview you should secure consent from the interviewee prior to the scheduled interview session. It is helpful to set up the audio-­recorder (on a table, with an attached microphone if necessary) prior to the interview so that you and the interviewee are not distracted by its presence. Also, when conducting interviews, it helps interviewees to relax if initial questions are relatively easy to answer. Contextual questions such as: ‘Tell me what it’s like to work here?’, ‘What do you enjoy about your work?’, and other questions like these are usually safe positions to start from. A typical interview guide or schedule is provided below. This is taken from a research project exploring career decisions for healthcare professionals. Each question is open and exploratory, encouraging an in-­ depth response from the interviewee.

EXAMPLE GUIDE/SCHEDULE – A CAREER IN MEDICINE Why did you decide to go into a medical career? What education, training, or other preparation did you need to become a medic? What are you working on now and what is enjoyable about it? Tell us what a typical day/week involves for you? What do you spend the most time doing? What continues to inspire you or hold your interest in medicine? What would you say to those interested in following your career path? Is there anything else you think I should know?

69

70

Collecting your data

Without use of an audio-­recorder some method of creating an account of the event is necessary. Taking notes is one possibility, but this may be difficult to maintain whilst giving full attention to the content of the interview. If you do take notes, attempt only to note key points made by the interviewee rather than trying to capture everything they say. Your notes should act as a reminder of what was said at the time. As the interview draws to a conclusion, it is useful to ask the interviewee if they have anything to add that has not been addressed by any of the questions or their responses. This helps avoid a situation where the respondent is prompted to add to the interview following the conclusion of the interview. All interviews should conclude with a heartfelt message of thanks from the interviewer. Interviewees are sharing some of their precious time with you and answering your questions, without their support your research would not be as rich or thorough. You should acknowledge this in your expression of thanks at the end of the interview.

BENEFITS OF THE RESEARCH INTERVIEW ●

Very useful for collecting in-­depth information about a topic

●

Can be an easier way than other approaches to collecting

●

Interviews are usually flexible research tools and can be

●

Interviewees are not required to write their responses.

or subject. potentially sensitive information. reactive to responses from interviewees. ● Interviewers can record non-­ verbal behaviour from the interviewee (body language etc.).

Focus group interviews There can be few individuals who have not heard of focus groups (sometimes referred to as discussion groups or group interviews). Political parties rely on them, market research agencies use them, and television companies are reported to amend their programmes following an analysis of focus group responses. Although historically conducted in a face-­to-­ face setting, more and more focus groups are now being carried out as an online activity (Kaufman et al., 2022). Whilst the traditional model of

Collecting your data

a single face-­to-­face focus group is still very popular, recent review work in the area has identified a range of permutations, including mini-­focus groups, groups with multiple moderators, and two-­way focus groups (Nyumba et al., 2018).

DEFINING FOCUS GROUPS The focus group is a unique research instrument in terms of its purpose, size, composition, and procedures. The central aim of such a group is to better understand how people feel or think about an issue, idea, product, or service. Focus groups are essentially used to gather opinions. (Krueger, 2015, p. 26)

The process is based on the principles of self-­disclosure, grounded in a comfortable environment, a particular type of questioning, and the establishment of focus group rules. Generally numbering between seven and ten individuals, groups have been conducted with a minimum of four. Beyond twelve participants, the group tends to fragment. The mix will probably consist of strangers, or people slightly acquainted with one another, but there will be similarities between them that bring them together as a group. The discussion in a focus group is led by a moderator or facilitator who introduces the topic, asks specific questions, or suggests themes for discussion. It is the moderator’s role to control the flow of the discussion and encourage all present in the session to make a contribution. Sometimes, there are dominant voices or opinions in the group, and it is the job of the moderator or facilitator to ensure that everyone can contribute irrespective of their view.

ADVANTAGES OF USING A FOCUS GROUP ●

Can receive a wide range of responses during one meeting.

●

They help researchers to find out more about what people think by encouraging discussion.

●

They are useful in allowing participants to explain perceptions of an event, idea, or experience.

71

72

Collecting your data

●

Participants can ask questions of each other and build on the comments of others.

●

They are usually informally set, thereby encouraging participants to contribute to discussions.

●

Participant interaction can be explored as part of the analysis process.

In some situations, it might be useful to have two moderators for a focus group. One may attend to the questions, and the second may provide very specific subject or specialised expertise on the core topics of the focus group (this might be very useful in a medical or health setting, for example) (Krueger, 2015, p. 524). Compared with one-­to-­one interviews, the questioner in a focus group interview situation plays the role of a ‘facilitator’, rather than a ‘director’ of the proceedings. Once the general topic for discussion has been fixed to everyone’s satisfaction, he or she is responsible for shaping and steering the path the participants themselves have chosen to tread. In contrast to one-­to-­one interviews, the job of determining the precise content of the discussion within the boundaries of the topic as a whole is deliberately left to the participants. The reasoning behind this lies in the implication that those aspects of the topic most important, meaningful or relevant to the participants will emerge first in the discussions. It is important that the moderator or facilitator encourages comments of all types, both positive and negative, taking care to avoid making judgements about responses and controlling body language communicating approval or disapproval (Robson and McCartan, 2016).

The focus group checklist Focus groups require careful planning and management and can therefore take up a lot of research time and effort. We’ve devised a useful checklist of factors to consider and account for to help make them run smoothly: 1. Make sure the venue is suitable (Is it available when you need it? Will it be free from external noise and disruption for the duration of your meeting?). If you are holding an online focus group practice with the software you are going to use so that you are comfortable with it.

Collecting your data

2. Check you can record voices so that the quality is suitable for your own analysis and transcription purposes. 3. Contact your participants and confirm their availability to attend. Over-­recruit so if you’d like six to attend, try to recruit ten to ensure you have enough participants. Contact your participants the day before the session to confirm their attendance (either via email or phone). 4. Be available at the venue (either online or face-­to-­face) early so that anyone who has arrived early or has logged on early knows you are there to welcome them. 5. When the focus group begins greet the participants, say who you are and introduce the purpose of the focus group. Be clear to indicate how long the session will last. Provide information relating to confidentiality and anonymity before seeking approval to begin with audio-­recording of the session. 6. When asking questions or raising topics for discussion, observe group dynamics and make written notes to accompany your audio-­recording. 7. At the end of the session, thank participants for their contributions and provide contact details of the research team so that they can follow up with any additional contributions they may have to the discussion. This also provides them with a further opportunity to ask you additional questions about the research.

Observation as a data collection tool With some research approaches, you may be required to observe directly the activities of members of a particular social group with a view to providing an accurate description or evaluation of those activities. This is known as ethnography. Essentially, there are two forms of ethnography – ­participant and non-­participant observation. With participant observation, you as researcher are a part of the situation you are observing. For example, you could be involved with a meeting you are recording for your research, or you may be exploring the way your work environment changes due to the introduction of new working practices. Non-­participant observation involves you, as a researcher, being more detached from the meeting you are observing. For example, you may be present at the meeting but, as a non-­participant, you will have no input or effect on the meeting. Recent

73

74

Collecting your data

expansion of digital approaches to interacting, communicating, and working has led to refinements in ethnographic methods, including specific rules and practices adopted by researchers when engaged in ‘digital ethnography’ (Pink et al., 2016), and approaches that target the internet as a data source (Hine, 2020).

ETHNOGRAPHIC RESEARCH Ethnographic research focuses on observing social interaction in a space or environment. This form of research provides an in-­depth account of people’s views, understandings, reactions, and behaviours. It allows the researcher to be immersed in the world of those they are researching. They can begin to understand the world from the perspective of those being observed. The emergence of ‘rapid ethnographies’, providing timely and in-­depth understandings of social worlds and environments, are popular in fast-­changing settings such as healthcare. (Vindrola-­Padros and Vindrola-­Padros, 2018)

Types of data collected: passive and contextualised There are two fundamental approaches to data collection within the observation or ethnographic approach. You can collect data in a passive sense (working as a non-­participant observer), or you can collect data in a more direct way (with an alignment to a participant observer) by asking questions and contextualising or framing your observations. Devices or collection instruments to support passive or non-­ participant observations include taking written notes, creating sketches or situations or environments (such as a teaching space, factory floor, meeting room etc.), capturing audio or video of interactions or behaviours. Contextualised approaches are more direct and intrusive devices for collecting observation data and include scheduled review or observation meetings, one-­to-­one interviews, or group discussions. When recording your observations, note the context of the event. A general description of the time, place, setting, and participants is a valuable adjunct to any data collection. Sensitivity to the atmosphere and noting any key events, which may include the late arrival of a particular person,

Collecting your data

helps contextualise activities. Charts and seating plans will help to identify participants in a group setting, but beyond the physical environment some form of observation reference guide might be useful. Whatever the device or approach used, it is often difficult to narrow down the focus of observations to something meaningful that supports the broader research aim or focus. Observation frameworks are often useful in this situation; they help shape and direct our efforts. The one below forms the basis of a research project investigating an organisation’s practices and behaviours following the implementation of new working practices.

WHAT TO OBSERVE: AN EXAMPLE FRAMEWORK FROM AN ORGANISATIONAL STUDY 1.

Observing time and space. Areas of interest here might be: How is time organised in the setting and who makes decisions about this? How is the space organised and who controls or manages this? Are all business spaces organised in the same way or do some receive special attention? What types of activities are promoted at different times of the day and in different places? Are there any tensions and conflicts due to time and space management; if so, how do these tensions manifest themselves?

2.

Objects and materials. What are the physical objects ­present – machinery, equipment, furniture, food, decorations, signs, images, computers etc.? What is used, what is not used? How are objects used and interacted with? Are some objects used by all and some only by a few?

3.

Actors and participants. Who interacts with others in the space or environment? Are people generally content with their roles and activities? Do they look and behave in a way that I’d expect? Who is behaving differently or stands out in the setting? What is the status of different people? Is there a variety or rather a homogeneity of appearances and behaviours?

4.

Interactions. What can I observe that people are actually doing in the setting? What tasks are begin performed and how regularly? What do I observe about non-­verbal

75

76

Collecting your data

behaviour and is this present in particular areas? What do people talk about and it what way? Are comments positive, negative, or neutral? How are differences in power expressed, reproduced, negotiated, or challenged? 5.

Routines. What happens regularly and who carries out these regular tasks? What happens less frequently and who is responsible for these? Are there any ritual behaviours expressed in the setting? Who is responsible for routine and ritual behaviours, and how are they expressed? Are any comments made about these behaviours? If so, by whom? Adapted from: Ciesielska, Wolanik Boström and Öhlander (2018, p. 37)

ADVANTAGES OF USING OBSERVATION ●

It provides direct evidence of the event or process under investigation.

●

You can observe behaviours and reactions to events.

●

You are the key research instrument or device and can collect large amounts of data quickly and with limited or no technical support.

●

The data you collect can provide insights into the complex realities of the setting being observed.

A note on using social media as data collection tools When the first edition of this textbook was written over twenty years ago, using social media to harvest or collect research data was limited. Now millions of people daily consult social media feeds and contribute posts, tweets, or upload images and video content (Snelson, 2016). All of these data are potentially rich sources of valuable material for research work (Bik and Goldstein, 2013). Recent work in the area of social media use for research purposes identifies that data from such sources can be flexibly collected and formatted, although some platforms are recognised as being more useful for specific opinion/behavioural research projects than

Collecting your data

others. Reviews of users and content accessed have also been illuminating in revealing the types and kinds of individuals accessing some of the larger social media platforms such as Facebook (Wilson, Gosling and Graham, 2012) and Twitter (Sloan et al., 2015). Social media platform

Advantages

Twitter







Facebook

 

Instagram

  

One of the largest microblogging platforms where users generate tweets, retweet, and like postings. The use of hashtags and Twitter handles increases the engagement of the account holders. Photo- and video-sharing services make the platform attractive for education and live streaming of meetings.

Disadvantages   

Billions of daily users worldwide. Actively used for live streaming meetings, and interconnecting with Zoom, YouTube, and other videosharing sites.



Platform primarily used for sharing photos and videos. Easy for users to access and share content. Embraces visual approaches to communications rather than focusing on textbased methods.



 



Limited number of characters in a tweet Presence of many Twitter bots with indiscriminate automatic posts. Some limited use in non Anglophone countries.

Mostly used for personal communications. Presence of bots with indiscriminate posts. Potential bias to posts and likes to posts. Used for personal and professional communications. Informal style of communication not suitable for all research topics.

Example social media plaorms and their use as data collecon devices (adapted from Zimba and Gasparyan, 2021, p. 70)

Figure 3.3  Advantages and disadvantages of social media platforms as data collection devices

Rapid access (often in real time) to social media data reveal one of its major advantages over other data collection tools or instruments you may wish to use. Using social media collected data it is possible to generate population level, and actionable, data in near real time; this is very useful for time-­sensitive research work. Therefore, using automated technological tools you can collect, clean, store, and analyse extremely large volumes of social media data very quickly (Edwards et al., 2013).

77

78

Collecting your data

COLLECTING SOCIAL MEDIA DATA – DISCOVERTEXT DiscoverText (https://discovertext.com/) is a web-­based collaborative text analytics system that allows researchers to access and download data from social media feeds, including Twitter and other sources. DiscoverText was released in 2009 and enables analysis of diverse medium-­and large-­scale text data and associated metadata. It combines human interpretation and machine-­learning principles to perform text classification. It has been used across a wide range of areas of research interest, including the impact of the 2020–2022 Covid-­19 pandemic on home-­based learning. (Chia, Ma and Tay, 2022)

The increase in the use of social media platforms as research data collection devices has influenced the emergence of a new approach to social research, termed Netnography. This is a research design that adapts ethnographic (essentially observational) research techniques to study the cultures and communities that are emerging through online communications. In a similar way to ethnographic research work, netnography focuses on human experiences and cultural understanding; is grounded in deep appreciations of the context of people’s everyday lives; explores social systems and shared meanings; and is informed by the self-­awareness or reflections of the researcher (Kozinets, 2020, p. 15).

Summary This chapter has provided detail on the various tools, instruments, and devices you can use as a researcher to help collect and collate data. The research literature review is often overlooked as a data collection source, but this is a valuable component of any socially framed research project. A competent review of published material allows you to frame your work in context and enables you to relate your work to that of others.

Collecting your data

We identified that other research tools or devices you can employ for your project work are essentially quantitative or qualitative in nature. These fundamental characteristics influence the approaches used to collect data with our list of tools or devices. Experiments are useful for collecting statistical data and are usually limited to specific philosophical traditions in research. Survey-­based research work is one of the most often utilised approaches in research work and, through online methods, collect vast amounts of data quickly and easily. Interviews and focus group approaches are more flexible and organic in nature and can be useful for contextualising topics of research interest. Observation or ethnographic approaches are useful mechanisms through which to immerse ourselves in the world of those we are researching. Finally, we considered the value of tapping into social media tools and devices to identify rich and valuable sources of research data.

References Bik, H. and Goldstein, M. (2013) ‘An introduction to social media for scientists’, PLoS Biology, 11, p. e1001535. Birt, L., Scott, S., Cavers, D., Campbell, C. and Walter, F. (2016) ‘Member checking: A tool to enhance trustworthiness or merely a nod to validation?’, Qualitative Health Research, 26(13), pp. 1802–1811. Boote, D. and Beile, P. (2005) ‘Scholars before researchers: On the centrality of the dissertation literature review in research preparation’, Educational Researcher, 34, pp. 3–15. Brace, I. and Bolton, K. (2022) Questionnaire design: How to plan, structure and write survey material for effective market research. 5th/Ian Brace, Kate Bolton. edn. London: Kogan Page. Braun, V. and Clarke, V. (2021) Thematic analysis: A practical guide. London: SAGE Publications Ltd. Bruce, C. S. (1994) ‘Research students’ early experiences of the dissertation literature review’, Studies in Higher Education, 19(2), pp. 217–229. Button, K. S., Bal, L., Clark, A. and Shipley, T. (2016) ‘Preventing the ends from justifying the means: Withholding results to address publication bias in peer-­review’, BMC Psychology, 4(1), pp. 59. Caretta, M. A. and Pérez, M. A. (2019) ‘When participants do not agree: Member checking and challenges to epistemic authority in participatory research’, Field Methods, 31(4), pp. 359–374.

79

80

Collecting your data

Chia, I., Ma, N. and Tay, J. (2022) Conducting Social Media Research Using an Online Forum: Researching Parent’s Perspectives on Online Schooling During the COVID-­19 Pandemic. London. Available at: https://methods.sagepub.com/case/social-­media-­research-­online-­ forum-­parent-­perspectives-­online-­schooling (Accessed: 14 August 2022). Ciesielska, M., Wolanik Boström, K. and Öhlander, M. (2018) ‘Observation methods’, in Ciesielska, M. and Jemielniak, D. (eds.) Qualitative methodologies in organization studies. Cham: Palgrave Macmillan, pp. 33–52. Cook, C. E. and Thigpen, C. A. (2019) ‘Five good reasons to be disappointed with randomized trials’, Journal of Manual & Manipulative Therapy, 27(2), pp. 63–65. Denney, A. and Tewksbury, R. (2013) ‘How to write a literature review’, Journal of Criminal Justice Education, 24, pp. 218–234. Dunning, T. (2012) Natural experiments in the social sciences: A design-­ based approach. Cambridge: Cambridge University Press. Edwards, A., Housley, W., Williams, M., Sloan, L. and Williams, M. (2013) ‘Digital social research, social media and the sociological imagination: Surrogacy, augmentation and re-­orientation’, International Journal of Social Research Methodology, 16, pp. 245–260. Fink, A. (2013) Conducting research literature reviews: From the internet to paper. 4th edn. Thousand Oaks, CA: SAGE Publications, Inc. Foulsham, M., Hitchen, B. and Denley, A. (2019) GDPR: How to achieve and maintain compliance. 1st edn. London: Routledge. Giles, J. (2005) ‘Internet encyclopaedias go head to head’, Nature, 438(7070), pp. 900–901. Hine, C. (2020) Ethnography for the Internet: embedded, embodied and everyday. 1st edn. London: Routledge. Hyman, M. and Sierra, J. (2016) ‘Open-­versus close-­ended survey questions’, NMSU Business Outlook, 14, pp. 1–5. Johnson, R. L. and Morgan, G. B. (2016) Survey scales: A guide to development, analysis, and reporting. New York: The Guilford Press. Kaufman, M. R., Wright, K., Simon, J., Edwards, G., Thrul, J. and DuBois, D. L. (2022) ‘Mentoring in the time of COVID-­19: An analysis of online focus groups with mentors to youth’, American Journal of Community Psychology, 69(1–2), pp. 33–45. Knopf, J. W. (2006) ‘Doing a literature review’, PS: Political Science & Politics, 39(1), pp. 127–132.

Collecting your data

Kozinets, R. V. (2020) Netnography: The essential guide to qualitative social media research. 3rd edn. London: SAGE Publications Ltd. Krueger, R. A. (2015) Focus groups: A practical guide for applied research. Focus groups: A practical guide for applied research. 5th edn. Thousand Oaks, CA: SAGE Publications, Inc. McIntosh-­Scott, A., Mason, T., Mason-­Whitehead, E. and Coyle, D. (2014) Key concepts in nursing and healthcare research. SAGE key concepts Los Angeles, CA: SAGE. Nyumba, T. O., Wilson, K., Derrick, C. J. and Mukherjee, N. (2018) ‘The use of focus group discussion methodology: Insights from two decades of application in conservation’, Methods in Ecology and Evolution, 9(1), pp. 20–32. Peters, U. (2022) ‘What Is the function of confirmation bias?’, Erkenntnis, 87(3), pp. 1351–1376. Pink, S., Horst, H., Postill, J., Hjorth, L., Lewis, T. and Tacchi, J. P. (2016) Digital  ethnography: Principles and practice. London: SAGE Publications Ltd. Ridley, D. (2012) The literature review: A step-­by-­step guide for students. 2nd edn. London: SAGE Publications Ltd. Robson, C. and McCartan, K. (2016) Real world research: A resource for users of social research methods in applied settings. 4th edn. Hoboken, NJ: John Wiley & Sons. Sloan, L., Morgan, J., Burnap, P. and Williams, M. (2015) ‘Who tweets? deriving the demographic characteristics of age, occupation and social class from Twitter user meta-­data’, PLoS One, 10, p. e0115545. Snelson, C. L. (2016) ‘Qualitative and mixed methods social media research: A review of the literature’, International Journal of Qualitative Methods, 15(1), pp. 1609406915624574. Vindrola-­Padros, C. and Vindrola-­Padros, B. (2018) ‘Quick and dirty? A systematic review of the use of rapid ethnographies in healthcare organisation and delivery’, BMJ Quality & Safety, 27(4), pp. 321–330. White, H. and Sabarwal, S. (2014) Quasi-­ Experimental Design and Methods: Methodological Briefs – Impact Evaluation No. 8. Available at: https://EconPapers.repec.org/RePEc:ucf:metbri:innpub753 Wilson, R. E., Gosling, S. D. and Graham, L. T. (2012) ‘A review of facebook research in the social sciences’, Perspectives on Psychological Science, 7(3), pp. 203–220.

81

4

ANALYSING YOUR DATA David Wilkinson

IN THIS CHAPTER WE EXPLORE: ●

Dealing with data.

●

Analysing qualitative data.

●

Content analysis.

●

Narrative analysis.

●

Discourse analysis.

●

Grounded theory.

●

Analysing quantitative data.

●

Descriptive tools and techniques.

●

Describing data.

●

A note on inferential analysis.

●

Presenting your data.

●

Dealing with ever-increasing amounts of data – the role of data visualisation.

Dealing with data It is often said that by the time you reach the analysis stage of a research project, all the hard work has been done. Earlier chapters have guided you through the potential minefields of research traditions and developing research questions, reviewing the literature and deciding on appropriate research tools or instruments. The purpose of this chapter is to assist with interpreting and analysing the data you have collected. Data come to us in

DOI: 10.4324/9781003180159-4

Analysing your data

many shapes and forms. The role of analysis is to bring data together in a meaningful way and enable us as researchers to interpret or make sense of it for ourselves and those we are reporting to.

Coding and classifying data Before analysing data, it must be classified or coded in some way. In doing this, we are preparing the data for analysis. Some people refer to this as cleaning or organising data. For example, data could be organised by entering it into a spreadsheet or grouping it into batches relating to the date it was received. Another method of coding would be to convert the responses in a questionnaire into, for example, numeric form. The data devices, tools, and instruments we have covered in this textbook collect data that are either quantitative (numerically-based) or qualitative (visual or text/word-based). Some devices, such as a survey using both open-ended and closed questions, allows for the collection of both data types. Other research tools, such as experiment-based research, rely solely on quantitative data types or sources. These fundamental classifications of data type dictate the analysis tools and techniques we have available to us as researchers.

EXAMPLES OF QUANTITATIVE AND QUALITATIVE RESEARCH DATA Quantitative data ●

Number of students enrolled on a training programme.

●

Trends in sales figures.

●

Imports of cars into the UK from other parts of the world.

●

Number of travellers per year using a particular airport.

●

Vaccinations administered by doctors across a period of years.

Qualitative data ●

Attitudes of a group of people towards euthanasia.

●

A museum visitor’s interpretation of a painting.

83

84

Analysing your data

●

The interaction of children in a playground.

●

Documents tabled at a meeting.

●

Social media images/video of a music concert.

Analysing qualitative data Qualitative data are often collected through research interviews, focus groups, and observations. Qualitative data can also be collected via openended questions asked in survey-based research work. They enable the voices of those being researched to be heard (Creswell and Creswell, 2018; Chandler, Anstey and Ross, 2015). Qualitative data are usually analysed by subjecting it to some form of coding process. In order to provide some structure and meaning to qualitative data it must be coded or cleaned up in some way. For example, an interview may (and usually will) produce a great deal of information relating to given topics. How do we sort this? How are we going to compare it to other interviews? How do we draw themes from it? We present a number of approaches, below, to deal with the analysis of essentially qualitative data.

Content analysis An often-used tool to aid the sorting and analysis of qualitative data is to assess its content through a structured process. This technique is used in many research organisations as a way of classifying data and drawing themes from it. For example, a theme may emerge, from a number of interviews with musicians, that playing a musical instrument began as a hobby for them whilst at school. This type of response to the question: ‘When did you first show an interest in music?’ could be categorised as Statement from (A) As a child I loved to sit with the teacher at the piano and listen to her play. (B) Following a car accident, I spent a great deal of time in hospital, and I found music helped pass the days. (C) I joined a music club at college.

Code Child Hospital College

Figure 4.1 Sample categories that may emerge from the data  when exploring important memories linked to music

Analysing your data

‘school’. The number of categories or themes which may emerge from your data will depend on a number of variables such as the amount of data collected and the breadth of views. As you begin to code your data you will discover that many categories will initially be created. However, the purpose of creating these categories is to reduce the data – so categories may need to be subsumed into super-categories in order for the reader to digest the information quickly. As an example, statements ‘A’ and ‘C’ in the samples within Figure 4.1 could be placed in the super-category of ‘During time spent in education’. The number of categories will depend on the amount of data you have and the requirements of those reading the eventual report. For example, if they are concerned with detail, more categories may be necessary, whereas if they want a broad overview, fewer categories may be more appropriate. When developing codes, it may be useful to take a sample of your interviews and then develop a coding frame from them. A good yardstick is to attempt to develop a framework from approximately 20–30 per cent of your interviews. However, if this involves a great number of interviews, you may notice recurrent and similar themes emerging by transcript 7 that are not expanded upon in subsequent transcripts. If This is the case, you will need to exercise your discretion as to whether the analysis of further transcripts is appropriate. 1 Select a sample of your interviews.

2 Read through the sample transcripts several times.

3 Identify an exhaustive list of emerging themes/categories and number these so that you can reference them quickly and easily.

4 Group linked categories into super-categories. An example of a supercategory might be ‘sport’, and categories within this might include ‘football’, ‘tennis’, ‘swimming’.

5 Create a coding frame reference by providing examples from the interviews of all your themes/categories into one document or spreadsheet.

6 Apply your coding frame to each of your remaining interview transcripts. Figure 4.2  Analysing content: Stages in developing a coding frame

85

86

Analysing your data

Carrying out a content analysis of your data usually involves a number of stages, in order to establish content categories as well as defining the value and impact of the categories or themes that emerge through the process. We’ve identified, below, the key stages involved in developing a robust coding framework through which to analyse the content of qualitative data sets.

Concept-driven and data-driven coding Researchers can prepare for the analysis of qualitative data with categories or classifications pre-defined. These may be dictated by a research sponsor or client for example. Pre-defined categories may also be based on understanding of the topic or research environment by the researcher, e.g. where the literature in the field of study may direct a researcher towards certain topics or themes. Pre-defined categories or focal points for codes within the data is often referred to as ‘concept driven’. The alternate, where categories or codes emerge through analysis of the data, is termed ‘data driven’ (Bouvier, 2022).

Guides to help you code There are many coding guides or frameworks that researchers can draw from in order to provide meaning and context to categorisations and classifications. For example, affective approaches to coding explore peoples’ emotions, values, and conflicts, as well as other subjective qualities of human experience. Other mechanisms for coding also include evaluative codes that might assign some element of value judgement about merit, worth, or significance of particular experiences or events (Saldaña, 2015).

Qualitative data analysis software Analysing qualitative research materials requires researchers to become immersed in their data in order to fully explore and appreciate its context and meaning. This process of becoming immersed in the data can

Analysing your data

be enhanced through using qualitative data analysis software. Although some believe that this is a fairly recent advancement in working with such data, some of the leading experts in qualitative data analysis recognised its potential and value decades ago. For example, the work of Miles and Huberman in the early 1990s articulated that the flexible, recursive, and iterative capabilities of software packages provided advanced opportunities for researchers to rapidly assess and re-assess rich text-based data sets (Miles and Huberman, 1994). Since this time, a number of advanced software packages have been developed to support qualitative researchers in dealing with the process of coding, classifying, categorising, and valuing data.

NVIVO (https://www.qsrinternational.com/nvivoqualitative-data-analysis-software/home) NVivo is a software program specifically designed to support the analysis of qualitative data. It has the ability to store multiple types of data within its system. These can include text-based files from interviews, focus groups, open-ended responses from surveys, social media posts, and textbook/journal article content. It can also hold other types of qualitative data such as audio, video, and image data. The software acts as a filing cabinet for all qualitative research data and, through extensive coding and classification procedures, can produce rapid assessments of large data sets. (Bazeley and Jackson, 2019)

Narrative analysis When dealing with qualitative research data, researchers are often tasked with telling a story with the data. Some tools or instruments (such as research interviews, focus groups, observations) contain rich stories or accounts. These can concentrate on an experience of a shopping visit, or the view on the level of service in a hotel following an overnight stay for example. A narrative analysis of data in situations like this allows researchers to better understand the overall experience and to explore

87

88

Analysing your data

the underlying elements that contributed to it (Clandinin, 2007; McIntoshScott et al., 2014). Narrative analysis has become a useful tool in visually interpreting research data as the process helps to structure and blend language, numbers, and graphics into accessible, widely understood outputs (Durante, 2019). Narrative analysis differs from other ways of dealing with qualitative data in that it embraces a distinct set of themes that define it as a valid analytical device. These include: ●

●

● ●

●

Narrative analysis maintains an interest in people’s lived experiences and an appreciation of the explanation of the nature of that experience. Within this approach there is a desire to empower research participants and allow them to contribute to determining what are the most salient themes in an area of research. Narrative analysis has an interest in process and change over time. Narrative analysis concentrates on the self and representations of the self. Within narrative analysis there is an awareness of the effect of the researcher as storyteller. (Elliot, 2005, p. 6; Phoenix, Smith and Sparkes, 2010)

Narrative analysis has a central focus on the story and the presentation of the story. The stories are analysed to reveal interpretations and perspectives on the social world (Mannay, 2015). In essence, they are concerned with the meanings and ideology the story conveys; the techniques and communicative devices the storyteller uses; and how the story links with the cultural and historical context within which it is told (Denscombe, 2017, p. 292)

Digital narrative – digital storytelling Narrative analysis is commonly produced as written accounts of experiences and interpretations of the social world. However, digital (visual) narrative or storytelling is a powerful tool to help explore understandings and perspectives from the position of participants in a research project (Alexander, 2017). Purists in storytelling technique cite Joe Lambert as

Analysing your data

the guru of this research or evaluative device. He developed seven steps for effective storytelling that help subjects/participants to engage with the process of telling their own story and exposing the impact that it has secured. Using this process, the control and ownership of the data used to produce the story, and the telling or sharing of the story, is assigned to the participant. This affords the researcher the role of storytelling guide and facilitator – helping the participants to explore what is important to them in the story (Lambert, 2018). The technique has been used in a number of settings to provide a ‘voice’ for those being researched and allows stories to be told that are related to real-life, lived experience. One example of this is the toolkit designed by Action for ME (Myalgic Encephalomyelitis or Chronic Fatigue Syndrome). This workbook provided M.E. sufferers with the tools, and the confidence, to tell their stories so that it helped others to understand the impact of M.E. (Action for M.E., 2014)

Discourse analysis Discourse analysis attempts to unpack text and language by setting it within a social, cultural, and political framework. This makes it different from other forms of analysis we might use with qualitative data, as it not only identifies what is within the data but also brings into focus what is absent. It includes explicit evidence, but also considers what is not said (and implied) (Denscombe, 2017, p. 289). Discourse analysis is particularly useful for specific forms of research endeavours. For example, you might wish to explore or examine the effects and impacts of different kinds of language used in a particular environment or setting. You may be interested in the cultural codes, rules, and conventions that are present when groups or individuals communicate. Another valuable use for discourse analysis would be to examine how language use is linked to its social, political, and historical context. This was the focus for work that explored the UK’s decision to withdraw from the European Union in January 2020. Here researchers examined how discourse influenced the decision to withdraw and what discourse emerged as a result of it (Koller, Kopf and Miglbauer, 2019).

89

90

Analysing your data

FRAMEWORK FOR DISCOURSE ANALYSIS Discourse analysis is based on three stages: 1. Analysis of the text – including the exploration of language and how it is structured in the setting being analysed. 2.

Analysis of practice – which includes how those being researched produce, assess, interpret, and consume text.

3. Analysis of social context – which includes exploring issues linked to power embedded in the text, and how this impacts locally and in wider society. (Fairclough, 2003)

Critical Discourse Analysis One feature within all applications of discourse analysis as a research analysis tool or device is its concentration on what people say in social and cultural contexts. A central aim, therefore, is to focus on how language is used in real-life situations. There are a number of modes through which discourse analysis can be explored; one of the most popular is Critical Discourse Analysis. There are two dimensions to Critical Discourse Analysis: The event or topic being examined (e.g. equality and diversity), and the way language is used within a particular social institution (e.g. in a school or college setting). In essence, the approach is ‘critical’ because it compares the researched study of language with a study of its context (and this can be societal, political, economic, for example) (Bloor and Bloor, 2007).

ADVANTAGES OF DISCOURSE ANALYSIS ●

Discourse analysis helps researchers uncover the motivation behind a text by allowing them to view a problem from different perspectives or vantage points.

●

As a research device it is useful for studying the underlying meaning of a spoken or written text as it considers the social and historical contexts.

Analysing your data

●

As a structured mechanisms for assessing text or language used it allows researchers to better understand the function of language and how discourse can be used to foster positive social change. (Wooffitt, 2005; Rapley, 2018)

Grounded theory Grounded theory is a research tool or instrument that can be used to analyse qualitative data (although it can consider, as part of the process, quantitative data such as performance-related information and health-related metrics). It is centrally concerned with the emergence of patterns in data, and the subsequent development of theories from that data. As an approach to research, it was developed in the US by Glaser and Strauss in the 1960s (Glaser and Strauss, 1967). This method of analysis concentrates on formulating a theory around data, thereby ‘grounding’ the research in actual data. Then additional cases can be examined to see if they are relevant and can add to the original theory (Urquart, 2012). Since its introduction as an approach to dealing with essentially qualitative research data, grounded theory methodology has been adapted and now has multiple variants all with unique interpretations of how theory can be ‘grounded’ in the data it emerges from (Bryant and Charmaz, 2019). Despite amendments and changes to the theory over the last half-century, all approaches agree that: (1) theory construction is a central element of grounded theory, (2) the operating principles of grounded theory differ from those typically associated with quantitative research, and (3) grounded theory emerges from rigorous data analysis, not from adopting theories that are already in existence (Corbin and Strauss, 2014).

A GROUNDED THEORY FRAMEWORK FOR ANALYSING DATA 1.

Data should be collected and analysed simultaneously.

2. Data should be analysed separately from a traditional literature review as this may introduce bias in theory generation from the data (Glaser, 1998).

91

92

Analysing your data

3. Researchers should endeavour to create analytic categories early in the research process. 4. Analysis should begin early in the process and continue throughout. 5. Take samples from data collected in order to develop multiple ideas and generate meaningful theory. (Charmaz, 2015)

Analysing quantitative data Quantitative data are produced by closed questions in surveys, data produced via experiment-based research, and the conversion of respondent data to numeric form. Quantitative data can be analysed in a number of ways, including describing characteristics of the data (descriptive analysis), and drawing conclusions from the data and making judgements about it (inferential analysis). We provide some of the traditional methods below of dealing with research data typically collected and collated for most socially-framed research projects.

DESCRIPTIVE ANALYSIS OF DATA As the name suggests, this type of analysis describes data. With descriptive techniques you can summarise, codify, and visualise collected data in ways that allow you to show patterns in the data. Descriptive analysis of data usually precedes subsequent inferential analysis of data.

INFERENTIAL ANALYSIS OF DATA Inferential analysis of data utilises more advanced tools and techniques to interrogate and explore data than are provided by descriptive methods. Inferential techniques are often used to compare the differences between groups studied in a research

Analysing your data

project (for example, those who have undertaken a training programme, compared to those who have not). Inferential statistics can be powerful when used to make generalisations about a larger population (such as all university students), based upon a smaller number of research project participants or subjects (in this case, a selection of university students).

Quantitative data analysis software There are numerous software packages available to help support the analysis of quantitative data. They can help you to describe your data, find patterns in it, test it to ensure it is representative of similar or wider groups of respondents, and produce visualisations from it. Microsoft Excel provides solid and useful functionality for researchers who are required to deal with numeric research data. Data analysis add-ins allow more advanced descriptive analyses of data to be carried out. In addition, some of the standard statistical (inferential) analysis techniques can also be deployed via Excel. The standard statistical package used historically in social science research work, to deal with quantitative data, is Statistical Package for Social Sciences (SPSS).

SPSS (https://www.ibm.com/products/spss-statistics) SPSS means ‘Statistical Package for the Social Sciences’ and was first launched in 1968. Since SPSS was acquired by IBM in 2009, it has been officially known as IBM SPSS Statistics, but most users still just refer to it as ‘SPSS’. It is a combined suite of software programs that analyses data related to research projects typically carried out in the social sciences. It offers a fast-visual modelling environment that ranges from the smallest to the most complex models. The core functionalities offered in SPSS are: ●

Statistical analysis of quantitative data, including frequencies, cross-tabulations and associated bivariate analysis.

93

94

Analysing your data

●

Modelling from data, enabling researchers to build and validate predictive models using advanced statistical procedures.

●

Text analysis, providing an opportunity to link and code numeric data with open-ended elements in surveys and interviews to provide additional context to the analysis.

●

Visualization designer, allows researchers to use their data for a variety of visual representations. (Pallant, 2020; Field, 2017)

All of the software packages available can potentially save valuable researcher time and produce analyses that accurately inform, support, and guide analysis. It can be easy to be blinded by the power of such packages and not fully understand the mechanics of the analyses they perform. For this reason, we’ve outlined below how to perform some of these ‘by hand’ in the sections exploring descriptive statistics. We have also explained some of the less complex processes and formulations conducted by inferential analysis techniques.

Academic Often, academic researchers work from a particular discipline area or perspective. This, in turn, influences the approach to analysis they adopt in their work.

Industry Commercially driven research work is heavily influenced by the impact on the ‘bottom line’ and might often favour quantitative indicators of research effect or impact.

Public sector Public sector research has historically veered from being tied to a particular approach to collecting and analysing data (whether that be quantitatively based or qualitatively focused). The importance is often influenced by the type and scope of research work being carried out.

Analysing your data

Society To have impact and resonance in societal terms, research is often converted to a narrative or story. This allows research work to connect with users and report readers.

Descriptive tools and techniques There are many ways to analyse quantitative data. A key concern here will be a reference to the knowledge of your audience. For example, an investigation of admissions into hospital could be conducted by collecting and presenting data on the number of admissions in a given year. However, the analysis might include a breakdown of admissions by gender and a comparison of recent years (see the examples below). These data may have been collected as part of a larger research project examining the management and performance of Paperfield Hospital, or they could have been obtained from a nationally available database relating to hospital admissions.

Figure 4.3  Charts showing admissions to Paperfield Hospital (year and gender)

95

96

Analysing your data

Another way of reporting or analysing these types of data would be to indicate the percentage of male and female admissions in particular time periods, as displayed in the charts below. This presents a useful and visual way of showing any increase or decrease in male and female admissions.

Figure 4.4  Admissions to Barnswell Hospital (2022 and 2023 compared)

Essentially, the types of interpretation charted and visualised above provide a mechanism through which to describe your data. They provide a way of reducing data into accessible and easy-to-understand summaries. Further exploration of the headline information presented above might reveal various sub-categories or classifications of the data – such as duration of admission, department admitting the patient, doctor in charge of the patient, site of admission, and so on. This may well be useful and informative, but it could confuse those reading the output of your research if they are merely concerned with the number of patients admitted. For those who are interested in the detail of the numbers admitted to hospital, you may wish to produce a separate report or a technical addition detailing the additional breakdown of the data. For example, you may wish to explore your data further to establish the ages of those admitted. Suppose that 50 women were admitted into a small department in Paperfield Hospital in 2022 and the data relating to their ages are made available to you. It might look like the data set below. What can you do with these data? When they are simply presented as in Figure 4.5, they are difficult to interpret. You could begin by taking the data and listing the ages from the highest to the lowest. However, this would not add a great deal to the analysis of the data (although it would

Analysing your data

50

49

33

38

60

63

64

70

81

45

26

20

19

45

58

56

22

33

45

47

84

30

28

29

37

42

41

62

26

30

46

48

45

39

25

66

45

50

49

33

38

60

63

64

70

81

44

26

20

19

Figure 4.5  Age of admissions (female) to Department A in 2022

allow you to quickly establish the oldest and youngest female admitted). A helpful way to present the data would be to produce a ‘tally chart’ indicating how many times each age appears on the list. This can reveal some interesting information about the data. Age

19

20

22

25

26

28

29

30

33

37

38

39

Number of patients

2

2

1

1

3

1

1

2

3

1

2

1

Age

41

42

44

45

46

47

48

49

50

56

58

60

Number of patients

1

1

2

4

1

1

1

2

2

1

1

2

Age

62

63

64

66

70

81

84

Number of patients

1

2

2

1

2

2

1

Figure 4.6  Tally chart of female admissions to Department A in 2022

97

98

Analysing your data

Whilst producing a tally chart (or frequency chart, as it is also known) helps you to analyse the age data, it still remains difficult to draw any useful meaning from it. Reducing the data further into larger categories or ‘chunks’ may help, and the use of age ranges may assist with this. When grouping data into ages you can cluster them in a number of ways. However, it is customary to use five-year categories, as shown in the example below.

Age Number of patients

Age Number of patients

19 to 23

24 to 28

29 to 33

34 to 38

39 to 43

44 to 48

49 to 53

5

5

6

3

3

9

4

54 to 58

59 to 63

64 to 68

69 to 73

74 to 78

70 to 83

2

5

3

2

0

3

Figure 4.7  Grouping of ages – female admissions to Department A in 2022

Describing data The categorisation (or ‘chunking’) of the data in the above examples now allows us to see that, of the women admitted to this department, more were aged between 44 and 48 than any other grouping. This statement refers to the age range with the most occurrences. This is also known as the mode age range.

The mode, the median, and the mean If you were interested in establishing the age range that was the centre of all your ranges, this is known as the median age range. To find the median, you would list your ages from highest to lowest and count from each end until you reached the middle. In this case the median age is 45. Where there is an even number of values (ages in this case), the median is the average of the two mid-points (45 + 45 divided by two equals 45).

Analysing your data

THE MODE The mode of a group of data is the most frequently occurring value. For example, in the results of an examination, it would be the most often occurring grade.

THE MEDIAN The median is the value that separates the upper half of a list of values from the lower half. The median is, therefore, the midpoint in an ordered list of values.

Whilst this is a useful exercise in determining the middle value, it is still a time-consuming exercise to perform. First of all, you need to rank or list your ages in order and then you must establish your middle point. Another way of calculating the middle of a set of ages (or values) is to use the mean value. The mean, or average as it is also known, is calculated by adding together all the ages and dividing that result by the number of women admitted. Therefore, the total of all the ages is 2264 divided by the number of women admitted (50) equals 45.28. The mean, or average, age for the women admitted to this department in 2022 was 45. The mode, median, and mean are all known as measures of central tendency. They provide single values that best describes the group.

MEAN The mean is defined as the sum of the values divided by the total number of values. For example, the mean following exam results would be: Exam result

45

67

70

55

42

78

59

Total

Mean =

45

+67

+70

+55

+42

+78

59

= 419

We then devide the total (419) by the number of results (7) = 59. Mean exam result = 59

99

100

Analysing your data

Following this, you may wish to explore further the frequency of ages of female admissions during 2022. A visual way of doing this would be to develop a frequency distribution graph, as shown below. Microsoft Excel can produce these very easily, as can SPSS and other quantitative data analysis software packages.

Figure 4.8  Frequency of ages of female admissions during 2022

This chart shows that most of the women admitted to Department A were under 46 years of age. You will notice that there is more activity in the chart between the ages of 19 and 46 (there are more of the female admissions falling within these categories). This is known as a skewed distribution, whereby the results are grouped to one side of the chart or graph. In many studies, researchers might expect to find a distribution of data where most of the values group around the middle of the chart or

Analysing your data

graph. This is known as a normal distribution. If this were the case you would notice that the figures for the mode, the median, and the mean were all similar in value. As an example, you might expect a normal distribution to occur when looking at the exam results of undergraduates. This might look something like Figure 4.9.

Figure 4.9  Distribution of exam results

Standard deviation From normally distributed data you can measure the distribution of values around the mean. Using the exam score example, this would be useful as it would allow you to establish the degree of dispersion or difference between the scores. If the standard deviation is large then the scores vary considerably, whereas if the standard deviation is small then the scores are more tightly clustered and closer together. The standard deviation is a key basic statistical technique, which forms the basis for many more advanced techniques. In essence, the standard deviation provides an average of all the deviations from the mean. There are a number of ways to calculate the standard deviation, most software packages utilise a formula and process similar to the one outlined below.

101

102

Analysing your data

Figure 4.10  Working out standard deviations

Having established the standard deviation for the exam scores, we can see that there is some dispersion among the results. In addition, if the distributions of scores are normal (as shown in Figure 4.11 below), certain statements can be made about the results. In a normal distribution the range from –1 standard deviation to +1 standard deviation contains 68 per cent of the results, the range from –2 standard deviations to +2 standard deviations contains 95 per cent of the results, and the range from –3 standard deviations to +3 standard deviations contains 99 per cent of the results. The standard deviation is a useful way of comparing across different sets of data. For example, it could be used to compare the variability in different exam results – such as Law and Accountancy – among a cohort of students. It is also used as a basis for many more detailed statistical analyses of your data – for example, inferential analysis discussed later in this chapter.

Analysing your data

Figure 4.11  Distribution of exam results showing standard deviation

STANDARD DEVIATION The standard deviation is a tool used to measure dispersion. The standard deviation shows the relation a set of values has to the mean. Assuming that the distribution of scores is normal, certain statements can be made about the data (69 per cent of values fall within 1 standard deviation of the mean, 95 per cent lie within 2 standard deviations of the mean, and 99 per cent lie within 3 standard deviations).

Associating data You may find that with some of your data you wish to explore possible relationships between two different sets of data (or variables, as they are also known). This is often referred to as correlation research. There are numerous techniques available for exploring the relationships between variables. Two of the most commonly used methods for exploring relationships between variables are Pearson’s Product Moment Correlation Coefficient and Spearman’s Rank Order Correlation Coefficient. Both of these analyses indicate whether an association is positive (with a maximum value of +1) or negative (with a maximum value of –1). It is usually the case

103

104

Analysing your data

VARIABLES A variable is one of the factors in your data. For example, height, weight, or test scores might be variables. Variables can be either independent or dependent. A dependent variable is one that you would expect to see change following an increase or decrease in an associated independent variable. For example, you might expect to see a change in exam results (dependent variable) following an increase in lectures attended (independent variable).

that scatterplots are used to show the results of the analysis. The example plots here show three types of relationship between the number of hours of TV watched and the age of the viewer. In the first example there is a strong positive relationship, represented by plots moving upwards as they progress from left to right. The second example shows a strong negative relationship between the two data sets, indicating that (with this particular sample of data) older viewers watch less TV. This is demonstrated by the plots moving in a downward direction as they progress from left to right. The final example does not appear to show any relationship between the two data sets as the plots do not move upward or downward as they progress from left to right.

Figure 4.12  Positive relationship (correlation) between two data sets

Analysing your data

Figure 4.13  Negative relationship (correlation) between two data sets

Figure 4.14  No relationship (correlation) between two data sets

Formula for correlating data The detailed formula of correlation research is not discussed here, but there are a number of excellent reference sources to help guide you through analysing your data using this, and other descriptive techniques (Winston, 2022; McFedries, 2019; Pallant, 2020). A simple example

105

106

Analysing your data

which outlines the formula and processes involved would be to explore the potential of a relationship between exam scores and the number of lectures attended. Using the Pearson Product Moment Correlation Coefficient approach (R), the analysis would be structured around using the formula shown in Figure 4.15.

Figure 4.15 An example using Pearson’s Product Moment Correlation Coefficient approach (R)

Analysing your data

The above manual calculation and assessment of correlation shows that there is a strong positive correlation between the two variables of exam score and the number of lectures attended. We probably expected this to be the case, and it is pleasing that the data confirmed this for us. However, researchers must take care when performing correlation analysis on collected data. The appearance of a relationship does not necessarily mean one exists – it does not indicate causation. In other words, correlation does not prove that one variable causes another to alter in value. A descriptive analysis or report of your data is a useful way of introducing the data to the reader. From a general analysis you can move on to a more detailed examination of your data (if this is appropriate, given the requirements of the reader or user of your research). Questions posed here may include: ‘What are the data saying?’ and ‘What do the data mean?’ Interpreting or questioning your data in this way often leads to making judgements or inferences about it, or (more often) the wider population from which it is taken from. These kinds of questions and interests are explored through the use of inferential data analysis.

A note on inferential analysis An inferential analysis of your data assists you in making conclusions about the data by performing certain operations on it. With inferential analysis you are inferring from your sample data (e.g. the exam scores) what the population scores are (say, the scores for an entire group of undergraduates).

SAMPLE A sample is a selection which is taken from a group; it is usually considered to be representative of that group. As a result, the findings from the sample can be generalised back to the larger group.

POPULATION A population is a group who share the same characteristics. For example, a population could be members of a club, nurses, students or children.

107

108

Analysing your data

The major difference between inferential analysis and descriptive analysis is that with descriptive techniques you are merely describing the data as it is represented to you. With inferential analysis, you test or perform some operation on the data in order to make conclusions about it.

Statistical significance One popular method of inferential analysis is to make judgements of the probability that the difference in, say, the mean scores in a law exam for a sample of male and female students is one that is representative of all male and female students on the law programme; or whether it is down to chance. In this way, we are said to be testing the significance of the difference in exam scores. In statistical terms, when we test for significance, we must first make a statement or hypothesis about the data.

HYPOTHESIS A hypothesis is a proposition or statement you wish to test with your data. They are commonly referred to as null hypotheses. These are negative statements which have to be disproved in order to validate the statement made.

For example, the hypothesis for the exam score instance could be: ‘There is no significant difference in male and female exam scores’. This statement is known as a non-directional hypothesis because it merely puts forward that no difference exists. The statistical tests to perform on this type of hypothesis are called two-tailed tests. However, if the hypothesis were stated as ‘males perform better than females in exams’ then the hypothesis becomes directional, and a one-tailed test is required. The requirements of a one-tailed test are stricter than those for a two-tailed test, as the latter is only concerned with proving a difference exists, while the former is concerned with exploring who the difference favours.

Analysing your data

Testing for significance There are a number of statistical tests available to researchers when exploring hypotheses. These include the t-test, Analysis of Variance (ANOVA), and the Analysis of Covariance (ANCOVA). These are more advanced statistical techniques and, although they are not detailed here, they are explored in traditional statistical texts (Pallant, 2020; Field, 2017; Rowntree, 2018).

Presenting your data As this chapter has shown, data can be presented in a variety of ways. When you report your data, it will be in one of two main forms: A table or a chart/figure. Tables often reproduce raw data. They should be clear and uncluttered. In research reports, tables are often used to present findings, emphasise a point made in the text or act as the starting point for a discussion or analysis of some aspect of the data. Even if your report has only one table, it should be clearly labelled with a title and reference number. Some organisations and institutions have a particular ‘house style’ for the presentation of data, so you may need to check this.

STATISTICAL SIGNIFICANCE Statistical significance refers to how much, for example, exam results for a group of students could be down to chance alone. If the results cannot be explained by chance, it is assumed that another factor, such as number of lectures attended, had an impact on the results. One of two significance levels are usually applied when testing for statistical significance – 0.05 and 0.01. These levels indicate degrees of confidence in the assumption that chance was not the cause. Whilst 0.01 is the stricter of the degrees, both can be understood as producing statistically significant results.

Charts or figures are more graphical representations of your data or the results of some analysis of it. These presentational tools also require

109

110

Analysing your data

careful and clear labelling. Presenting your data as a figure or chart may include constructing a histogram, bar chart or pie chart of your data. These, as shown earlier, are often used to present descriptive analyses of your data. More complex analyses of your data, such as an exploration of correlation, are best presented as scatterplots or line graphs.

TIPS FOR PRESENTING DATA ●

Remind the reader of the research question or questions when presenting data. This helps provide focus.

●

Move from the general to the specific. For example, indicate general findings before moving on to the more specific and detailed elements.

●

Keep linked data together in your report and deal with them in one chapter or section if possible. This aids the flow and structure of your report.

●

Keep tables and figures simple! Any detail should be provided in an accompanying key, or in an appendix at the back of the report.

Dealing with ever-increasing amounts of data – the role of data visualisation The total amount of data created, captured, copied, and consumed globally is forecast to increase rapidly, reaching 64.2 zettabytes in 2020 (a zettabyte is one billion terabytes) (Statistica, 2022). In order to make data accessible and useful, it must first be processed and prepared for general consumption. Preparing and condensing data for more general consumption is the purpose of data visualisation. At its simplest definition data visualisation is the graphical representation of data (Boy, Detienne, and Fekete, 2015; Knaflic, 2015; Kosara and MacKinlay, 2013). Data visualisation has the power to communicate complex information in an understandable, easy-to-access way and can allow users to interrogate it to suit a particular need (Durante, 2019). It has

Analysing your data

historically tended to focus on exploration and analysis, but now data can be deployed to tell rich stories through visual and imaginative ways (Feigenbaum and Alamalhodaei, 2020; Telling, 2017; McCandless, 2022; McCandless, 2021). It is generally recognised that data storytelling doesn’t necessarily have to follow a linear sequence of storytelling; it can also be achieved through interactivity inviting verification, new questions, and alternative explanations (Segel and Heer, 2010).

FIVE STEPS OF EFFECTIVE DATA VISUALISATION 1.

Analysing: Understand the data and what it is telling you. Avoid data pitfalls. Check the sources of your data – are they reliable?

2.

Building: Determine your goal and clarify who you are creating the visualisations for. Think about your audience and how they deal with data and usually consume it.

3.

Designing: Choose appropriate colour palettes (making sure they are colour-blind-friendly. Put labels on all of your charts, use widely accessible and readable fonts.

4.

Telling: Use storytelling to engage and encourage the reader. Use annotations where needed to give the reader/ viewer additional context and meaning.

5.

Sharing: Use the right tools for displaying data and make as much of it as shareable as possible. Dedicated websites make distributing and sharing visualisations much easier. (Dijk, 2022)

Summary Research data are collected in a variety of forms. In this chapter, we have identified a number of ways that it can be processed and analysed. We have categorised analytical tools or techniques as either essentially qualitative or quantitative as these relate to the type of data collected (visual,

111

112

Analysing your data

text, audio-based for qualitative, numeric for quantitative). Content analysis is a popular method for dealing with qualitative data and converting it to meaningful codes and categories. Narrative analysis explores stories in qualitative data; discourse analysis examines language and text use and explores its context and meaning. Grounded theory is an approach to analysing qualitative data that develops meaning and theory ‘grounded’ or emerging from the data. Quantitative data analysis tools have been presented that include descriptive tools or techniques. These seek to manage and reduce data by processes that include categorisation and classification. Typical ways to group and classify data include frequency counts, proportions or percentages, data distributions and deviations from the average or mean. Inferential analysis and making value judgements about data have also been briefly discussed. Finally, the chapter closes by looking at the emerging field of data visualisation, the creation of accessible charting and visual representation of complex data. In their work dealing with large and complex data sets, data visualisation experts have created innovative approaches to present data in meaningful and accessible ways.

References Action for ME (2014) Digital storytelling toolkit. Keynsham: Action for ME, p. 46. Alexander, B. (2017) The new digital storytelling: Creating narratives with new media. 2nd edn. Santa Barbara, CA: Praeger. Bazeley, P. and Jackson, K. (2019) Qualitative data analysis with NVivo. 3rd/edited by Pat Bazeley, Kristi Jackson. edn. Los Angeles, CA: SAGE. Bloor, M. and Bloor, T. (2007) The practice of critical discourse analysis: An introduction. London: Hodder Arnold. Bouvier, G.A. (2022) Qualitative research using social media/Gwen Bouvier and Joel Rasmussen. 1st edn. London: Routledge. Boy, J., Detienne, F. and Fekete, J.-D. (2015) ‘Storytelling in information visualizations: Does it engage users to explore data?’, Proceedings of the ACM CHI’15 Conference on Human Factors in Computing Systems, 1(February 2016).

Analysing your data

Bryant, A. and Charmaz, K. (2019) The SAGE handbook of current developments in grounded theory. London: SAGE Publications Ltd. Chandler, R., Anstey, E. and Ross, H. (2015) ‘Listening to voices and visualizing data in qualitative research hypermodal dissemination possibilities’, SAGE Open, 5, pp. 1–8. Charmaz, K. (2015) ‘Grounded theory’, The Blackwell Encyclopedia of Sociology, pp. 2023–2027. Malden, MA: Blackwell Pub. Clandinin, D. J. (2007) Handbook of narrative inquiry: Mapping a methodology. London: SAGE. Corbin, J. M. and Strauss, A. L. (2014) Basics of qualitative research: Techniques and procedures for developing grounded theory. 4th edn. Thousand Oaks, CA: SAGE Publications, Inc. Creswell, J. W. and Creswell, J. D. (2018) Research design: Qualitative, quantitative, and mixed method approaches. 5th edn. Los Angeles, CA: SAGE. Denscombe, M. (2017) The good research guide: For small-scale social research projects. 6th edn. London: McGraw Hill Education/Open University Press. Dijk, D. V. (2022) A Guide to Effective Data Visualization: Datylon. Available at: https://www.datylon.com/blog/guide-to-effective-data-visualization (Accessed: 23 August 2022). Durante, N. (2019) Data story: Explain data and inspire action throgh story. Oakton, VA: Ideapress Publishing. Elliot, J. (2005) Using narrative in social research: Qualitative and quantitative approaches. London: SAGE Publications Ltd. Fairclough, N. (2003) Analysing discourse: Textual analysis for social research. London: Routledge. Feigenbaum, A. and Alamalhodaei, A. (2020) The data storytelling workbook. London: Routledge. Field, A. (2017) Discovering statistics using IBM SPSS statistics: North American Edition. 5th edn. London: SAGE Publications Ltd. Glaser, B. G. (1998) Doing grounded theory: Issues and discussions. Mill Valley, CA: Sociology. Glaser, B. G. and Strauss, A. L. (1967) The discovery of grounded theory: Strategies for qualitative research. New York: Aldine de Gruyter. Knaflic, C. N. (2015) Storytelling with data: The effective visual communication of information. 1st edn. New Jersey: John Wiley & Sons.

113

114

Analysing your data

Koller, V., Kopf, S. and Miglbauer, M. (2019) Discourses of Brexit. 1st edn. London: Routledge. Kosara, R. and MacKinlay, J. (2013) ‘Storytelling: The next step for visualization’, Computer, 46(5), pp. 44–50. Lambert, J. (2018) Digital storytelling: capturing lives, creating community/ Joe Lambert, Brooke Hessler. 2nd edn. London: Routledge. Mannay, D. (2015) Visual, narrative and creative research methods: Application, reflection and ethics. London: Routledge. McCandless, D. (2021) Beautiful news. London: HarperCollins. McCandless, D. (2022) Rhetological Fallacies. Available at: https:// www.informationisbeautiful.net/visualizations/rhetological-fallacies/ (Accessed: 29 August 2022). McFedries, P. (2019) Excel data analysis for dummies. For dummies 4thedn. Hoboken, NJ: John Wiley & Sons, Inc. McIntosh-Scott, A., Mason, T., Mason-Whitehead, E. and Coyle, D. (2014) Key concepts in nursing and healthcare research. SAGE key concepts. Los Angeles, CA: SAGE. Miles, M. B. and Huberman, A. M. (1994) Qualitative data analysis: An expanded sourcebook. 2nd edn. Thousand Oaks, CA; London: SAGE. Pallant, J. (2020) SPSS survival manual: A step by step guide to data analysis using IBM SPSS. 5th edn. London: Routledge. Phoenix, C., Smith, B. and Sparkes, A. (2010) ‘Narrative analysis in aging studies: A typology for consideration’, Journal of Aging Studies, 24, pp. 1–11. Rapley, T. (2018) Doing conversation, discourse and document analysis. 2nd edn. London: SAGE Publications Ltd. Rowntree, D. (2018) Statistics without tears: A primer for non-mathematicians. Harmondsworth: Penguin. Saldaña, J. (2015) The coding manual for qualitative researchers. 3rd edn. London: SAGE. Segel, E. and Heer, J. (2010) ‘Narrative visualization: Telling stories with data’, IEEE Transactions on Visualization and Computer Graphics, 16(6), pp. 1139–1148. Statistica (2022) Volume of Data/Information Created, Captured, Copied, and Consumed Worldwide from 2010 to 2020, with Forecasts from 2021 to 2025: Statistica. Available at: https://www.statista.com/ statistics/871513/worldwide-data-created/ (Accessed: 23 August 2022).

Analysing your data

Telling, M. E. (2017) Digital storytelling toolkit. Keynsham: Action for ME. Urquart, C. (2012) Grounded theory for qualitative research: A practical guide. London: SAGE Publications Ltd. Winston, W. L. (2022) Microsoft excel data analysis and business modeling. 7th edn. Redmond, WA: Microsoft Press. Wooffitt, R. (2005) Conversation analysis and discourse analysis: A comparative and critical introduction. London: SAGE Publications Ltd.

115

5

WRITING IT ALL UP Dennis Dokter

IN THIS CHAPTER WE COVER: ●

Getting started: preparing your writing.

●

Tips for getting started.

●

Time management.

●

Formatting and structuring your writing: an example.

●

Fallacies (mistakes) to avoid when writing.

●

Thinking about your audience when writing.

●

Presenting your work to others.

●

A note on ethics when writing up.

●

Submitting your research report to others.

●

Styling your report.

●

Social media and publication.

● Storytelling. ●

Visualising your data.

●

Tone and voice.

Getting started: preparing your writing An important factor related to research is ‘writing’ it down. This process can come across as daunting as it is not solely applicable to people new to the field. Even people with many years of research experience can lack confidence in their work. It can also take different forms, depending on what was the purpose of your research and which sector you might represent. DOI: 10.4324/9781003180159-5

Writing it all up

Academic As an academic or student, you will often have to write papers, articles, run tests, and write a thesis or dissertation. This chapter will give you guidelines and tips on writing these while keeping your audience in mind. It will also give tips on how you can use new techniques to make your results more understandable.

Industry From an industrial point of view, you will have to write reports for your board or for managers. This chapter will give you the initial information you need about how to get started in writing it down. It is also relevant when you develop new products and services and you want to explain it not only internally, but also to an external audience.

Public sector Within the public sector it is very common to write reports in order to explain your decision-making process. This chapter will give you a number of guidelines to make sure that enough attention is being paid to keeping it understandable. It will also provide the necessary tools to see how you can implement other research results and make them relevant for your audience.

Society As a foundation or charity, you might commission or do your own research. This chapter will take an approach that can be used both by novices and by more advanced researchers. It will demonstrate how you can use different publication methods in order to widen your audience. It will also give society in general tips to look for when it comes to argumentative mistakes. It will teach us how we can look more critical at (news) articles and their claims so that we can assess the value and validity ourselves.

Writing still remains one of the main ways to communicate research. Whatever context you are researching in, some form of written output will be required. For example, there are the more conventional reports,

117

118

Writing it all up

dissertations and scientific articles we might think of initially, but also think of reports, product descriptions, and (social) media articles. New modes of communication can then involve using video or sound such as in podcasts or vlogs. For that reason, this chapter has been written to be applicable in whatever research setting you may find yourself, it will give the right tools. In a lot of the ‘how to do’ literature on research, there is limited attention given to advice on the writing itself. However, as much as people dread writing, they don’t want to waste valuable time reading about how to do it. This chapter offers a concise and practical approach to the process of writing as well as give examples of common fallacies, tone and voice, your audience, and different forms of presenting your work to others. If you adopt a more systematic approach to writing. It will also demonstrate other modes of publication and showing the role of ethics while writing and the potential of storytelling and data visualisation.

Tips for getting started The first question is: when should writing begin? It is important to note that you can start writing down as soon early as you can. It is not necessary to start writing up after the data has been collected and analysed at the end. This will make the process flow more easily and shows how your research develops over time.

START WRITING EARLY Writing up your research should start early and become a regular and continuing activity. It is also likely to be an iterative or cyclical process. That is, you will draft a section or chapter, then move on to some other activity and return one or more times to redraft your original version. This is partly because as the totality of the research thesis or report takes shape, what you have written in subsequent sections affects what you wrote earlier and necessitates changes in it. It is also the case that as your research proceeds you find out more, read more and change your mind about some things. (Blaxter, Hughes and Tight, 2010, p. 223)

Writing it all up

Many researchers experience motivational problems relating to the writing process. It is easy to underestimate the length of time that is necessary for writing, which can lead to problems with deadlines and the quality of reports due to rushed writing to meet a deadline. There is a link between ‘putting things off’, and a lack of confidence because people put pressure on themselves to get the writing perfect the first time. A lack of focus and knowledge of what is expected of you add to this pressure. Allocating plenty of time in your workplan for writing early on and throughout the process will reduce the amount of stress towards the end. Start by writing something down on paper and know that this will be of use to you in some kind of form later on. Use bullet points as a way to identify topics and then build on that. When having problems writing, it is a good idea to get your content laid out and then write what you know without the aid of notes, literature, and so on, and then build from that. This can help you to get the main themes down on paper. Another useful idea to get you focused is to decide on the number of words you will devote to each section based on the word limit. Even the most experienced of researchers don’t come up with the ideal version first time round. If you schedule writing into your workplan regularly, then it will gradually get easier.

An example for organising the write-up When producing your written account of the research work your emphasis should always be on organising your information closely in line with the structure of your report before writing. When determining what you want to write, it is important to pay attention at organising your information properly. The example below provides a good framework for the research report-writing process.

Stage 1 Write a draft contents list and break it down into chapters or sections, or both. Then label your sources, data and any other information in line with these. This labelling will be done continuously

119

120

Writing it all up

throughout the whole process as you build up more references, do more testing and developing, and capture more data throughout the research timeline.

Stage 2 Think of your report as a whole and mark out the order of ideas and the connections between them. If it is useful, draw a diagram showing these connections separately.

Stage 3 Within each chapter or section, decide on the themes, draw links, and number the themes to show the order in which they will be placed. Then add these themes to your contents list. Next, code each theme with a different shape and colour and mark your sources accordingly. Then pile them in the relevant order for each chapter.

Stage 4 You can start writing after stage 3, but if the report is particularly lengthy, it can be useful to sketch the framework of what you are going to write in the form of double page spreads. This provides a useful structure to write into and can aid with the logic and flow of your argument.

Stage 5 Writing. It wastes time to write in a less-focused way, hoping that something will come together. Even after these stages it is unlikely that you will be able to write the perfect version of your report the first time round. Decide what you want to say briefly under each section or heading and then build up the detail gradually from that. Writing is a process of progressive refining, and you should not underestimate the time it takes. Also, there is no set way to sequence your argument, so you have to choose the structure, linkages and cross-referencing that work best for each report. (Orna and Stevens, 2009)

Writing it all up

Using the approach above helps you to arrange and cluster sources, data, and information in such a way that you can concentrate your energies on writing. Whether writing for an essay, journal, book, strategy, policy, or report, this method can be applied to all types of writing outputs. Also, it can be applied to either a manual or computer database of sources. Do not see this framework to rigid but see it as something that can be adapted to you own needs. Even when you are more experienced, this method can help to improve the quality of your output.

Time management Having established when to start writing, discipline has to come into play in order to manage your research time efficiently and effectively in order to meet the deadlines. Break down the writing into key stages according to particular themes or sections and then set targets for each, setting aside a specific amount of time for each task. Observe how much you get written within this writing plan and then you can adjust it accordingly (Bell and Waters, 2018). This method gives you more focus. Of course, if you feel that you can add a lot more within each session then do so. Set time frames are useful when you are having problems with discipline and focus. In terms of appropriate time allocations, these are difficult to gauge at first, but they will be easier to estimate with practice when you get more of a feel for your pace of writing. When working on large research projects, it is very common to be working with multiple partners. These can be other colleagues from your own organisation, but they could also be partners from other sectors; for instance, when a company needs academic support for the development of a new innovative product or when a charity is working with the public sector to analyse societal challenges. When working together, knowing who will contribute what to the research will help identify the relevant timeline of the entire project which will also be the basis for your writing structure. If you are writing the report with multiple people, it should be made clear early on in the research who will contribute to each chapter or section. When there is a general project leader, they could provide the initial overview and divide the tasks of writing amongst the wider team within a workplan. When there are multiple work packages, their individual leaders can organise it within their team, as well as be held responsible

121

122

Writing it all up

within the wider project. Then relevant deadlines can be set to increase the chances of you all completing your tasks at the same time so that the report can be pulled together without major delays.

GETTING THE MOST OUT OF MANAGING YOUR TIME – SOME TIPS ●

First, make sure that you work in a place where distractions can be kept to a minimum because they reduce flow and efficiency in your writing. If you think that interruptions will be inevitable, then write a more detailed plan of key tasks that you have to do within each writing session. Therefore, you can keep track of what you are doing.

●

Write regularly so that you gain momentum – it will get eas-

●

Consider the quality of time and effort that you are putting

ier and quicker with practice. in – keep focused – and don’t kid yourself that because you have been sitting at your desk or computer for six hours that you have achieved a lot. Keep reviewing the outputs of your research sessions. ●

Do the more difficult subjects or tasks at the time of day that you know you work best. For some this is in the morning. A lot of my ideas unfortunately tend to flow most rapidly in the early hours of the morning!

●

Another time-saving tip: if you have access to a PC or a word processor is to write the bulk of your work at the computer. This will save a lot of time compared to writing hand-written notes and then typing them in word for word. This may seem strange at first, but not only will it save time but also mean that you can keep a closer eye on the word limits and cut and paste sections easily.

Whilst acknowledging that drafting and redrafting will play a part, aim to get each section to a good standard first time. It is always worth trying to produce the final version at the first attempt. To write with the idea that what is written will be redrafted encourages a degree of carelessness, which can produce drafts that require complete revision. On the other

Writing it all up

hand, if you aim to get it right first time, there is every chance that all that will be required is minor amendment. This is as much an attitude of mind as a matter of style.

Formatting and structuring your writing: an example It is important to have make sure that what you are writing down happens in a structured manner. Below are some general guidelines that can be adapted depending on the requirements of your audience, and the nature, detail, and context of the research being undertaken. These guidelines will offer a detailed breakdown of how a report should be structured and are based on a typically used and styled format. They are presented as a comprehensive list of sections that follows the academic model but can be adapted to suit the purpose of the research. In short, a write-up always consists of a preliminary part, the main text and the end matter (Bell and Waters, 2018). These checklist and guidelines are not only useful reminders of what needs to be done; it also promotes self-regulation and focus (Jagaiah, Howard and Olinghouse, 2019).

Preliminary part Preliminary sections of typical research reports include the following elements:

Title This should reflect the contents of the report but has to be brief. Some researchers find it necessary to include a sub-heading in order to give more detail.

Abstract This is a one-page summary of the work of approximately 250–300 words. It should give the broad aims of the work and conclusions. People often only read this to see if the report will be of any use and so it needs to be written well.

123

124

Writing it all up

List of contents List paragraphs or, if it is a lengthy report, each chapter and paragraph. The appropriate page numbers should also be given.

Lists of figures and tables On the figures/tables themselves make sure you acknowledge the sources, and label axes on graphs, etc. correctly.

Preface This is a personal statement from the author who mentions something about the events leading up to the research and the significance of it.

Acknowledgements Acknowledge the people who have supported you during the research. Also acknowledge those people or organisations that have cooperated with it (Denscombe, 2017). Do not be alarmed by the long list presented here. As mentioned before, this is more of an example of a comprehensive list that you might use for a comprehensive report as a researcher for a university, not an obligated one. For instance, when you are doing a small market research as a company, e.g. examining how certain roads might become safer as a city council or trying to investigate why certain citizens are more at risk for homelessness as a charity, it might not be necessary to have a preface and acknowledgements. This preliminary part is mainly to provide an overview of what will be in the report and set the scene for the context in which it was written. It is within the main text that you want to give a proper introduction. This should given an account of: what work has been done before; why you had to do the research; how you did it; what your end results are; and what you intend to do with the results. This is the bulk of the work and here is where you will spend most of your time.

Writing it all up

The main text The main body of the research report contains the substance of the research effort. It covers a number of sections you might typically expect to see. These are:

Introduction Give some background to the research and outline the purpose. Then aims, objectives, and hypotheses should be set out, along with an indication of the scope of the project and the main gaps in knowledge that will be addressed.

Literature review This should cover background theory/knowledge and also key concepts and definitions, and it will show how the research fits in with these. However, it should not be an account of everything that you have read which vaguely relates to the research. You have to be selective here.

Methods Here you show how the methods were used to address each of the objectives. Therefore, it should include a justification of the overall research design and methods – for example, a case study or survey. Also, address which type of instruments were used to gather the data – questionnaire, interview, etc. In addition, list what type(s) of data were gathered – quantitative, qualitative, or both? If it is a survey, you should cover things like the population surveyed and the sample size. If it is related to product development, justify your selection of materials or processes and how you plan to apply them. If appropriate, you should give an account of the research at each stage and mention any problems that may have affected the results. Finally, you should say how the data were analysed – be it statistically or otherwise. Also acknowledge the limitations of the research in relation to such things as time constraints and accuracy.

125

126

Writing it all up

Results Here you should set out the results in an organised way – for example, in relation to a particular theme or issue. The more academic the report, the more specific.

Analysis and discussion This should interpret the results and highlight the most significant ones and perhaps deduce things from them. Here you discuss the findings in relation to the back ground theories and knowledge and the original aims of the project. Some researchers prefer to have the results, analysis, and discussion as one section because they think that it gives a more rounded summary of the research. This is a matter of taste and convention.

Conclusions and recommendations Here the researcher needs to assess the extent to which the original goals of the research have been met. It will also reflect on the methods used. It might recommend action or show how it has increased our understanding. Consider the questions remaining or generated by the research and recommend further research. It is important, at these stages, to draw together the threads of the research in order to arrive at some general conclusion and, perhaps, to suggest some way forward. Attempt to make them positive and constructive (O’Leary, 2014; Denscombe, 2017).

The end matter Regardless of whether you make use of all the above guidelines for the structure of the report, it will remain important to have the proper references of other literature sources, project prototypes or data sets as well as overviews of the design conditions and test results in appendices. This allows the readers to be able to validate and critically analyse the research you have done. It provides the necessary transparency at the end of the report.

Writing it all up

TYPICAL END MATTER CONTENTS IN A RESEARCH REPORT Appendices Here you should include anything that might interrupt the flow of the arguments in the report. Material included here is generally too detailed for inclusion in the main report but should be available for examination by readers to show them the material or instruments you have used – e.g. questionnaires and interview schedules.

References This should be an alphabetical list of all of the authors cited or referred to in the text. It should not be confused with a bibliography, which is a list of everything that you have read during the research. The Harvard System is the most widely used. Within this system the ideas of the author are referred to in summary or by direct quotation. Then, in the back of the report, the authors or organisation are listed in alphabetical order. The next section deals with this in more detail.

Fallacies (mistakes) to avoid when writing Fallacies are argumentative mistakes you can make while you are writing your research down. These are very common to a lot of research work and take place across the spectrum of publication modes, whether academic articles or (social) media posts. Below are some examples of commonly encountered fallacies, showing the scope and breadth of material currently in existence.

MISTAKES TO AVOID WHEN WRITING Begging the question: When someone restates a claim via a different phrase, that is called circular. Scare tactics: When there is the aim to frighten people into agreeing with the arguer by threatening them or predicting unrealistically dire consequences.

127

128

Writing it all up

Correlation/causation fallacy: Claiming two events that occur together have an automatic cause-and-effect relationship. (Correlation is equal to the cause.) Cherry-picking: Using evidence that supports your idea while ignoring contradicting evidence. Hasty generalisation: Drawing a general conclusion from a tiny sample. Category mistake: Things of one kind are presented as if they belonged to another. Appeal to anonymous authority: Using evidence from an anonymous ‘expert’, ‘study’ or generalised group (like ‘scientists’) to claim something is true. Appeal to common practice: Claiming something is true because it’s commonly practiced – everybody does it. Relativist fallacy: Rejecting a claim because of a belief that truth is relative to a person or group. Using subjectivity as an argument to reject research instead of counter-research. (McCandless, 2022; Krobová and Zàpotocký, 2022)

A good exercise is to look at other publications and see whether you are able to identify which rhetorical fallacies are made. Gaining more experience in recognising them not only helps your own writing process, but it also helps in better evaluating the validity and argumentation of what is published elsewhere.

Thinking about your audience when writing The target audience for a research project will differ according to the topic. There are also no specific rules or guidelines that are applicable to all sectors or situations when it comes to writing up your research (Denscombe, 2017). This means that whatever you write or produce, you need to adapt it to the appropriate circumstances and needs for the end-user/reader. Think about who or what the research is for and what you are trying to achieve by reporting it. Also, think about what is already familiar and what they need to know. As a writer you must find out what types of readers will be receiving your

Writing it all up

output and what they will be using it for (Bryman, 2016). For example, you might need to consider whether they are an academic, civil servant, charity or CEO. It is important to put yourself in the position of your audience. This can give you additional insights towards what style is preferred and whether the audience is likely to understand and read everything (Robson, 2017). Try to identify whether your audience is relevant when it comes to adding many quotations or whether they are only reading the main conclusions (Dawson, 2019). You might have to produce more than one report in order to cater for different groups. For example, one for the client and one for the more general reader. For some types of readers, you might have to use plain and clear language whilst for others you will have to convince them of your knowledge on the subject. In the case of the latter, also make sure you understand all the specialised terminology. You will also need to consider how your written output will be used, for instance, as a reference to make recommendations, to provide an overview of a subject or issue, to provide data for further use, to describe or to be critical or to demonstrate the development and delivery of a new product or service. The type of project and the target audience will determine the format and content of the report. For example, within industry, workplace reports may be short and less detailed, whereas academic reports are often complex and lengthy.

Presenting your work to others The best way to improve and develop your writing is to be willing to accept criticism, either formally or informally. For obvious reasons people feel uneasy with this but the best thing to do is to see it as a positive opportunity to learn. Particularly for a large-scale project, it is important to schedule for feedback in your workplan at key stages, say after each chapter or each milestone of product development. Without the discipline from this activity, the writing can drift on for longer than necessary. In an academic or industrial context you would arrange to meet with your supervisor or manager. You could present a working paper at a seminar, conference, workshop, or even a focus group. This can provide additional moments for feedback and lead to new insights and implementations that will improve your research (Patton, 2017).

129

130

Writing it all up

If the work is commissioned by another organisation, it can vary as to whether they want you to give feedback on the progress of the work at intervals, say via an interim report or upon completion of the project. This can often be the case during multi-annual studies or projects that require multiple rounds of investment for materials. If the latter is the case, it would be advisable for you to arrange to meet with the client when you have some form of draft so that you can be sure that their requirements are being met. This could save time at the end and avoid the problem of the report being thrown back at the end for major changes. Whatever the research context, make sure that you meet with your supervisor, manager, or client before the final draft stage to ensure that you have fulfilled the requirements and addressed any problem areas. Once you have received feedback, you must consider whether your report needs to be adjusted at all. Also, it is worth evaluating the criticisms and ensure that they were made for all the right reasons. Because there is a tendency to become immersed in your writing it is important to take a break from your writing for a day or two in order to be able to edit and evaluate your work effectively. Working through the revision checklist can be helpful if you are experiencing problems finishing off. Remember that it is normal to have multiple drafts as you go through the writing process and that a good tip is to make a separate folder to keep the previous drafts in for later reflection as they can still be very useful (Bell and Waters, 2018). One common reason for delaying this is that you don’t think that your work is good enough. Once you are satisfied that all of these points have been met, then proof-read the final draft. It is useful to do this by reading out loud. Finally, work through a revision/editing checklist, such as the one below, it will help to reveal any outstanding editorial or content issues in your reporting.

REVISION/EDITING CHECKLIST ●

Check for accuracy in spelling, referencing, quotations, grammar, and punctuation.

●

Check that your arguments are clear with logic and flow and that any headings and sub-headings are used appropriately.

●

Test for readability – this will be affected by the sentence and paragraph length and links between sections and chapters.

Writing it all up

●

Make sure that the layout, presentation, and referencing style meets with the appropriate conventions.

●

In the literature review assess whether an adequate number and type of sources have been included – for example, between academic and practitioner. Also ensure there is a balance in articles from books, journals and other sources.

●

Have the methods and analysis techniques been adequately justified?

●

Also, are the data reliable and appropriately sourced?

●

Ensure that the findings are clearly presented, and that the discussion is analytical and critical and not just a mere description.

●

Also ask yourself: Have the original objectives been achieved? If applicable, have hypotheses been proved or not?

●

Make sure that your conclusions are clearly based on evidence from your findings.

●

Ensure the layout and style are consistent throughout your report. (Jagaiah, Howard and Olinghouse, 2019; Bell and Waters, 2018)

A note on ethics when writing up Throughout your research ethics will always play an important part; you therefore have to be aware of the ethical standards in your field. As you have gathered data throughout your research, making sure it is valid is one aspect, making sure that it is aggregated and non-traceable is another. Having taken the right steps to ensure data remains anonymous and confidential when writing up your research is essential. This can be easier for large data sets, but when it comes to small-scale research, it can be more difficult (Rice and Atkin, 2013). You can provide a detailed discussion on how you implemented ethics within your research, topics you can address might include: ethical board approvals, informed consent (explaining benefits and risks), the voluntary nature of participation, confidentiality, the right for participants to ask questions, relational ethics, and the dissemination

131

132

Writing it all up

of the results. Finally, admit your limitations, address potential bias, and avoid exploitation (Leavy, 2017). Transparency through democratising the knowledge-building process allows your participants or audience to understand and to critically analyse its implications and format. One way of achieving this is, for instance, by sharing the findings and making them as understandable as possible. Using different formats and modes for your research can help establish this (Leavy, 2017). Think about how you might have used participants to test a new product you are developing, or to explain the clinical trials on which your results are based.

Submitting your research report to others Before you submit your work, you need to check that you have met the appropriate presentation requirements, such as margin sizes, line spacing, paper size, number of copies, and whether or not it needs to be bound. In the academic setting it is likely that you will be assessed internally by the relevant tutors or committee. In industry or the public sector, senior management or the board who requested the work may assess you. Within society, as a charity, your stakeholders and end-users will be relevant for the assessment. Regardless of the sector, you may be requested to do a presentation, product demonstration, public engagement, or oral examination. There is no such thing as a standard demonstration or presentation. For either it is important to prepare and ‘perhaps the best mental preparation of all is for the researcher to be in a position to exploit the strengths of their writing and to pre-empt criticism of its weaknesses’ (Sharp, Peters and Howard, 2002, p. 223). You need to demonstrate a clear grasp of the research context, what your research has contributed to the problem, and any limitations.

Styling your report Style relates to the way you write in connection to factors such as detail, complexity, language, terminology, and references, and is a key ingredient to ensuring the quality of a report. Whatever the written output required,

Writing it all up

you should aim for clarity in order to avoid confusion and ambiguity for the reader. Therefore, you should pay particular attention to the length of sentences and paragraphs and vocabulary. In addition, use language that makes the report interesting to read. Try to mix and match the length of sentences because this makes for a more interesting read. Be careful not to put in sentences that are too long because they may be less easy to understand and could make it hard work for the reader. Break up lengthy pages of text with headings and bullet points; however, be aware that certain conventions might not accept this. Short and simple words are better unless the conventions or the field you are working in particularly require these. Consider the tone of your writing, depending on your audience; you need to take care not to be patronising. At the same time, you have to be cautious in assuming certain things are self-evident. This is particularly important in the use of terminology or jargon. Take extra notice of your punctuation and do not use more than necessary in order to make a point. When developing an argument that is complex, break it up into separate parts and create linkages to make it easier for the reader to follow. Another important factor to consider is whether to write using a passive or active voice. A passive voice could be used to de-personalise the research – which would be useful if confidential data were consulted. An example of a passive voice would be ‘The conclusion that some managers receive more than others was indicated by information relating to salary levels’. In contrast to the passive voice, the active voice provides more information and can aid clarity. Whatever style you use the important thing is to be consistent. An example of an active voice would be ‘Salary information indicates that some managers receive more than others’. Finally, it is generally accepted that research reports should be written in a non-biased way – that is, in such a way that does not discriminate or exclude particular groups of people on the basis of what may be fairly arbitrary characteristics such as sex, race, religion, physical and mental abilities, or sexual orientation.

Social media and publication Using (social) media as a mode for publication or drawing attention to your research is an important tool that can be utilised very effectively. It can allow for open access to publications, fundraising opportunities, end-user

133

134

Writing it all up

education, and product promotion. It can even lead to an increase in academic productivity (Bean, 2016). This does not mean that there is a causality, but it indicates the importance and trends that demonstrate the power of (social) media (Bouvier, 2022). The reason behind using this online presence as an organisation is usually to attract public attention and this works across the different sectors as it can be either altruistic, as a public service, to build a reputation, for education or more commercial in nature in order to promote new innovative products and influence the market. Despite the limitations in terms of words, you can use (social) media as a channel to draw attention to the wider research project. You can therefore view (social) media more as a part of a broader strategy related to your publication and dissemination than as a solely purposed one. It may be that this forms a part of the overall plan related to your research, rather than the complete plan you wish to use to generate impact or create awareness (Rice and Atkin, 2013). The next chapter will continue on how you can use social media for knowledge dissemination.

Storytelling Another way of publishing work is though storytelling (as discussed in relation to narrative analysis in Chapter 4). It can be seen as an effective way of exchanging information and knowledge. It is used as a technique to present relationships in a dynamic manner through interaction (Tong et al., 2018). When explaining ideas, storytelling can be very useful as it creates very little distance between the intended meaning and the perceived meaning. You will have to be precise and make sure that these ideas are supported by the proper argumentation following your research. The idea is to take your audience on a journey where you want to relate your own experience to them and to create an emotional connection (Lambert, 2018). Bear in mind, however, that despite the assumption that the meaning you are trying to get across is evident, this can still lead to misinterpretation.

Visualising your data How you then visualise some results also heavily influences how your audience is going to interpret your report. When it comes to visual

Writing it all up

representations of data, there is a difference between being able to read complex graphs and tables vs. heatmaps and word clouds (Schwabish, 2021). Technological developments have allowed us to gather more data, and have then created the need to make this data explainable. It is important to not try and collect large data sets just with the intention to find evidence; it should be viewed as a supportive tool rather than an answer in itself (Bell and Waters, 2018). Visualisation of this data then allows us to turn something that can be quite complex into something that can be used for evidence-based decision-making and data-driven policy. It is a skill that is becoming more important and an effective visualisation can mean the difference between a success and a failure when it comes to communicating the results of your research, whether this is for an academic study, raising money for your charity, presenting to your board, or engaging with society (Knaflic, 2015). Research can be made more understandable through the use of new visualisation tools and narrative agents such as artificial intelligence. It will allow you to demonstrate simulations and digital twins that would have otherwise been impossible to explain (Cohen, Manion, and Morrison, 2017). Simulations ensure a high reliability with the added benefit that they avoid failures or system breakdown in a societal system. These new methods allow for a new narrative and can help put actions into context where you can experiment with intense scenarios (Spierling, 2002). There are many tools available that can help with the visualisation process, and these can be used by novices as much as by more advanced researchers. The important aspect is to not let your tools be a limiting factor in what you can do within your research, but to focus on picking a tool that will help you to communicate effectively with data you have collected. Do not be discouraged when it takes multiple attempts to create the visualisation, but continue playing with the tools and searching online for solutions (Knaflic, 2015).

Tone and voice You should know how to pitch the tone of what you are writing. For example, a news article for society will be lighter in tone than a purely academic one. Additionally, do not use more words than necessary in order to prove

135

136

Writing it all up

a point. Instead, make the article interesting by using lively vocabulary. Be innovative with your title and make it eye-catching. A lot of people only read the introduction and conclusion, so make sure that these are well written and give a good overview, especially if you are severely limited in terms of wordcount or time. Finally, don’t be intimidated or put off by the prospect of publishing. As long as you remain transparent and admit your limitations you are able to demonstrate results in an as open format as possible. Remember, writing down your research takes different forms and depends on what the purpose of your research was and which sector you might represent. It is important to start writing down as soon as you can. This will make the process more flowing and shows the role tone and voice will play.

Summary In this chapter we have demonstrated that you need to allocate plenty of time in your workplan for writing and that you should start as soon as possible. You need to consider how your written output will be used because the type of project and the target audience will determine the format and content of the publication. The best way to improve and develop your writing is to be willing to accept criticism, either formally or informally. Once you have received feedback, consider whether your report needs to be adjusted. Working through the revision checklist can be helpful if you are experiencing problems. You should be aware of the ethical standards in your field and make sure the data you have gathered is valid and anonymised. You need to demonstrate a clear grasp of the research context, what your research has contributed to, as well as the problem and any limitations. Whatever the written output required, you should aim for clarity in order to avoid confusion and ambiguity on the part of the reader. Using (social) media as a mode for publication or drawing attention to your research is an important tool that can be utilised very effectively.

Writing it all up

Despite the limitation of words, view (social) media as a part of a broader strategy related to your publication and dissemination. Both storytelling and the visualisation of data allow us to turn something that can be quite complex into something understandable. These new methods allow for a new narrative and can help put actions into context. Finally, know how to pitch the tone of what you are writing. Make the content interesting by using lively vocabulary, remain transparent, and admit your limitations.

References Bean, J. (2016) ‘Academic output and social media: A marriage of opposites’, World Neurosurgery, 90, pp. 651–653. Bell, J. and Waters, S. (2018) Doing your research project: A guide for firsttime researchers. 7th edn. Maidenhead: Open University Press. Blaxter, L., Hughes, C. and Tight, M. (2010) How to research. 4th edn. Maidenhead: McGraw-Hill/Open University Press. Bouvier, G. (2022) Qualitative research using social media/Gwen Bouvier and Joel Rasmussen. 1st edn. London: Routledge. Bryman, A. (2016) Social research methods. 5th edn. Oxford, New York: Oxford University Press. Cohen, L., Manion, L. and Morrison, K. (2017) Research methods in education. 8th edn. London, England : Routledge. Dawson, C. (2019) Introduction to research methods: A practical guide for anyone undertaking a research project. 5th edn. London: Robinson. Denscombe, M. (2017) The good research guide: For small-scale social research projects. 6th edn. London: McGraw Hill Education/Open University Press. Jagaiah, T., Howard, D. and Olinghouse, N. (2019) ‘Writer’s checklist: A procedural support for struggling writers’, The Reading Teacher, 73(1), pp. 103–110. Knaflic, C. N. (2015) Storytelling with data: The effective visual communication of information. 1st edn. Hoboken, NJ: John Wiley & Sons. Krobová, T. and Zàpotocký, J. (2022) ‘“I Am Not Racist, But …”: Rhetorical Fallacies in arguments about the Refugee crisis on Czech Facebook’, Journal of Intercultural Communication, 21(2), pp. 58–69. Lambert, J. (2018) Digital storytelling: Capturing lives, creating community/Joe Lambert, Brooke Hessler. 2nd edn. London: Routledge.

137

138

Writing it all up

Leavy, P. (2017) Research design: Quantitative, qualitative, mixed methods, arts-based, and community-based participatory research approaches. New York: The Guilford Press. McCandless, D. (2022) Rhetological Fallacies. Available at: https:// www.informationisbeautiful.net/visualizations/rhetological-fallacies/ (Accessed: 29 August 2022). O’Leary, Z. (2014) The essential guide to doing your research project. 2nd edn. Los Angeles, CA: SAGE. Orna, E. and Stevens, G. (2009) Managing information for research: Practical help in researching, writing and designing dissertations. 2nd edn. Maidenhead: Open University Press. Patton, M. Q. (2017) Facilitating evaluation: Principles in practice. 1st edn. Los Angeles, CA: SAGE. Rice, R. E. and Atkin, C. K. (2013) Public communication campaigns. 4th edn. Thousand Oaks, CA: SAGE. Robson, C. (2017) Small-scale evaluation: Principles and practice/Colin Robson. 2nd edn. London: SAGE. Schwabish, J. A. (2021) Better data visualizations: A guide for scholars, researchers, and wonks. New York: Columbia University Press. Sharp, J. A., Peters, J. and Howard, K. (2002) The management of a student research project. 3rd edn. Aldershot, Hants, England; Burlington, VT: Gower. Spierling, U. (2002) ‘Digital storytelling’, Computers and Graphics (Pergamon), 26(1), pp. 1–2. Tong, C., Roberts, R., Borgo, R., Walton, S., Laramee, R. S., Wegba, K., Lu, A., Wang, Y., Qu, H., Luo, Q. and Ma, X. (2018) ‘Storytelling and visualization: An extended survey’, Information (Switzerland), 9(3), pp. 1–42.

RESEARCH IMPACT Dennis Dokter

6

IN THIS CHAPTER WE EXPLORE: ●

What is impact?

●

Examples of impact.

●

Different interpretations of impact.

●

Co-production as impact.

●

Assessing impact.

●

Limitations of assessing impact.

● Dissemination. ●

Contextualising research impact (by using Merton’s norms).

Research impact has multiple ways of manifesting itself depending on the desired outcome. In this chapter we will focus on the different forms of research impact and how these can be measured. It will look at how this relates to the dissemination of research. Furthermore, it will explain how you can evaluate and reflect on your own research as well as how you can use Merton’s norms in order to critically challenge research in general.

Academia It will teach you how impact can help to expand the state-of-theart knowledge within your academic field, help you develop new teaching material, and publish new insights. DOI: 10.4324/9781003180159-6

140

Research impact

Industry It demonstrates how impact generates new ways of engaging with your product users, developing and creating new innovative products and achieving a better understanding of the market.

Public sector Impact can help this sector create new policy for their governmental organisation, generate new solutions for societal challenges and become more data-driven in their decision-making processes.

Society Impact teaches us to recognise how knowledge, services, policy, and products affect us personally. It allows us to think more critically about parliamentary decisions, publications, news articles, and products.

What is impact? When looking at impact, it is important to realise that this is very much related to how different knowledge is valued, incentivised, and rewarded (Williams, 2020). Individuals, research organisations, governmental organisations, and companies each have their own idea on what impact might mean in relation to their specific context. This makes it even more important to look at how research impact is measured and how this relates to different stakeholders and end-users.

IMPACT DEFINED An effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. (Alla et al., 2017)

Research impact

The wider impact of research is becoming increasingly relevant as universities have to make it more of a key component to their societal and economic role (beyond academia). This is partially related to their own realisation of how their role within society is developing as well as to the need for funded research, which has been paid for by taxpayers, to be accountable for its spending. This has led to a wider view of impact as well as a different funding process. When you are doing research, it is important to consider your relationship towards the other sectors, not only when designing your research framework but also when looking at the impact you are aiming to create. It is becoming increasingly important to demonstrate the impact your research will have as it is closely related to the amount of public funding and support you can receive (Bornmann, 2012).

The development of impact Historically, scientific impact from an academic’s point of view would be derived via bibliometric methods (i.e., the number of publications and citations generated from those publications produces an ‘impact factor’). These types of metrics focus on how you generate new knowledge within the academic field and how you therefore influence that sector (Williams, 2020). Other quantitative metrics supporting this would then be, for instance, research income and the amount of research projects executed and published within each year. Although bibliometrics are not the ideal way of measuring the complete breadth of what impact would mean, they do play a substantive role as a metric within academia and are influential within science (Petersohn and Heinze, 2017).

Examples of impact When you are thinking of what impact your research will generate, think about which sector you work in and which sectors your research is relevant to (either academic, industry, public sector, or society). Very often the generation of new academic knowledge falls under academic impact, but you should also consider the other forms of impact and effect your research work might have.

141

142

Research impact

As mentioned, the conception of impact has changed: following the introduction of the Research Excellence Framework (REF), societal/ industrial and governmental impact have been added as additional relevant factors. An analysis of the 2014 impact case studies produced by one of the UK’s leading universities further amplifies how the other sectors are represented within research impact: the top three categories were; Commercialisation (industry), Parliamentary Scrutiny (Society), and Influencing Government Policy (Public Sector) (Hinrichs-Krapels, Grant and Gill, 2015).

Academia From an academic point of view, impact can demonstrate new knowledge gaps and teaching methods that can be explored. It can provide you with new insights in how to teach your own classes as well as educate you with new knowledge within your field by reading papers and publications. As an academic, when you are designing a research proposal, it can be to fill those established knowledge gaps that have been identified by other research projects. Or it can be that a company comes to you with a problem for the development of new innovative solutions for their product development and that you are able to bring in your expertise.

Industry When examining the industrial sector, impact can have an effect when you want to know about the cutting-edge developments within your sector. By comparing the market and looking at publications and new products, this can give ideas for your own organisation. New market research can also provide you with the tools for engaging with your own market audience, allowing you to adapt your strategy in order to remain competitive. Similarly, as a company, you might wish to create a new innovative product in order to disrupt the current market and generate new income. In this instance you would design a research product to identify what steps you need to take in order to generate that impact. This might mean engaging with other sectors in order to get more insights on the available skills, current legislation, and current societal needs. An internal research

Research impact

project might be to analyse the current organisational structure and exploring how this might be improved.

Public sector As a public sector organisation, impact of research can demonstrate effects of policy interventions or demonstrate new toolkits in order to improve your services and increase efficiency. It can also demonstrate the need to change legislation and provide new transparency on the current state of your sector and expose certain societal challenges. The purpose of conducting research within a public sector organisation can be related to a desire to innovate within your organisation, as well as to commit to policy goals. You might want to visualise the status of sustainability within your local council, for example, or see how you might improve healthcare outcomes. You might also wish to develop tenders and funding grants. Knowing what impact might be generated gives you tools in order to develop these opportunities in such a way that they do meet your needs.

Society We are all influenced by the impact of research both directly as well as indirectly. Directly, it can be because of research impact caused by new products that you are using, or the effects a new policy implementation might have on your way of living. Indirectly, you can be influenced by news articles, social media posts, and other sources that are based on the research impact from the other sectors. Knowing the context on how policy, products, or media articles are designed will give you the tools to critically evaluate and compare them for yourself. Generating impact from a societal point of view can be very individual, for instance, when you are researching which, new product might be the best for your household as well as when you want to buy someone a gift or apply for a permit. You will want to review which products are the best or what documents are necessary in order to generate the desired impact. Similarly, when you are critical of certain articles, you might want to research where these insights came from. However, on a larger scale, certain foundations, charities, or representational organisations

143

144

Research impact

might want to commission research themselves or via bids. This can be to improve living situations, to represent patients, or to help your sports or other pastime club get the necessary funding or meet the obligated criteria for a new location or equipment. Figure 6.1 shows some examples of how you might establish impact within the four sectors of academia, industry, public sector, and society. They are useful starting points to help you to think about how you might wish to create an impact before you start your research or project. Similarly, working in one of the other sectors, these examples can demonstrate how you can create an impact within your own sector and how you might engage with a university, company, public sector organisation, or society in order to amplify the desired effects.

Sector

Impact Examples

Academia

-

Industry Public Sector Society

-

New ways of Teaching and new content. Bibliographic metrics. Knowledge creation and contextualisation. Market adoption. New products/services developed. Improved business model. Adopted policy implementations. Integrated toolkits. Evidence Based decision making/increased transparency. Different user behaviour (for instance; more use of public transport, improved health outcomes). Monitoring different behaviour of social media and review/comment platforms. Media exposure.

Figure 6.1  Exploring impact and examples

Sharing impact (and research) findings with others Heidi Fisher, a social entrepreneur and expert in examining effect and impact, has identified a number of ways to share impact findings with others. These are valuable considerations for anyone assessing the impact of research-driven work: ●

●

Be really clear who you are sharing your impact findings with – and what they will be interested in knowing. Decide what format is best for the audience you are targeting (report, audio, video, interactive graphics, infographics etc.).

Research impact

●

●

Be honest and transparent – share details of how you have measured your impact, areas where you have not measured anything, and how confident you are that the impact that has been achieved results from your activities. Do not be afraid to share your negative or unintended outcomes. Include what you are going to do to minimise these going forward. (Fisher, 2021, p. 161)

Different interpretations of impact There are different interpretations on impact (e.g. commercial applications, policy interventions, and cultural engagement). All of these vary and relate to different disciplines, institutions, sectors, or markets. The broader you design your research scope, the wider (potentially) will be your range and variety of impacts (Terama et al., 2016). Funded research implies a connection towards impact. This means that researchers will have to contribute, in one way or another, to one of the four sectors we have identified (academic, industrial, public sector, or society). It has changed the way research has been done in the past by broadening the field and increasing the influence of other sectors on its actual effects (Papatsiba and Cohen, 2020). An effective way of making sure you are addressing the impact across the different sectors of academia, industry, public sector, and society is via co-production. This means that you collaborate between academic and non-academic partners to, on the one hand, fill the knowledge gap or generate new knowledge as well as generate impact in the other sectors, depending on your partners and user involvement. And, as mentioned earlier, this does not deteriorate the quality of your research, but helps fill and address the wider demands of impact (Durose).

Co-production as impact Via co-production, you can, for instance, also affect what is taught. It shows that engaging with external actors and collaborating with them to identify knowledge gaps allows academics to generate new knowledge,

145

146

Research impact

papers, and teaching materials while simultaneously also helping an industrial partner develop a necessary innovation to solve their needs.

CO-PRODUCTION IN RESEARCH Research benefits of including people from outside the research community in a process of shared learning. This may be described in a variety of ways: co-production, collaboration, or participant and public involvement. It includes working with a range of people involved in project or research work, such as: participants in a project, patients, carers, and service users, as well as people from the wider society, like public policy-makers, community groups, third sector organisations, and businesses. The involvement of individuals with a stake in the project who are not researchers can enhance the quality and help it to bring about positive change for society and the economy. Co-production can take place throughout the project. It may encompass identifying research questions, design and priority setting, governance, the co-delivery of research activities, the communication of key findings and involvement in knowledge exchange. (UKRI, 2022)

Collaboration across the different sectors promotes value-based rationality and practices (Darby, 2017). It takes the conversation and research in a direction that is not only academically achievable but also relevant and desirable across the other sectors. Creating impact statements prior to the beginning of your research will help shape what you do, providing a more relevant mapping of how it will impact beyond the academic environment. One way of contributing towards the direction that your impact might generate is by leaving space for reflection within your publications, presentations, and other research projects. It also allows for the other sectors to critically reflect on research and support them in how it might be relevant for them towards their own research and innovation agenda. It is therefore important to remain transparent and develop new approaches within your work that leave opportunities for new approaches for co-produced impact (Wynne-Jones, North, and Routledge, 2015).

Research impact

Example of co-production across the sectors Each of our four sectors, academia, industry, the public sector and society, may view a research issue or topic from a particular viewpoint. As a researcher, you might notice that there is a knowledge gap when it comes to the different effects on patients who are required to wait for emergency care at the hospital versus waiting at home. From a public sector point of view, you might wish to create a new policy and implement new solutions in order to improve waiting times at hospitals, thereby improving patient care and experience while reducing costs. A commercial organisation might be working on developing a new application for the healthcare sector where they might want to focus on benefit of digital triaging and planning as a way of selling their product, while improving existing services and reducing other costs. From a societal perspective, you might notice an increase in frustration when it comes to waiting times or by having to physically wait within the hospital building.

Academia Researcher observes gap in waiting times for healthcare appointments.

Industry Digital patient appointment system offers business opportunity.

Public sector Development of new policy in terms if patient care may reduce waiting times.

Society Service users become frustrated by extended waiting times.

Figure 6.2  A co-produced view of a research topic in health

Examining the scenario above, you can observe an overlap between all the different sectors, by identifying the communalities and then using co-production as a collaborative approach to finding a solution. The important part is that there exists an awareness of how necessary it is for the sectors to work together, and to create and broker these opportunities. For instance, the public sector can create an opportunity by writing out a national funding bid with the purpose of solving waiting times and experiences within hospitals, together with organising sessions where interested parties can meet and discuss a potential proposal. The academic realises that this matches their desire to fix the knowledge gap and starts writing a high-level overview of what the bid might look like from an academic point of view (including potential papers they might

147

148

Research impact

be able to publish); they will also start looking for partners at the session. Similarly, the company sees this bid as an opportunity to further develop and implement their new product and will also join this session. When the two organisations meet, they might decide to collaborate, but they may also realise that in order for their ideas to actually have an impact and be successful, they will need to engage with end-users and patient representatives (society), hospitals, and legislation surrounding the implementation of new technology (public sector) in order to create the desired impact. This is just a brief example of one possibility that can foster co-production for the purposes of impact. The important takeaway would be to realise that engagement and collaboration with other sectors can amplify and speed up your own interests as much as it does those of the collective. Finding these partners might not always be as simple, but realising that these are valuable already makes you take the first step in terms of engagement.

Assessing impact As identified earlier, scientific research is essential in generating knowledge to better help us understand and contextualise contemporary social, economic, and political issues. Being able to measure and assess impact is an important step into providing insights to how research and innovation has led to desired, or even unintended, impact. Research impact has traditionally been assessed by using metrics, such as human, physical, and financial capital (Williams, 2020). Crosssector impact has been assessed by using a range of techniques and methods that look at the creation of intellectual property, statistical differences, commercial income, case studies, and economic measures (Penfield et al., 2013). However, as we have seen with the REF, there have been growing policy imperatives that have caused a shift. Australia started this trend in 2005 the Australian Research Quality Framework (which used predominantly case studies as an impact demonstrator), the Netherlands introduced the wider interpretation of impact within its standard evaluation protocol in 2003 and in the UK, it was the 2005 Memorandum from Research Councils, which led to the REF (which also uses case studies). Each assessment panels within the REF has different

Research impact

definitions within their guidelines depending on the impacts that are being assessed (society, culture economic, policy, production etc.). The impacts presented in the REF should still be seen as a proxy of looking at impact and that there are multiple approaches. Again, the measurement of impact can be different for each actor in the different sectors. There are three substantive approaches to assessing impact that have been developed for academia, but have purpose and utility for other sectors: 1. Forward tracking: start looking at the research and trace it forward to policy and practice settings (new products, behavioural change, legislation) 2. Backward tracking: in contrast to forward tracking, you look at policy implementations, knowledge exchange activities, or new services and products and try to retrace this back to research 3. Evaluation of mechanisms to increase research use: these focus purely on the knowledge exchange activities themselves and therefore demonstrate immediate uptake; however, it causes challenges for a longer period of time. (Morton, 2015) When using any of the above examples to assess research or academic impact, it is important that you take into account the societal, industrial, and public sector effects and impacts. This helps to create a more fully-rounded and inclusive view of impact. Widening your approach also provides better qualitative and quantitative data that potential funding agencies use to make decisions, or that actors within the other sectors will use to find engagement with and allows them to innovate themselves. Measuring the effect or impact of research is complex, non-linear and unpredictable. This causes us to more quickly count what can be more easily measured and not what counts in terms of significance (Milat, Bauman and Redman, 2015). Keep that in mind when evaluating impact assessment.

Limitations of assessing impact Generating impact and assessing how this can be achieved is very important in understanding how research places a role within academia, industry, public sector, and society. However, it is important to realise

149

150

Research impact

that there are, of course, problems and limitations to predicting accurately whether the impacts you expect are actually achieved. These can be summarised as: Causality: it is very difficult to make clear which impacts relates to which causes. Attribution: as impact is very complex, it can be hard to identify what other inputs might have attributed to the impact besides your work. Internationality: research and innovation has an international nature, this means that multiple effects take place globally simultaneously, influencing and diluting your work as well. Timescale: the initial measurement of impact happens on its short-term effects, it is difficult to assess its more longer-term implications. (Martin, 2007) Do not be discouraged by these problems; rather, you should view them as guidelines and limitations that should be addressed within your work. These acknowledgements can then help provide transparency to your research whilst allowing for new opportunities to arise, additional knowledge gaps to be identified and other products to be developed, by either yourself or others, in a follow-up project.

Dissemination To understand how knowledge can be spread in order to both monitor as well as assess impact, we have to understand knowledge dissemination. For a long time, researchers have been aware of how important knowledge dissemination is. Whilst most communication of research effect or impact has been through publications and academic articles, we are now seeing other communication tools and technologies being used to disseminate research work. This offers a greater opportunity to share the outputs of research work with much wider audiences. Newer approaches to dissemination, such as those offered by socially driven media, allow sectors that are not academic (industry, public policy, society) to share knowledge and consume knowledge produced by others (Schnitzler et al., 2016).

Research impact

KNOWLEDGE DISSEMINATION Knowledge dissemination involves distributing knowledge to those who may need it. (Kingston, 2012)

Typically, when sectors outside of academia wish to share knowledge on new policies, products, or services, they cannot always enter the world of academic publications. This forms part of an issue known as information asymmetry, where information is distributed unequally amongst your targeted audience, meaning that not everyone would be aware of the possible implications. However, open, transparent, and public disclosure of information leads to a reduction in asymmetry (Blankespoor, Miller and White, 2013). A wide variety of knowledge dissemination tools and techniques are available for researchers. Knowing who your target audience is will make it easier to look at what approach of dissemination will be most effective for that sector, both internally and externally (Kingston, 2012).

DISSEMINATION TOOLS AND DEVICES In addition to traditional reporting, there are a number of output options for your research work. These include: ●

Videos.

●

Podcasts.

●

Case studies.

●

Infographics.

●

Interactive images and virtual tours.

●

Podcasts.

●

Photo gallery/wall or graffiti wall.

●

Stakeholder events. (Fisher, 2021, p. 164)

151

152

Research impact

A dissemination identifier In order to communicate research work, we can utilise a four-way classification of systems that can support in identifying and disseminating the exchange of knowledge. Within this identifier, there are two dimensions; the first dimension makes a distinction between ‘collect’ and ‘connect’. ‘Collect’ is knowledge that is recorded or written in a repository and ‘Connect’ is where knowledge is communicated directly between individuals. The second dimension makes a distinction between ‘formal’ and ‘informal’. ‘Formal’ indicates that there is a defined framework or set of rules that you are operating in (structured), whereas ‘Informal’ is more unmanaged and ‘bottom-up’ (conversational) (Milton, 2010). Based on these, Figure 6.3 gives examples of dissemination you might be able to use as a way of knowledge distribution for your own research work. Connect - Social networking.

- Virtual Teams.

- Community engagement.

- Conference presentations.

- Patient participation.

- Formal networking events.

- Drop-in sessions.

- Teaching. Formal

Informal - Wikis.

- News articles.

- Blogs.

- Academic publications.

- Online reviews.

- Online presentations.

- Social media posts.

- Case studies.

- Opinion pieces.

- White papers. Collect

Figure 6.3  Dissemination identifier. Adapted from (Milton, 2010)

Limitations to dissemination When it comes to the dissemination via online sources, there are some specific limitations and risks that must be considered when using (social)

Research impact

media. This includes biased perspectives, knowledge within echo chambers and algorithm-driven filter bubbles that selectively display information based on user preferences (Chan et al., 2020; Pariser, 2011). This is often compounded when there is not sufficient source content to allow others to see a difference between true and untrue information (Johannsson and Selak, 2020). Given all of this, the following provides a useful review mechanism for evaluating the use of (social) media to disseminate research content.

TIPS FOR THE RESPONSIBLE USE OF SOCIAL MEDIA WHEN DISSEMINATING RESEARCH 1. Preferential use of established professional forums, or communication groups, to deliver information. 2. Clear identification of the information source – allows user to judge the likely veracity and quality of information. 3. Declaration of conflicts of interest, when appropriate. 4. Identify methods to verify the source when appropriate or necessary – website address if source not readily accessible by simple search strategies, or institutional email address of originator. 5. Transparent methods for peer review and feedback, and the provision of author/institutional contact details so that criticisms can be directed directly to originators. 6. Transparently acknowledge and document collaborations with identified professional experts, and, when necessary, adjust information to meet contextual needs. (Adapted from: Chan et al., 2020)

Contextualising research impact (by using Merton’s norms) In the first chapter of this textbook, we introduced Merton’s norms. These norms are seen as a way of how science should be performed within the scientific community. However, as Mitroff has shown through his study of the Apollo moon scientists, there are also counter-norms. These

153

154

Research impact

counter-norms should be interpreted as the norms that actually influence the field (Mitroff, 1974). The fact that they contradict the Mertonian norms makes the counter-norms relevant when valuing how research takes place; it puts the research in its context and helps us to assess its impact. Assessing how the norms or counter-norms are present in research has a relationship to what is happening within the practical world and places it in its societal, cultural, and historical context. By using these norms and counter-norms, you are able to ask more critical questions when assessing the validity of research but you are also able to describe the situation in which the research was performed and are able to challenge this in relation to its context. The following example can give an indication of how the contextualisation of these norms and counter-norms are present within research performed within a university setting. However, the same sort of analysis can be applied to any sector that performs, designs, or funds research – whether that be industrial research, public policy research, or societal research.

COMMUNALITY (COMMON OWNERSHIP) VS INDIVIDUALISM (RESTRICTIVE AND CLOSED) Imagine a university research lab where they are trying to develop a new method in order to find a more effective method to treat a disease. In a university setting it then becomes clear that sharing information about this research is important. One of the examples is how videos on YouTube can serve as a way to inform the general public on what they are researching and how they were able to get certain results. The use of open-source sharing and certain collaborations is to share published results and to promote the advancement within their field. Conferences are also visited in order to discuss and present unpublished research. There is a high level of trust between the scientists that discussed information at conferences is not ‘stolen’ for one’s own research. Sharing information with the public has commercialisation implications, for when the public is interested certain research

Research impact

sponsors are more inclined to supply funding for their work. The protection of the work researchers perform is also necessary to make sure that they are the first to publish their results. Commercial contracts allow more funding and access to technology, equipment, and information which otherwise would be unavailable because of intellectual property laws.

As you can see, there can be a clear struggle between being transparent as an organisation versus trying to protect your work or even your researchers. This norm focusses on how, despite the best intentions, it can be sometimes necessary to not share all information. However, it will always remain important to have the opportunity to scrutinise, even if that has to happen within a more formal setting.

UNIVERSALISM (EVERYONE CAN DO THIS) VS PARTICULARISM (RESTRICTIVE – ONLY A FEW CAN DO THIS) A diverse and multicultural environment provides different viewpoints on ideas which helps in constructing rich, new knowledge from a research project. However, from the perspective of particularism, you need to be a member of an exclusive ‘club’. For instance, when applying for a postdoc you must have a number of first author publications, if you do not have these publications the chances of getting hired are much lower. The attractiveness of a country or organisation also plays a role for research applications. How many grants you received and publications you have produced contribute to your chances of getting funding for your new research. It is important to “sell” yourself as a researcher. Focusing on who you know, what you’ve done, and where you came from are decisive factors. The distribution of funds within the university itself are related to this. Who is on what awarding panel helps to improve funding, making it relevant to how researchers develop themselves through the hierarchy of a university.

155

156

Research impact

It becomes clearer that a diverse workspace is more effective than a homogenous one. However, we need to keep in mind that a lot of bias influences our decision-making processes for who we hire and why. Being able to have a transparent and honest overview will help provide insights towards the conditions that people are working in an organisation.

DISINTERESTEDNESS (BEING OBJECTIVE AND CONSIDERED) VS SELF-INTERESTEDNESS (BEING SUBJECTIVE AND SELFISH) As a researcher you have to ‘generate’ knowledge. When there are more positions within a research department, for example, they are able to apply for more funding and therefore generating more money. These positions are not created for the mere interest in certain research but also for the intention of funding the institution. When you find something which is not necessarily in your field of research, it becomes difficult to continue this research. A way of being able to continue would then be to inform the relevant field which is connected to that research and hope that they are willing to cooperate and clear funds. Being resourceful with how you use your funding helps create reserves that support scientific creativity, and this is also promoted by principal investigators. The pressure to publish enough papers and to get a positive outcome in order to get enough funding of your research also influences the path your research has to follow.

The above illustrates the constant trade-off between purpose and profit. Within each sector, the main goal is, of course, to do what your institution was set up to do. The point of the matter remains though, that in order to keep your organisation running, you do need to make sure you retain or generate enough funds, be it by grant funding, sponsoring, or by selling your products/services. On the other hand, it also provides that drive to keep continuing to innovate.

Research impact

ORGANISED SCEPTICISM (CONSTANTLY CRITICAL AND EVALUATIVE) VS ORGANISED DOGMATISM (CLOSED THINKING AND UNCRITICAL) Because of the lack of funds in the scientific world, it becomes nearly impossible to replicate or reproduce certain experiments and results. There is no commercial interest for the reproduction of results, so no one is willing to fund them through grants. This causes researchers to take certain results for granted and to use them in their research without replicating them. With scientific research, something must be delivered. As shown before, more playing room for different side-tracks of research becomes possible because of certain grants and the way they are spent. It allows radiotherapy to replicate certain results. It is seen as important to be aware of this fact and to try replicate and reproduce their own results as much as possible. Competition restricts certain research to be done; everyone wants to do research in basically the same direction. Your research has to be economically feasible, which causes a lack of freedom within your research to replicate and question other results.

The last example shows the fragility between trying to be as safe and valid as possible versus timeliness and lack of funds. Deadlines will always play a part in making sure research and innovation is delivered efficiently, making sure it is safe and true are then very important factors. By making your processes as transparent as possible and referring to the limitations and demarcations of the research yourself, you are able to demonstrate due diligence and set an example. Remember, when using these norms and counter-norms in order to contextualise the quality of someone’s research, product, or policy, do not use it as a ‘be all and end all’ of scientific validity, but use it as an acknowledgement that research is indeed messy and non-linear and therefore needs to be viewed from multiple perspectives.

157

158

Research impact

Summary This chapter has identified that impact has an effect on academia, society, public sectors, and industry and that the wider impact of research is becoming increasingly relevant. Keep in mind of what impact your research might generate in which sectors. The broader you design your research scope, the wider the impact. By using co-production, you show that by collaborating benefits for the sectors you are working with as well as your own are created. Do keep in mind the limitations of attributing impact. Being able to measure and assess impact is therefore an important step into providing insights into how research and innovation led to impact. Again, impact can be different for each actor in the different sectors as research impact is complex, non-linear and unpredictable. The distribution of knowledge and knowing who your target audience is will make it easier to look at what approach of dissemination will be most effective; however, there are some specific limitations and risks that must be considered Think of the norms and counter-norms for putting research and impact into context. By making your processes as transparent as possible and referring to the limitations and demarcations of the research yourself, you are able to demonstrate due diligence and set an example. Remember, research is messy but necessary.

References Alla, K., Hall, W. D., Whiteford, H. A., Head, B. W. and Meurk, C. S. (2017) ‘How do we define the policy impact of public health research? A systematic review’, Health Research Policy and Systems, 15(1), pp. 84. Blankespoor, E., Miller, G. S. and White, H. D. (2013) The Role of Dissemination in Market Liquidity: Evidence from Firms’ Use of Twitter: Stanford University, Graduate School of Business. Available at: https:// EconPapers.repec.org/RePEc:ecl:stabus:2106r. Bornmann, L. (2012) ‘Measuring the societal impact of research: Research is less and less assessed on scientific impact alone–we should aim to quantify the increasingly important contributions of science to society’, EMBO Reports, 13(8), pp. 673–676.

Research impact

Chan, A. K. M., Nickson, C. P., Rudolph, J. W., Lee, A. and Joynt, G. M. (2020) ‘Social media for rapid knowledge dissemination: Early experience from the COVID-19 pandemic’, Anaesthesia, 75(12), pp. 1579–1582. Darby, S. (2017) ‘Making space for co-produced research ‘impact’: Learning from a participatory action research case study’, Area, 49(2), pp. 230–237. Fisher, H. L. (2021) Impact first: The social entrepreneurs guide to measuring, managing and growing your impact. London: Nielsen. Hinrichs-Krapels, S., Grant, J. and Gill, A. (2015) The nature, scale and beneficiaries of research impact. London: Kings College. Johannsson, H. and Selak, T. (2020) ‘Dissemination of medical publications on social media - is it the new standard?’, Anaesthesia, 75(2), pp. 155–157. Kingston, J. (2012) ‘Choosing a knowledge dissemination approach’, Knowledge and Process Management, 19(3), pp. 160–170. Martin, B. (2007) ‘Assessing the impact of basic research on society and the economy’, Paper presented at the Science Impact: Rethinking the impact of basic research on society and the economy (FWF-ESF International Conference, 11 May 2007), Vienna, Austria. Milat, A. J., Bauman, A. and Redman, S. (2015) ‘Narrative review of models and success factors for scaling up public health interventions’, Implementation Science, 10(1), pp. 113. Milton, N. (2010) The lessons learned handbook: Practical approaches to learning from experience. Oxford: Chandos. Mitroff, I. I. (1974) ‘Norms and counter-norms in a select group of the Apollo Moon Scientists: A case study of the Ambivalence of Scientists’, American Sociological Review, 39(4), pp. 579–595. Morton, S. (2015) ‘Progressing research impact assessment: A ‘contributions’ approach’, Research Evaluation, 24(4), pp. 405–419. Papatsiba, V. and Cohen, E. (2020) ‘Institutional hierarchies and research impact: New academic currencies, capital and position-taking in UK higher education’, British Journal of Sociology of Education, 41(2), pp. 178–196. Pariser, E. (2011) The filter bubble: What the Internet is hiding from you. London: Viking. Penfield, T., Baker, M. J., Scoble, R. and Wykes, M. C. (2013) ‘Assessment, evaluations, and definitions of research impact: A review’, Research Evaluation, 23(1), pp. 21–32.

159

160

Research impact

Petersohn, S. and Heinze, T. (2017) ‘Professionalization of bibliometric research assessment. Insights from the history of the Leiden Centre for Science and Technology Studies (CWTS)’, Science and Public Policy, 45, pp. 565–578. Schnitzler, K., Davies, N., Ross, F. and Harris, R. (2016) ‘Using Twitter™ to drive research impact: A discussion of strategies, opportunities and challenges’, International Journal of Nursing Studies, 59, pp. 15–26. Terama, E., Smallman, M., Lock, S. J., Johnson, C. and Austwick, M. Z. (2016) ‘Beyond academia - interrogating research impact in the research excellence framework’, PLoS One, 11(12), p. e0168533. UKRI (2022) Co-production in Research: UKRI. Available at: https://www. ukri.org/about-us/policies-standards-and-data/good-researchresource-hub/research-co-production/ (Accessed: 24 August 2022). Williams, K. (2020) ‘Playing the fields: Theorizing research impact and its assessment’, Research Evaluation, 29(2), pp. 191–202. Wynne-Jones, S., North, P. and Routledge, P. (2015) ‘Practising participatory geographies: Potentials, problems and politics’, Area, 47, pp. 218–221.

INDEX

Pages in italics refer figures. abstract 35, 53–54, 123 academia: analysing data 94; collecting data 58; planning 26; purpose of research 2; research impact 139–142, 144, 144–145, 147, 149, 151, 158; writing up 117 acknowledgements 124, 150, 157 action research 13–14 active voice 133 analysing data: coding and classifying data 83; content see content analysis; descriptive analysis see descriptive analysis of data; digital narrative–digital storytelling 88–89; discourse see discourse analysis; inferential analysis see inferential analysis of data; interpreting and 82–83; narrative analysis 87–88, 112; presenting data 109–110; qualitative data 83–84; quantitative data 83, 92–95; quantitative data analysis software 93–94; relating to hospital admissions

see investigation of admissions into hospital; role of data visualisation 110–111 Analysis of Covariance (ANCOVA) 109 Analysis of Variance (ANOVA) 109 anonymity 17–18, 28, 73 appendices 110, 126–127 assessing impact: crosss sector impact 148; limitations 149– 150; metrics 148; REF 148–149; substantive approaches to 149 attribution 150 audience 32, 53, 68, 95, 111, 117–118, 123, 128–129, 132–134, 144, 150 audio-recording 68–70, 73 Australian Research Quality Framework 2005 148 bias: academics 58; industry 59; information 58; and predispositions 57; public sector 59; society 59 bibliometrics 141 Blanche, R. 5

162

Index

causality 150 charts/figures 109–110 coding frame 85, 85–86 coding guides 86 Cohen, L. 28 collaboration 146, 148, 153–154 collecting data 31, 33, 38; experiment-based research and RCTs 60–62; observation as data collection tool see observation; quantitative and qualitative 59; social media as tool see social media; through literature review see literature review; tools/instruments 60 commercial organisation 147 communality vs. individualism 154–155 communication 25, 69, 77, 78, 118, 146, 150, 153 conclusions 126, 133, 136 confidentiality 28, 29, 73 confirmatory bias 57 constructivism 8–9, 10 constructivist research traditions 9 content analysis: coding guides 86; concept-driven and datadriven coding 86; develop a coding frame 85, 85–86; NVivo, software program 87; qualitative data analysis software 86–87; sample categories 84, 84 contents list 119, 124 contextualised/applied research 4, 4 contextual questions 69

co-production as impact 145; across the sectors 147–148; collaboration 146; in research 146; research topic in health 147 correlation research 103; formula for 106, 105–107 counter-norms 16 Covid-, 19; pandemic 78; vaccines 2 critical discourse analysis 90 Dallimer, M. 5 data analysis add-ins 93 databases and online reference resources: EBSCO 51; Google Scholar 51; JSTOR 51; ResearchGate 51; ScienceDirect 51; Web of Science 52; Wikipedia 52 data visualisation 110–111 dependent variable 104 descriptive analysis of data 92; associating data 103–105; formula of correlation research 106, 105–107; the mean 99–101, 100–101; the median 98–99; the mode 98–99; relationships between variables 103–105, 104–105; standard deviation 101–103 digital ethnography 74 discourse analysis 89; advantages of 90–91; critical 90; framework for 90; grounded theory 91–92 DiscoverText 78 disinterestedness vs. selfinterestednes 156

Index

dissemination: identifier 152, 152; knowledge 150–151; limitations 152–153; social media 153; tools and devices 151 Dokter, Dennis 116–137, 139–158 EBSCO 51 end matter 126; appendices 127; references 127 ethics of research 14–17; accurate reporting 18; anonymity 17–18; confidentiality 18; consenting to participate 17; potential for harm 18; questionable practices in social research 19; voluntary participation 17 ethnographic research 74 European Union 65, 89 experiment-based research data 60–62 Facebook 77, 77 fallacies (mistakes) 127–128 feedback 29, 129–130, 136 Ferré, M. 5 figures and tables 124 Fisher, H. L. 144 focus group interviews: advantages of using 71–72; checklist 72–73; defining 71; face-to- face setting 70–71; vs. one-to-one interviews 72; as online activity 70 funded research 141, 145 GANTT chart 36–37, 37 Gatekeepers 29–30

General Data Protection Regulation (GDPR) 65–66 Google Scholar 51 government policy (public sector) 142 graphical representations 110 grounded theory 91–92, 112 The Harvard System 127 Holden, J. 5 hypothesis 34; non-directional 108; two-tailed tests 108 independent variable 104 industry: analysing data 94; collecting data 59; planning 26; purpose of research 2–3; research impact 140–145, 144, 147, 149–150; writing up 117, 129, 132 inferential analysis of data 92–93; hypothesis 108–109; population 107; sample 107; statistical significance 108–109; testing for significance 109 informed consent 41–42, 42 Instagram 77, 77 internationality 150 interview: benefits of 70; conducting 68–70; in face-toface setting 68; focus group see focus group interviews; guide/ schedule–career in medicine 70; before interview 67–68; member checking interview data 68 introduction 125, 136

163

164

Index

investigation of admissions into hospital: age of admissions (female) 96–97, 97; Barnswell Hospital 96; grouping of ages– female admissions 98; Paperfield Hospital 95, 95–96; tally chart of female admissions 97, 97–98 JSTOR 51 knowledge dissemination 150–151 knowledge gaps 142, 145, 147 Lambert, J. 88 Lawford-Rolfe, R. 5 literature: aggregating material 55–56; critically analysing 56–57; Endnote 53–54; selecting and maintaining notes 53–55 literature review 46, 125; developing strategy 50, 52–53; as knowledge enhancer 48; as list 48; locating 49; as report 49; as search 48; as steering instrument 49; as survey/scan 48; types of 47–48 main text: analysis and discussion 126; conclusions and recommendations 126; introduction 125; literature review 125; methods 125; results 126 Manion, L. 28 Martin-Ortega, J. 5 the mean 99–101, 100–101

the median 98–99 Memorandum from Research Councils 2005 148 Merton, R. K. 15 Merton’s norms 139, 153–154; communalism/communism 15; communality vs. individualism 154–155; disinterestedness 16; disinterestedness vs. selfinterestednes 156; organised scepticism 16; organised scepticism vs. organised dogmatism 157; universalism 15–16; universalism vs. particularism 155 Mitroff, I. I. 16, 153 the mode 98–99 moderator/facilitator 71–72 moral compass 15 Morrison, K. 28 multiple-choice questions 62, 64, 64 National Institute for Health Research (NIHR) 7 negotiated access approach 30 Netnography 78 new knowledge 141–142, 145, 155 new products 142–143, 144, 148–149 NIHR see National Institute for Health Research non-directional hypothesis 108 non-participant observation 73 non-verbal behaviour 70 NVivo, software program 28, 87

Index

observation 79; advantages of using 76; ethnographic research 74; framework from organisational study 75–76; participant and non-participant 73; passive and contextualised 74–75 online survey 64–66 open-ended questions 62, 63 organised scepticism vs. organised dogmatism 157 parliamentary scrutiny (society) 142 participants 16–19, 26–31, 40–42, 60–61, 71–75, 88–89, 131–132, 146 passive voice 133 Pearson Product Moment Correlation Coefficient 103; approach (R) 106, 106 planning 24; academic 26; accessibility of research sample 29–30; ethical framework 41–42; industry 26; informed consent protocol 41–42, 42; methodology 39–40; negotiating access 28, 30; public sector 26; report writing 29; resources 27–28; society 27; timeframe 29; and timing see timing and planning policy research 3, 154 positivism 8–9 pragmatism 9–10 preface 124 preliminary part of write-up: abstract 123;

acknowledgements 124; list of contents 124; lists of figures and tables 124; preface 124; title 123 presentation 131, 146; conference 152; of data 109–110; online 152; requirements 132; of story 88 publication 4, 42, 51, 118, 127–128, 133–134, 136–137, 141–142, 155 public sector: analysing data 94; collecting data 59; planning 26; research for 3, 15, 17; research impact 140–143, 144, 144–145, 147, 147, 148–149; writing up 117, 121, 132 punctuation 130, 133 pure research 4, 4 qualitative data 12–13, 40, 59, 83–84, 88–89, 91, 111–112; analysis software 86–87 qualitative methods 12–13, 27 quantitative data 8, 11, 83, 92; analysis tools 112; and qualitative data 40, 59 quantitative data analysis software: academic 94; industry 94; public sector 94; society 95; SPSS 93–94 quantitative methods 11, 13 quantitative metrics 141 randomised control trials (RCTs) 60 RCTs see randomised control trials real-world setting 25–26

165

166

Index

recommendations 47, 126, 129 Reed, M. S. 5 REF see Research Excellence Framework references 53–56, 58, 85, 127, 129, 132 report writing 29, 54, 119 research: academic researchers 1; data see analysing data; ethical considerations in see ethics of research; experimental data collection design 61–62; focus of 3; Gatekeepers 29–30; hypothesis 34; importance of data 45–46; interviews see interview; market researchers 1; organisations 2, 84, 140; plan and structure 25; pollsters 1; pure and applied 4, 4; purpose of 1–3; and setting 32; in social sciences 24; strategy see research plan; survey 65; tradition/paradigm 8, 10 research-based disciplines 3–4 Research Excellence Framework (REF) 142, 148–149 ResearchGate 51 research impact: academia 139, 142; assessing see assessing impact; contextualising, using Merton’s norms see Merton’s norms; co-production see co-production as impact; definition 140–141; development 141; examples of 141–144, 144; industry 140, 142–143; interpretations 145; public sector 140, 143; sharing

impact findings with others 144–145; society 140, 143–144 research plan: visual plan – GANTT chart 36–37, 37; Week 1 to 2 35; Week 3 to 10 36; Week 10 to 13 36; Week 14 36 research project: collecting data 31; data analysis 31; developing a strategy 31, 35–36; exploring literature 31, 34–35; framing your questions 30–32; negotiating access 30; planning stage see planning; stages in process 24–26, 25; writing up results 32 research questions 34; defining 33; framing your questions 32–33; purpose of 33 research work 2, 3, 141, 146; advantages of using survey 67; dissemination 150–152; effect/ impact of 4–7; planning stage see planning; true/valid 7 resources 27–28; useful databases and online reference 51–52 revision/editing checklist 130–131 ScienceDirect 51 sectors 141–151, 158 skip logic/branching 66 small and medium-sized enterprises (SMEs) 2, 26 social media 152–153; DiscoverText 78; Netnography 78; platforms 76–77, 77; and publication 133–134; rapid access to 77 social research 19–20, 26

Index

society: analysing data 90, 95; collecting data 59; planning research 27, 41; purpose of research 2, 7, 15–16; research impact 140–147, 144, 147; writing up 117 software packages 28, 94, 100–101 Spearman’s Rank Order Correlation Coefficient 103 SPSS see Statistical Package for the Social Sciences standard deviation: distribution of exam results 102, 103; for exam scores 101, 102 Statistical Package for the Social Sciences (SPSS) 93–94, 100 statistical significance 108–109 storytelling 88–89, 111, 118, 134, 137 styling report 132–133 survey-based research 79; GDPR legislation 65–66; multiplechoice and scale questions 62, 64, 64; online survey 64–65; open-ended and closed-ended questions 62, 63; potential respondents identifying 66–67; routing and piping 66 survey response rate 38 target audience 128–129, 136, 151, 158 timeframe 29, 34 timescale 150 timing and planning: analysing data 38–39; collecting data 37–38; drawing conclusions 39; writing and submitting report 39

title 65, 109, 123, 136 traditions and methods 7; action research 14; constructivism 9; mixing quantitative and qualitative methods 13; positivism 8–9; pragmatism 9–10; qualitative methods 12–13; quantitative and qualitative research 10, 13; quantitative methods 11 t-test 109 Twitter 77, 77 UK Research and Innovation (UKRI) 6 UK Research and Investment (UKRI) 41 universalism vs. particularism 155 variables: negative relationship (correlation) 105; no relationship relationship (correlation) 105; positive relationship (correlation) 104; relationships between 103–104 visualisation of data 134–135 web-based collaborative text analytics system 78 Web of Science 52 wikipedia 52 Wilkinson, David 1–20, 23–43, 45–79, 82–112 write-up: end matter 126–127; ethics 131–132; getting started/ start early 116–119; main body of research report see main text;

167

168

Index

notes when writing see writing; organising information 119–121; preliminary part of see preliminary part of write-up; results 32; time management 121–123 writing: fallacies (mistakes) to avoid 127–128; presenting work to others 129–130; revision/

editing checklist 130–131; social media and publication 133–134; storytelling 134; styling report 132–133; submitting research report to others 132; thinking about audience 128–129; tone and voice 135–136; visualising data 134–135 written output 117, 129, 132, 136