1 Qualitative Research: - PowerPoint Presentation

Slide1

1

Qualitative Research: how to do good work, get it published and have an impact

Professor Stewart Clegg

UTSSlide2

Social science

No need for physics envy! Social science that apes physics will have little useful to contributeFollow Popper’s idea of critical rationalism – we must try to falsify our initial hunches about the relations between phenomena in our data

Only then, if we cannot falsify, can we speak about ‘objective’ knowledge – but it is always provisional on the next study ….

2Slide3

Validity enhancing approaches

Analytic inductionConstant comparative methodDeviant case analysis Comprehensive data treatmentTabulations

3Slide4

Generalizability

How to generalize in qualitative

research

Deductive

inference

Choosing a critical or deviant

case to refute an existing theory

Comparative inference

Maximize variation in cases – if findings hold across cases then this is a good sign

of generalizability

Exemplary or emblematic inference The case stands as an exemplar of the phenomenon under consideration

4Slide5

Flyvbjerg: Five misunderstandings

about case study research

1. General

, theoretical (context-independent) knowledge is more valuable than concrete, practical (context-dependent) knowledge

But: take us close to data

, build skills in data analysis

2. One cannot generalize

on the basis of an individual case

But: single cases are crucial in refuting initial hypotheses – the ‘black swan’ argument

3. Case studies can generate initial hypotheses but other methods have to be used

for hypotheses testing and theory buildingBut: exemplary or atypical cases can can reveal more information than the randomly sampled average 4. Case studies contain a bias towards verification – to confirm preconceived prejudices

On the contrary, the bias is towards falsification, because of constant comparison

5. Difficult to summarize and develop general propositions and theories on the basis of specific case studies But: complex narratives represent complex situations

5Slide6

Theoretical sampling

Clarity about unit of analysis – could be one organization but many samples of meetings, encounters, etc.Sample chosen not to prove but disprove the case

Extend sample as new insights emerge to test them out – maybe search for new deviant cases

6Slide7

Writing it up

Write clear, grammatically elegant language.

It

should

not

be necessary

to be a sub-disciplinary, or even disciplinary, specialist, to be able to

comprehend

Have you made it clear what new insights

flow

from the research?Why these are significant, innovative and valuable?Is it evident‘

who benefits?

’from the research?

7Slide8

Introduction

The background to the research

The context of the research, particularly the major ideas (or theoretical perspective) from which the research is derived.

The reasons for doing the research

The aims and

purpose, setting them

in the context

that

is relevant

The rationale for the design of the inquiry

Why the research is important, valuable or significantGive clear directions by the end of the 3rd or 4

th

paragraph:In this paper I will, first…, second…, third… .

8Slide9

Literature review

The literature review indicates gaps by pointing out:

Aspects of the field which have not previously been researched

Limitations or shortcomings of previous research

Areas which other researchers have indicated are in need of further examination

The literature review indicates that the researcher:

Is transparent and reflexive about conduct, theoretical perspectives and values

Understands the theoretical contexts within which the literatures reviewed have been generated

9Slide10

Literature Review

Check out the International Journal of Management Reviews

As

the first reviews journal in the field of business and management, the

International Journal of Management Reviews

(IJMR) is an essential reference tool for business academics and doctoral students alike.  The journal covers all the main management sub-disciplines including, for example, HRM, OB, International & Strategic Management, Operations Management, Management Sciences, Information Systems & Technology Management, Accounting & Finance, and Marketing.  Each issue includes state-of-the-art literature review articles/surveys which examine the relevant literature published on a specific aspect of the sub-discipline, for example, HRM: Appraisal Systems. 

Has someone already saved you a lot of work?

10Slide11

Methodology

This section outlines the methods

used

to achieve the research aims.

It should describe in as much detail as possible the data collection procedures

used

, whether they be experiments, surveys, questionnaires, observations, participatory methods, case studies, document collection or other methods

.

Reference should be made to the research methods literature or literature specific to the field of study

to

justify the choice of methods. 11Slide12

Methodology

The methods must be shown to be reliable (that is, they can be in principle, replicated) as well as internally and externally valid.

Internal validity means that the conclusions drawn from the study actually reflect the situation under consideration.

External validity means that the results are generalizable to a wide range of situations.

It is often necessary to present evidence in this section that the

research is

actually achievable, which may involve describing sources of funding, the previous experience and attributes of the researcher, or the extent to which preliminary enquiries into the feasibility of the research have been made.

12Slide13

Methodology

The methodology, or a separate section, should also describe the methods of data analysis to be used. As with the data collection methods, the analysis methods should be justified by reference to the relevant literature.

A methodology section can contain a flow chart which summarizes the way in which the various processes involved in the project fit together.

13Slide14

Checkpoints

How do you represent data or evidence faithfully?

How

do you

convey the depth, diversity, subtlety and complexity that the research will generate?

How

do you

demonstrate that the data or evidence that you are dealing with will be critically interrogated?

Remember that for every claim you raise you must support this with evidence, usually a page specific citation.

14Slide15

Checkpoint

How do you demonstrate that the study, although qualitative, is not merely going to be a piece of subjective impressionism?

How do you guard against just re-telling what the subjects already know?

How do you guard against

‘

taking sides

’

or being

‘

romantic

’ about subjects?Why is this suite of qualitative methods the best for this research?What is the sampling frame? What is its rationale?

15Slide16

References

It is important to ensure that all the key journals and books in the field have been referred

to.

You will be judged on the quality of the journals and authors that you reference: make sure the majority of journals are top-tier and the majority of authors are first-rate

16Slide17

Writing as a practice

For most academics, the first substantial writing that they will do is probably a thesisSeveral important lessons:

Never throw anything away – ideas that don’t work where and when they were first drafted may come in useful elsewhere

Always file drafts with an easy version/date retrieval system

Don't assume that you begin at the beginning and work systematically through the middle to the end:

Often, it is easier to begin with the middle, with some data analysis, or research design issues, than the beginning.

17Slide18

Writing as a practice

Usually, I write the first chapter or the abstract last. It makes more sense – how do you know what you are going to say until you have said it? (Karl

Weick

said that!)

One thing I often find myself doing is trying to sort out the literature into a dialectical debate

If you can establish a thesis and an anti-thesis in the literature then you create the space in which you can provide a new synthesis that does not just reproduce the existing terms of debate but transcends and repositions them. I’ve done this a few times.

Tables can help here. Sometimes they are used in the paper/book; other times they are just useful in sorting out thinking, and don’t appear.

18Slide19

Practicing

Irrespective of anything else that you become as an effective researcher, you must become an effective writer.These things help:

Read widely

Choose exemplars and model your style on their sentence structures, punctuation, grammar, vocabulary, etc.

Choose the target journal carefully – does the paper contribute to the debates that occur within it?

19Slide20

Good writing

Good writing has rhythm through:The balance of long and short sentences

Clever use of punctuation, to break up complex ideas

A strong narrative that unfolded directionally – from beginning through middle to end

Structure – appropriate use of sub-heads

Good writing doesn’t try too hard – it guides the reader – its form makes its function more effortless

Good writing amuses, pleases, informs, impresses

Bad writing dulls, irritates, bores, and depresses

20Slide21

Rules for writing

One paper = one ideaOne sub-headed set of material = one themeOne paragraph = the development of one aspect of one theme

One sentence = one subject

The

importance of a good editor

21Slide22

More rules

Don’t start paragraphs with “indexical pronouns” or “conditional phrases”Indexical pronouns

E.g.:

It, This,

Conditional phrases

E.g.:

However, Because, Thus, Yet,

Don’t start sentences with them either!

Avoid e.g., i.e., etc. in prose

22Slide23

Visual checks

If it looks wrong it probably is:Quick visual checks:

Length of sentences – problem is that they are usually too long

Length of paragraphs – here the problem can either be that they are too long or too short: a rough rule – about three paragraphs a page.

Microsoft green lines – they are probably telling you something you need to know

Spell checkers – they do the job – so how come so few people seem to use them … or use them lazily?

Over-repetition in form: stock phrases, such as:

“It is argued that …”, “Smith argues …”, “According to …”

23Slide24

Further visual checks

Structural levelsFirst level sub-heads

Second level subheads

Third level subheads

Don’t go any further than third level – too messy and indicates too much going on and author not in control

of material – also it buggers up the fonts.

Spelling; if it’s a US journal use US spelling; Australian, Australian spelling; British, UK spelling – and so

on.

Does

the journal have a length rule? If not, establish the norm – and follow it.24Slide25

Where to publish

Which journal?Make sure that you cite debates in the journal in question

Always check the journal style requirements and conform to them

Nothing

irritates a

journal editor more than the thought that because the paper is written to a competitors style guidelines it is a reject

Always check the Editorial Team and Board:

It is to one or more of these people that the paper will go – have you cited the likely suspects?

25Slide26

Publishing

Publish in journals! This is what national metric systems tend to count.Go for the best journals you can, realistically

The worst they can do is reject

The best they can do is offer excellent feedback from which you can learn

Always resubmit – either to the original journal if invited or elsewhere if not.

Always respond to advice received

Always write a clear letter to the editor explaining what you have done and not done in response to the suggestions received.

26Slide27

Why that journal?

SensemakingYou know the ranking orders for the journals; you’ve checked their citation impact factor

That journal publishes the right kind of stuff

Key references were published there

It’s a favourite journal

Anyone would kill to be published there!

27Slide28

Drafting

Look at six or eight articles from the journal decided onDraw up a a series of columns and in these identify:

The number and identity of sub-headings used

The hierarchy of sub-headings used

Model the paper’s structure on what seems to work there

Try to have a sense of audience – do people that write in that journal seem like the kind of scholars you want to address and whom you respect?

28Slide29

Test the paper

Draft early, draft often

I ordinarily draft anything from 30 – 100 times; I have drafted even more.

Send it to

friendly others

to read

That’s how I come to draft so much – they offer good advice

Once you have done ten or twenty drafts and it is getting in shape send it to some

significant others

Always keep old drafts and always number sequentially so that one knows where one is in the series

29Slide30

Finalizing the paper

Use Google to track down referenceshttp://www.scholar.google.com/

Someone

will have cited that hard to find reference somewhere for

which

they don’t have all the details

Have you done:

An abstract?

Keywords?

Checked the bibliographic items?Sent an anonymous text and a separate author bio?30Slide31

Final check

ChecklistHave you exceeded the word count?

Have you got the format and style right?

Have you sent the additional material that they need, such as a brief biographical note?

Have you checked the spelling, grammar and bibliography thoroughly and at least several times?

31Slide32

Letter from the editor

Chill out!

No one gets accepted first time

Plenty of people get rejected

The important thing is to learn from the Editorial letter

What one wants, realistically, is a Revise and Resubmit

Keep cool!

Maybe leave returning to the paper for some weeks after getting the Editorial letter

Try and deconstruct the paper from the point of view of the reviewers

Try and respond to them by reconstructing the paper

32Slide33

Responding to reviews

Surviving the review processA R&R is a

good

thing!

Read letters of review

very

diligently

Write letters to editor

extremely

carefully

33Slide34

After acceptance

Getting something accepted is hard enoughNow you have to demonstrate ‘impact’And that is more than being published

34Slide35

Research impact

Impact: “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life beyond academia”. (Higher Education Funding Council For England 2011: 26;)

“The demonstrable contribution that excellent research makes to society and the economy …[including] … all the diverse ways that research-related skills benefit individual,

organisations

and nations: (ESRC 2012: 1)

35Slide36

Types of research impact

Instrumental impact, influencing changes in policy, practices and behaviour though knowledge mobilizationConceptual impact, changing people’s knowledge, understanding and attitudes towards social issues

Capacity building

where involvement in research developed the skills of those involved.

36Slide37

Impact through Knowledge Mobilization

Knowledge Mobilization: Getting the best evidence to the appropriate decision makers in both an accessible format and in a timely fashion so as to influence decision makingTwo elements:

Evidence-based policy

Need to demonstrate economic impact

37Slide38

Knowledge Mobilization

The movement for evidence-based policy began in health research in the 1990s It was rapidly joined to a political mandate demanding demonstration of economic impact from research council funding in terms of measured outcomes rather than just research outputs.Question: What’s the difference between a research outcome and a research impact?

38Slide39

Measuring research outcomes

Peer review is the traditional way of measuring research quality so that research is judged by the quality of the journals it is published in: impact factors then become a de facto measure of research outcomesPeer review is a good guide to perceived research excellence but not infallible:

Strongly weighted towards US journals because their impact is situated in a large domestic market and benchmarked globally

39Slide40

Moving away from peer review

Citation analysis: measures such as the h-index (Harzing’s Publish or Perish).Case studies from research demonstrating its contribution to economic, social and/or public policy as well as cultural/quality of life impacts with some supporting quantitative indicators

Pathways to impact statements

40Slide41

Pathways to instrumental impact statements

In the UK the ESRC now expects research applications to include a Pathway to Impact document which outlines the possible pathways through which the proposed research will make an impactAs well to follow the fashion – but how?

ESRC developed an Impact toolkit advising

How to achieve maximum impact

How to develop an impact strategy

How to promote knowledge exchange, public engagement and effective communication with stakeholders

41Slide42

Measuring instrumental impact

Payback frameworksTracking ForwardsTracking Backwards

42Slide43

Payback Framework

Payback Framework methodology adapted from health research field and applied to major research centres seeking demonstration of:Knowledge production

Impacts on future research

Policy impacts

Practice impacts

Wider social and economic impacts

43Slide44

Tracking Forward

This method seeks to track forward from the research outputs to assess the ways in which they have been incorporate into practice.Uses surveys of and interviews with researchers and users Looks at relationships established between researchers and recipient communities as key to knowledge mobilization

Looks for involvement of users at all stages of the research and for well-planned user engagement strategies – these require good infrastructure and management support

44Slide45

Tracking Backwards

ESRC Evaluation Committee has sought to establish how and in what ways quantified economic assessment of the value of major program investments could be madeMixed success – the measures produced can only ever be approximations of value This the default position of most politicians and commentators unfortunately – instrumentalism rules!

45Slide46

Issues in demonstrating impact

Knowledge transfer is a reflexive rather than linear process – sustained by dialogue and conversation rather than one-shot publication or advice Have to know:When to look for research impacts – how long before effects materialize?

How

to assess that the specific contribution made by research was the key factor – in a multi-causal world establishing causal paths is extremely difficult.

46Slide47

Conceptual Impact

Conceptual impact occurs when research makes significant changes to practice-based thinking, debate, culture and direction.Conceptual impact is increased byEffective media training for communication on the part of researchers

Translation of research findings by skilled knowledge brokers for different audiences

47Slide48

Capacity Building

Developing the training and capacity of next generation researchers Two studies have investigated the impact of PhDs who have moved into practice on practice:Johnson & Williams (2011)

Evaluating the impact of social scientists

, ESRC publication

PhDs’ high-quality skills in interpreting and evaluating research findings have more impact than specific substantive knowledge

48Slide49

Demonstrating impact

The Social Sciences and Humanities Research Council of Canada ask applicants for research grants to outline the following:

Expected Outcomes

Elaborate on the potential benefits and/or outcomes of your proposed research and/or related activities.

Indicate

and rank up to 3 scholarly benefits relevant to your proposal

.

Social

Benefits

Indicate

and rank up to 3 social benefits relevant to your proposal.AudiencesIndicate and rank up to 5 potential target audiences relevant to your proposal.Expected Outcomes SummaryDescribe the potential benefits/outcomes (e.g., evolution, effects, potential learning, implications) that could emerge from the proposed research and/or other partnership activities.

49Slide50

Penultimate words

Being a successful researcher has never been easier in some respects – the range of acceptable investigative techniques and the affordances of the digital ecologyBeing a successful researcher, however, has never been harder in some important respects – particularly the need to demonstrate ‘value for money’ and ‘impact’

The time for ‘pure’ scholarship is greatly reduced these days

50Slide51

Select references

Alvesson, M. (2011) Interpreting Interviews

, London: Sage.

Alvesson

, M. & Karreman

, D. (2011)

Qualitative Research and Theory Development: Mystery as Method

, London: Sage.

Alvesson

, M. and

Skoldberg, K. (2009) Reflexive Methodology: New Vistas for Qualitative Research, London: Sage.Buchanan, A. (2013) ‘impact and knowledge mobilisation: what I have learnt as Chair of the Economic and Social research Council Evaluation Committee’, Contemporary Social Science (Special issue on Knowledge mobilization and the Social Sciences: Research Impact and Engagement, 8(3): 176-190.Chamaz, K. (2006)

Constructing

Grounded Theory: A Practical Guide through Qualitative Analysis, London: Sage.Denzin, N. and Lincoln, Y. (2005) The Handbook of Qualitative Research, London: Sage.Hennink, M.,

Hutter

, I., and Bailey, A. (2011)

Qualitative Research Methods

, London: Sage.

Silverman, D. (2011)

Interpreting Qualitative Data

, London: Sage.

Silverman, D. (2010)

Doing Qualitative Research

, London: Sage.

Silverman, D. (2007)

A Very Short, Fairly Interesting, Reasonably Cheap Book about Qualitative Research

. London: Sage.

51