Presentation structure background aims and objective of QT design of cognitive interviews review of methods of analysis NatCen approach issues for discussion Background aims and objectives of QT ID: 733108
Download Presentation The PPT/PDF document "Analysing and interpreting cognitive int..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Analysing and interpreting cognitive interview data: a qualitative approach
Slide2
Presentation structurebackgroundaims and objective of QTdesign of cognitive interviewsreview of methods of analysisNatCen approachissues for discussionSlide3
Backgroundaims and objectives of QTdoes test question meet measurement objectivesif not, what problems ariseimplications of problems for surveydesign of cognitive interviewsprobe sheetinterviewing techniquessampling strategySlide4
Review of methods of analysislittle written on analysis of cognitive interview datatwo main methods cited:standardised coding scheme qualitative analysisstandardised coding scheme approaches are documented (usually)qualitative analysis descriptions are often scant on detailSlide5
Standardised coding schemescan be used to as stand alone question appraisal tool (e.g. QAS)can incorporate some elements of behaviour codinginterviewer has a problem reading the question or recording the answerfocus on cognitive Q & A modelcomprehension/communicationrecall/computationbias/sensitivity (judgement issues)response category plus ‘logical issues’Slide6
Issues with standardised schemesPROS lend themselves to presenting data quantitatively (x Rs had this problem) perceived (by some) to be more robust process is replicable useful in cross-national/ cross cultural settings, where standardisation importantCONS need a lot of detailed codes under each main heading (particularly comprehension) time consuming loose context: why did R interpret Q in that way? lend themselves to presenting data quantitatively (x Rs had this problem)Slide7
Qualitative approach“Just naming and classify what is out there is usually not enough. We need to understand the patterns, the recurrences, the whys. As Kaplan (1964) remarks, the bedrock of inquiry is the researcher’s quest for ‘repeatable regularities’.” Miles & Huberman (1994)Slide8
Approaches to qualitative analysisEthnographic accountsdetailed ‘thick’ descriptions of cultures or organisationsLife histories analysed as individual cases or mined for common themesContent analysis identifies content and context of documents, often involves counting (not strictly qual)Grounded theorygenerates analytical categories and the links between them through an iterative process of collecting and analysing dataSlide9
Approaches to qualitative analysis Narrative analysisexamines how a story is told and the intention of the teller Conversation analysisexamines the structure of (usually) naturally occurring conversations Discourse analysisfocuses on how knowledge is produced through the use of language
Interpretative analysis
attempts to present and re-present the world of those studied, by identifying and describing substantive themes, and searching for patterns between themSlide10
Key stages of the analytical processData managementidentifying themessorting and reducing data
Generation of findings
describing
classifying
finding linkages and patterns
identifying explanations
Characteristics
of ‘good’ analysis system
Remains grounded in the data
Transparent data reduction process
Facilitates and displays ordering
Permits within and between case analysisSlide11
Seeking wider applications
Developing explanations
Detecting patterns of association
Establishing typologies
Identifying elements & dimensions
Summarising / synthesising data
Sorting data
Tagging data
Identifying initial concepts / themes
Primary data
Data
management
Descriptive
analysis
Explanatory
analysis
The analytical hierarchy in qualitative research
Data collection
?
Slide12
Thematic analysis - purposes and principlesStructured display of data by theme (Q) across all casesCreating categories and classifying data within them*Demonstrating range and diversityUsing examples to illustrate and amplifyMust be comprehensiveLabelling and categorising must be validSlide13
Carrying out thematic analysis - NatCen approachFamiliarisation with dataIdentification of factorshighlight, summarise, provisionally labelCategorisationis this a different manifestation of thatis this a subset of thatis this of the same order as thatIterative process of refinementStart close to the data - become more abstract and interpretativeMust be comprehensiveAim is analytical coherenceSlide14
Looking for explanationsInformed by:hunches and hypothesesreflections during fieldwork and analysisother research or theoriesProcess involves:detailed within case analysiscomparison between casesrepeated interrogation of the datamilking datamoving back and forth between casessearching for rival explanationsExpect multiplicityMust be comprehensiveSlide15
Summary of NatCen approachDetailed notes made on interviewsNotes reviewed Chart set upNotes charted (chart revised)Charts reviewed Data interpretedFindings emergeRecommendations madeReport writtenSlide16
Example chartSlide17
Issues to considerbetter documentation of qualitative analysis approachintegration of code frames within thematic approachdevelopment of best practice for analysis of cognitive datacognitive interview data as one component of testing strategycollaborate findings using other data sources (e.g. split ballot experiments)Slide18