Statistical discourse analysis of a rolebased online discussion To appear in the International Journal of ComputerSupported Collaborative Learning Alyssa Wise Simon Fraser University alyssawisesfucas ID: 267395
Download Presentation The PPT/PDF document "Temporal patterns of knowledge construct..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion
To appear in the International Journal of Computer-Supported Collaborative Learning
Alyssa WiseSimon Fraser University alyssa_wise@sfu.casMing Ming ChiuState University of New York –Buffalo mingchiu@buffalo.edu
I appreciate the research assistance of Choi Yik TingSlide2
Online, asynchronous forumsCan participate anywhere – no geographic limitsCan share ideas at any time – more time to thinkBut often disconnected, only lists of isolated ideasGuzdial & Turns, 2000; Herring, 1999; Thomas, 2002
SummariesConnect previous ideas and develop themBut often occur at end of discussion & Do not benefit other members
De Wever et al., 2007; Schellens et al. 2005; 2007Encourage summaries in the middle of discussions?Motivation for the Study Slide3
Knowledge Construction (KC) FrameworkGunawardena
et al.’s (1997) Five-Phase ModelSlide4
Research Context for the Study Emerging Themes in Collaborative Learning Research
(e.g. Chiu & Khoo, 2005; Kapur, 2001; Reimann, 2009)
(e.g. Cress, 2008; Suthers & Teplovs, 2011)(e.g. Arvaja, 2007; Stahl, 2004; Strijbos et al., 2004)Slide5
Possible KC Patterns
Knowledge Construction Phase
Post NumberSlide6
Research QuestionsWhat patterns characterize knowledge construction processes during an online discussion?What characterizes pivotal posts that divide a discussion into distinct segments? Summaries?Which characteristics of a post influence the knowledge construction phase of the next post?Slide7
Pivotal
Post
Functions
Summary (+)
…
Roles
Synthesizer (+)
…
Individual
Control variables
Gender
Age
…
Post
Control variables
# of words
Time of post
…
Time context
WeekSlide8
Knowledge
Construction
Functions
Summary (+)
…
Roles
Synthesizer (+)
…
Individual
Control variables
Gender
Age
…
Post
Control variables
# of words
Time of post
…
Time context
Week
SegmentSlide9
MethodsSlide10
Function
Role
Give
Direction
New
Idea
Bring
Source
Use
Theory
Respond
Summarize
Starter
X
X
Inventor
X
Importer
X
X
Mini-me
X
Questioner
X
Elaborator
X
Devil
’
s
Advocate
X
Traffic
Director
X
Synthesizer
X
X
Wrapper
XSlide11
Content Analysis
Variable
Inter-rater reliability ()Knowledge construction .84
New Idea
.65
Bring in Source
.92
Use Theory
.73
Respond
.98
Give Direction
.76
Summarize
.88
Unit of analysis
:
Post / Note / Message
Objectively identified unit that its author defines
Rourke, Anderson, Garrison, & Archer, 2001
Inter-rater reliability
Krippendorf’s
(range: -1 … 1; desired: > .67)Slide12
4 types of Analytical DifficultiesTimeOutcomesExplanatory variablesDataset - No missing data
Statistical Discourse AnalysisSlide13
Statistical Discourse Analysis
Difficulties regarding TimeSegments differ (S2
S4)Serial correlation (p8 → p9) Branches of notesStrategies Breakpoint analysis + Model Multilevel analysis (MLn, HLM) Test with I2 index of Q-statistics Model with lag outcomes, KC (-1) Store path: Identify prior turn
1
2
3
8
4
5
6
7Slide14
ID
Action
Turn #Valid?
Previous
Turn
Valid
(-1)
Ana
Do three times four.
1
–
–
Ben
Three times four is seven
2
X
1
Eva
Three times four is nine.
3
X
2
X
Jay
Three times four is twelve.
4
3
X
ID
Action
Turn #
Valid?
Respond
to post?
Valid
(-1)
Ana
Do three times four.
1
–
–
Ben
Three times four is seven
2
X
1
Eva
Three times four is nine.
3
X
1
Jay
Three times four is twelve.
4
3
XSlide15
Statistical Discourse Analysis
Difficulties regarding Time Segments differ (S2
S4) Serial correlation (p8 → p9) Multiple topics Branches of notes (→→ )Strategies Breakpoint analysis + ModelMultilevel analysis (MLn, HLM) Test with I2 index of Q-statisticsModel with lag outcomes, KC (-1) Store path: Identify prior turn
Vector Auto-Regression Lag explanatory variables e.g., Valid (-1), Girl (-1)
Valid (-2)
1
2
3
8
4
5
6
7Slide16
Statistical Discourse Analysis
Outcome Difficulties Ordered outcome (KC 1-5)
Infrequent outcomes (00010)Strategies Ordered Logit / Probit Logit bias estimatorSlide17
Statistical Discourse Analysis
Explanatory model Difficulties People, Groups & Topics
differ Mediation effects (X→M→Y) False positives (+ + + +)Strategies Multilevel analysis Multilevel mediation tests 2-stage linear step-up procedure Slide18
Results – KC Phases
KC Phase
% of Posts1) Sharing Information
60
2) Exploring Dissonance
3
3) Negotiating Meaning
16
4) Testing / Modifying
4
5) Agreeing / Applying
17Slide19
Results: Summaries as Pivotal PostsEach discussion averaged
1 pivotal post (2 time periods)Slide20
Results - KC Patterns
Knowledge Construction Phase
Post Number
No Regressive
Segments
Pivotal Posts
→
Distinct
Segments
No Regressive
Segments
Segments Skipped
KC phases
Slide21
Predicting Pivotal PostsSynthesizer
Pivotal Post
Extensive SummaryWrapper
Role
Current PostSlide22
Time 2 posts ago
Previous post Role
Current post Knowledge ConstructionSummaryAfter 1st pivotal post
New Idea (-1)
after 1st pivotal post
Respond
(-2) after 1
st
pivotal post
Wrapper
Synthesizer
Predict Knowledge ConstructionSlide23
KC pattern KC phase 1 KC phase 3 or 5 (Share) (Negotiate Meaning or Agree/Apply)
Few KC phases 2 or 4 (Dissonance, Testing)Pivotal post
Extensive Summary often By Synthesizer or Wrapper usuallyExtensive Summary Showed higher KC Elevated KC of subsequent postsSummary of ResultsSlide24
Teacher / Designer Assign Synthesizer Role - Increase midway summaries and elevate KC - Simple, effective intervention
Productive online discussions do not require all phases
Researcher Empirically test Gunawardena et al’s KC model New method for analyzing online discussion - Statistically identifies pivotal posts & segments - Test hypotheses about relationships among posts - Examine variables at multiple levels - Examine differences over TimeImplicationsSlide25
Further QuestionsWith many choices of dimensions for the breakpoints, which one(s) should we use?What do identification of same vs. different breakpoints across different dimensions tell us?How can we do meta-analyses of multiple data sets with somewhat different codes?
Which analyses (qualitative and/or quantitative) might be fruitful on the same data set?