41 Ratio of Understanding Users papers to Systems Tools Architectures and Infrastructure papers submitted to the Interaction Beyond the Individual track at CHI 2011 Trouble Exponential Growth ID: 413914
Download Presentation The PPT/PDF document "Michael S. Bernstein, Mark S. Ackerman, ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Michael S. Bernstein, Mark S. Ackerman, Ed H. Chi, Robert C. MillerSlide2
4:1
Ratio of “Understanding Users” papers
to “Systems, Tools, Architectures and Infrastructure” papers
submitted to the Interaction Beyond the Individual track
at CHI 2011.Slide3
Trouble: Exponential Growth
Your usage
data is not really compelling
because only a
small fraction of Facebook
is
using the application.
Worse
, your numbers
aren’t growing in anything like an exponential fashion.
– CHI
metareviewer
,
paraphrasedSlide4
Suggestion: Exponential Growth
Separate evaluation
of
spread
from
steady-state
.
Which claim is the paper making?Slide5
Trouble: Snowball Sampling
The authors’ choice of study method – snowball sampling their system by advertising within their own social network – potentially leads to serious problems with validity.
– CHI
metareviewer
,
paraphrasedSlide6
Suggestion: Snowball Sampling
Snowballing is
inevitable
in social systems. It is fundamental to how they operate.Slide7Slide8
Novelty
Between a Rock and a Hard ScienceSlide9
sociotechnicalSlide10
socio
technical
studiers
buildersSlide11
socio
technical
studiers
builders
Fatal Flaw Fallacy
[Olsen]
Ecological validity at the cost of internal validity
[Ackerman 2000], [
Barkhuus
and Rode 2007], [Chi 2009], [Greenberg and Buxton 2008], [Kaye and
Sengers
2007], [
Landay
2009]
, [Lieberman 2003
], [Olsen 2007], [
Zhai
2003]Slide12
socio
technical
studiers
builders
Show us elegant complexity.
(simple ideas that enable complex scenarios)
That’s it? What is possible now that wasn’t before?
Nothing — but focus on emergent social activity.
Can you add
multitouch
?
Not using IE8.
We let people type messages up to 140 characters.Slide13
socio
technical
studiers
builders
Build a technically interesting system
(that is hard to spread or evaluate), or
Simplify to a system with socially interesting outcomes
(that builders find less novel).Slide14
Build a technically interesting system
(that is hard to spread or evaluate), or
Simplify to a system with socially interesting outcomes
(that builders find less novel).
The contribution needs to take one strong stance or another. Either it describes a novel system or a novel social
interaction. If it’s a system, then I question the novelty. If it’s a social interaction, it needs more development.
– CHI
metareviewer
, paraphrasedSlide15
Create a shared understandingof research contributionsSlide16
social
technical
New forms of social interaction
Shared organizational memory [Ackerman 1994]
Designs that impact social interactions
Increasing online contribution [
Beenen et al. 2004]Enable fluent social interaction in a new domain Socially translucent systems [Erickson and Kellogg 2000]Slide17
social
technical
Designs collecting or powered by social data
Wikidashboard
[
Suh
et al. 2008]; sense.us [Heer et al. 2007]Algorithms to coordinate crowds or derive signal from social data
Collaborative Filtering [
Resnick
et al. 1994]; Iterate-and-Vote [Little et al. 2010]
Platforms and infrastructures
TurKit
[Little et al. 2010]Slide18
social
technical
Paired contributions can increase each others’ value
×
ManyEyes
[
Viégas
et al. 2007]Slide19
In
c
onclusion
introduction:
What are our millennium challenges?
What is our relationship with industry
and walled gardens?
How can (and should) we evolve
our standards of proof?Slide20
Michael S. Bernstein, Mark S. Ackerman, Ed H. Chi, Robert C. MillerSlide21