Details 1 Capetown 7814 In Class Experiment 2 2 Capetown 7814 Procedure Ia General Rules Unless the experimenter says otherwise you may NOT show your cards to anyone else There is NO talking ID: 489566
Download Presentation The PPT/PDF document "PODER" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
PODER
Details
1
Capetown
7/8/14Slide2
In Class Experiment 2
2
Capetown
7/8/14Slide3
Procedure Ia
General Rules
Unless the experimenter says otherwise, you may NOT show your cards to anyone else
There is NO talkingRecord your earnings honestly
Everyone MUST reveal their cards to the experimenter
Specific Rules
You’ll be given 4 cards
Each time you’ll show one card
Majorities rule
3
Capetown
7/8/14Slide4
Procedures Ib
Majorities
Experimenter will count # with same type of card
Those in majority (picking same type of card) earn the HIGH payoffThose in the minority (picking the same type of card) earn the LOW payoff
Payoffs
HIGH Payoff (majority):
10 ECU
LOW Payoff (minority): 0
ECU
Questions?
4
Capetown
7/8/14Slide5
Treatment 1
Pick up the 2 BLACK cards
(Match on suit)
Pick one and put both cards against your chest
Show me your choice (all at once)
Count the number matching each suit
Record the majority suit
Repeat
5
Capetown
7/8/14Slide6
Treatment 2
Pick up the 2 RED cards
(Match on suit)
Pick one and put both cards against your chest
Show me your choice (all at once)
Count the number matching each suit
Record the majority suit
Repeat
6
Capetown
7/8/14Slide7
Treatment 3
Pick up 1 RED card and 1 BLACK card
(Match on Color)
Pick one and put both cards against your chest
Show me your choice (all at once)
Count the number matching each color
Record the majority color
Repeat
7
Capetown
7/8/14Slide8
Treatment 4
Pick up ALL the cards
(Match on Suit)
Randomly choose 1 individual – pick and reveal card choice
Everyone else: pick one card and put it against your chest
Show me your choice (all at once)
Count the number matching each suit
Record the majority suit
Repeat (if necessary)
8
Capetown
7/8/14Slide9
What exactly have we just done?
Implementation of a game
People “received” incentives, (see Induced Value Theory)
Example: Coordination:
Induced values mapping actions to outcomes
Definition of a group decision institution
Predictions over “treatments”
9
Capetown
7/8/14Slide10
Problems with this design?
Answer 1Answer 2
10
Capetown
7/8/14Slide11
Practical Design Considerations
7/8/14
11
CapetownSlide12
What must be designed?
“Laboratory experimental design involves designing a microeconomic system”Vernon Smith, AER, December, 1982
Environment:
Agents (Number, type, motivation)Commodities -- what do decisions get made over?
Endowments -- what do the decision-makers have at the outset?
Mechanism by which learning can occur (search opportunities, practice)
Institution:
Decisions available to subjects
Rules about choices
Rules about communication
Connection between decisions and payoffs
12
Capetown
7/8/14Slide13
Fatal
Errors in Design
Inadequate or inappropriate incentive
Nonstandardized
instructions
Inappropriate context
Uncontrolled effects of psychological biases
Insufficient
statistical power
Loss of control due to deception or biased terminology
Failure to provide a calibrated baseline
Change in more than one factor at a time
13
Capetown
7/8/14Slide14
Incentives: Induced Value Theory
Smith (AER 1976; AER 1982)
In many experiments the experimenter wants to
control
subjects’ preferences. How can this be achieved?
Subjects’ homegrown preferences must be “neutralized” and the experimenter “induces” new preferences. Subjects’ actions should be driven by the induced preferences.
Reward Medium: Money
Note:
In other situations experimenter may be interested in homegrown preferences:
Assess some other preference: – e.g. fairness over money (sharing). Money allows other motives or norms to be explored.
Money may function as the “price” of other motives:
e.g
altruism = willingness to forego money
14
Capetown
7/8/14Slide15
Incentives Continued
Minimal Conditions for Control
Monotonicity/nonsatiation
:
Subjects must prefer more of the reward medium to less and not become satiated.
Salience:
The reward depends on a subject’s actions (note: show up fee is not salient).
Dominance:
Changes in a subject’s utility from the experiment come predominantly from
the reward medium
and the influence of the other motives is negligible (this assumption is the most critical).
If these conditions are satisfied, the experimenter has control of the subjects’ preferences, i.e., there is an incentive to perform actions that are paid.
15
Capetown
7/8/14Slide16
Analysis of 74 studies
about different topics with no, low and high financial incentives.
Camerer & Hogarth, 1999.
Incentives Continued:
Effects?
16
Capetown
7/8/14Slide17
Incentives Continued: Other considerations
In experiments in which incentives have an effect, the difference between no and low incentives is often bigger than the difference between low and high incentives. (Forsythe, et al., Games and Economic Behavior 1994)
Even very high stakes typically don’t change behavior
(Cameron, 1999)
Higher incentives may lead to a reduction of the variance of decisions (
Smith&Walker
,
IntJGameTheory
1993)
Incentives and homegrown preferences
(Cardenas and
Ostrom
, 2004; Barr and Serra, 2010).
17
Capetown
7/8/14Slide18
Uncontrolled Psychological
biases
Loss aversionAvoid losses or zero payoff options
Status quo bias
Avoid accidentally anchoring subjects
Experimenter demand: experimenter can accidentally set the status quo by signaling expected
behavior
In the field, status quo may be very strong
Endowment effect
Willingness to accept v. willingness to
pay
Emotion
Ss may vary in their mood
18
Capetown
7/8/14Slide19
Insufficient Statistical Power
You must have enough data to do a statistical test
Plan ahead – decide what test you want to do and run the experiment that will let you do
it
“Decide what regression you want to run and then design the experiment to give you what you need to run it.”
Ernst Fehr, January, 2005.
Avoid too many
treatments
Complete Factorial Designs
(# factors)*(#factors)*(#factors)
Calculate your power test (see
Duflo
et al. for details)
19
Capetown
7/8/14Slide20
Nuts and Bolts
7/8/14
20
CapetownSlide21
First Steps (Practical Advice)
First step: Develop a Research Question.
Lab or field? Or a combination?
Begin with a theory. Translate theory to lab/field setting.
Begin
with phenomenon. Design experiments to dissect
Begin with something you want to measure. Design experiment to measure it.
21
Capetown
7/8/14Slide22
Second Steps:(After the Question/Theory)
InstrumentationTailor game/instructions to target population
Paper/Pencil or Computer?Timeline of experiment
InstructionsSampling/Randomization
What subject pool?
How will Treatment be randomized?
Analysis Plan
What are the units of analysis
Power tests
22
Capetown
7/8/14Slide23
A simplified trust game
23Slide24
A Behavioral Measure of Trust
0
20
35
60Slide25
A Simple Risk Measure
Valencia 2011 Trust
25Slide26
A Simple Time Preference Measure
26
More than 35% never wait.Slide27
TimeLine Example – Eckel/Wilson
27
Capetown
Subject Check-in
General Instructions
Risk Task (Everyone)
Public Officials Risk Choice for Citizens
Time Discounting Task
Within Group Trust Task
Public Official/Citizen Trust Task
Charitable Giving Task (social distance)
Charitable Giving Task (social distance + choice of charity)
Choice of Task to Pay
Questionnaire
Payment
7/8/14Slide28
Second Steps:(After the Question/Theory)
Instrumentation
Construct Validity – how will I test what I want to test?Paper/Pencil or Computer?
Timeline of experimentInstructionsSampling/Randomization
What subject pool?
How will Treatment be randomized?
Analysis Plan
What are the units of analysis
Power tests
28
Capetown
7/8/14Slide29
Subject Selection, I
Convenience Samples: students
Students advantages: Convenient, inexpensive and relatively homogeneous
Student disadvantages:May behave differently from target population, young, educated, and talk to each other (diffusion)
Classroom:
Representative sample of students
Environment might affect behavior:
Lab:
May select certain students
Neutral environment
Data: Eckel and Grossman
ExEc
:
Students give more to charity in the classroom than in the lab
Why?
29
Capetown
7/8/14Slide30
Subject Selection, II
Specialized Groups:
ElderlyProfessionals
Medical cases
Poor
Residents of hurricane-vulnerable areas
Public officials
Population Samples
Pluses: External validity, Heterogeneity
Minuses: Costly,
risk of decreased
control, heterogeneity
30
Capetown
7/8/14Slide31
Subject selection III
Subject selection should suit the question you are askingTheory testing:
Independent of subject characteristics?
Policy (measurement or institutional design):
Target group
subjects
Examples:
WEIRD people
(
Henrich
, et al. 2010)
People from other cultures
(Barr and Serra 2010)
31
Capetown
7/8/14Slide32
Second Steps:(After the Question/Theory)
Instrumentation
Construct Validity – how will I test what I want to test?Paper/Pencil or Computer?
Timeline of experimentInstructions
Sampling/Randomization
What subject pool?
How will Treatment be randomized?
Analysis Plan
What are the units of analysis
Power tests
32
Capetown
7/8/14Slide33
Nuts and Bolts, I
Lab log.
IRB and EthicsPilot
experiments.Lab set-up
Subject registration
Experimenter(s
)
Monitor(s
)
Randomizing Devices
Instructions
Subject confidence (non-deception)
33
Capetown
7/8/14Slide34
Lab Book (Lupia & Varian 2010)
1. State your objectives.
2. State a theory. 3. Explain how focal hypotheses are derived from the theory if the correspondence between a focal hypothesis and a theory is not 1:1. 4. Explain the criteria by which data for evaluating the focal hypotheses were selected or created.
5. Record all steps that convert human energy and dollars into datapoints.
6. State the empirical model to be used for leveraging the data in the service of evaluating the focal hypothesis. (a) All procedures for interpreting data require an explicit defense. (
b
) When doing more than simply offering raw comparisons of observed differences between treatment and control groups, offer an explicit defense of why a given structural relationship between observed outcomes and experimental variables and/or set of control variables is included.
7. Report the findings of the initial observation.
8. If the findings cause a change to the theory, data, or model, explain why the changes were necessary or sufficient to generate a more reliable inference.
9. Do this for every subsequent observation so that lab members and other scholars can trace the path from hypothesis to data collection to analytic method to every published empirical claim.
ELNs
: OneNote in Microsoft or
Growlybird
Notes for the Mac (http://www.growlybird.com/GrowlyBird/Notes.html
)
34
Capetown
7/8/14Slide35
Nuts and Bolts, I
Lab log.IRB and Ethics
Pilot experiments.Lab set-up
Subject registrationExperimenter(s)
Monitor(s
)
Randomizing Devices
Instructions
Subject confidence (non-deception)
35
Capetown
7/8/14Slide36
Ethics
IRB keeps us honest (some countries don’t have)Focus on potential harm to subjects
Consent, debriefing limit harm, but may impact sampleBalance between potential benefit and riskField experiments:
No consent process! Unwitting subject, high potential costFindley et al 2014. – no consent, no debriefingCorrespondence studies on discrimination (more later)
Intervention studies: elections, political institutions
Facebook study on emotional contagion: no consent, potential risk, very low potential benefit
7/8/14
36
CapetownSlide37
Nuts and Bolts, I
Lab log.
IRB and EthicsPilot experiments.
Lab set-upSubject registrationExperimenter(s
)
Monitor(s
)
Randomizing Devices
Instructions
Subject confidence (non-deception)
37
Capetown
7/8/14Slide38
Nuts and Bolts, II
Subject questions“Learning periods”
ExperimentRecording dataTermination of experiment
DebriefingSubject paymentBankruptcy
Backup plan
38
Capetown
7/8/14Slide39
Writeup and reporting
Biases in published dataRegistration and CONSORT
7/8/14
39
CapetownSlide40
Biases in published data
Selective reporting + publication bias => many published studies have p=.05.Data mining and selective presentation of results have been a concern in economics for a long time
These concerns are not limited to Economics:Medical trials, Ioannidis (2005, “Why most published research findings are false”)
Psychology, Simmons et al. 2011, “False - positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant”)Political science
, Humphreys et al. (2012, “Fishing”)
Finds affect millions of people. How to fix?
7/8/14
40
CapetownSlide41
Example: (Gerber and Malhotra
, AJPS) 7/8/14
41
CapetownSlide42
Economics, “Star Wars” (Brodeur et al 2013)
7/8/14
42
CapetownSlide43
Consort/Registration (Humphreys et al, 2013)
Capetown
43
CONSORT Statement: improve the reporting of a randomized controlled trial (RCT), enabling readers to understand a trial's design, conduct, analysis and interpretation, and to assess the validity of its results.
http://www.consort-statement.org
/
7/8/14Slide44
Registration
Benefit:Limits selective reporting/fishingRounds out “body of evidence”
Forces researcher to think through design, statistical analysisPotential costsLimits exploratory research
Serendipitous findings may be hard to publishBut: frees it from the burden of (false) presentation as formal hypothesis testing.
7/8/14
44
CapetownSlide45
General Remark: lab v. field
Lab has greater internal validity
Lab is cheap, field is costly
Lab mistakes can be fixed; often not so in fieldStudents v. population
Population has higher variance, harder to detect effects
Selection bias is not limited to lab
Greater monitoring costs to ensure population sample
45
Capetown
7/8/14