Frode Svartdal University of Tromsø Oct 2013 Extinction Basics Extinction is defined in terms of a reinforcement process Extinction contingencies The stimulus S R or US is discontinued ID: 390223
Download Presentation The PPT/PDF document "The partial reinforcement extinction eff..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The partial reinforcement extinction effect (PREE)
Frode Svartdal
University
of
Tromsø
Oct. 2013Slide2
Extinction: Basics
Extinction is defined in terms of a reinforcement process
Extinction contingencies
The
stimulus
(S
R
or US) is discontinued
The learning
contingency
is discontinued
Extinction process
The conditioned response is reduced (strength, frequency, etc.)
Relearning, … not forgettingSlide3
Catania, 1984)
Extinction: Basics
Operant conditioningSlide4
Extinction: Basics
Classical conditioningSlide5
Factors affecting the extinction rate
In general:
Fast acquisition / high rate of responding
fast extinctionAmount of rewardHigh
fast extinctionVariabilityStimulusResponseReinforcementSome forms of learning do not extinguish (easily)Evaluative conditioning (e.g., Diaz, Ruiz, & Beyens, 2005)
= high ext. persistenceSlide6
Factors affecting the extinction rate
Partial Reinforcement Extinction Effect
Partial (Intermittent) Reinforcement (PRF)
increased extinction responseContinuous Reinforcement (CRF)
reduced extinction persistenceSlide7
First demonstrations
Operant conditioning;
free operant; rats;
Skinner (1938)
Classical conditioning;
blink response; students;Humphreys (1939)
100%
50%Slide8
Ferster & Culbertson, 1975
Free operantSlide9
PRF
CRF
Free operant
Compared to CRF:
PRF
higher asymptotes
more persistent
responding
under
extinction
EXTINCTIONSlide10
Rats, maze running speed under extinction
(Weinstock, 1954)
CRF
PRF (30%)Slide11
Classical conditioning (rats): PREE
25%
50%
100%
Extinction
PRF response rate
LOWER
than CRF
response rate
15%Slide12
Classical conditioning; eyelid; human subjects
(Svartdal & Flaten, in prep.)Slide13
Operant conditioning; humans;
Svartdal, 2003, Exp. 4Slide14
Conclusions (… preliminary)
PREE is a very robust outcome
Measures & species
Bar pressing, rats
Maze running, ratsPecking, pigeonsBlink reflex, humans, rabbits…Contingency
Operant/instrumentalDiscrete trialFree operantClassicalSlide15
But…
How general is the PREE?
Reversed PREE observed under some conditions
Generalized PREE observed under some conditions
Alternative methods of analysisNevin (1988): ”PREE is an artefact because of wrong method of analzing extinction performance”Response unit issuePREE or not dependig on how the response is defined (Mowrer & Jones, 1945!Slide16
Reversed PREE
What happens if the subject
is exposed to a mixture
of PRF and CRF contingencies?Slide17
Reversed PREE
Pavlik & Carlton, 1965: Rats; bar pressing, free operant
Gr. 1: Single contingency; CRF
Gr. 2: Single contingency; PRF
Gr. 3: Two signalled schedules alternated for the same subjects; CRF + PRFSlide18
Reversed PREE
Conventional
PREESlide19
Reversed PREE
Reversed
PREESlide20
Reversed PREE
Pavlik & Carlton (1965):
Single reinforcement schedules (CRF vs. PRF) in between-groups experiments
PREETwo schedules (CRF vs. PRF) for the same subjects
Reversed PREEOther researchReversed PREE observedGeneralized PREE (overall increased persistence, but no difference between conditions) Conventional PREE rarely if ever observed in within-subjects manipulations of CRF - PRFSlide21
PREE as a generalization: Ecological validity
If applied to a situation with a very specific schecule for a specific behavior
PREE
Example
:
Single mother – child is begging for toys only from mom
If applied to various situations with mixed contingencies Reversed PREE
Generalized PREE
Example:
Mother and father
– child begs for toys from bothSlide22
Response unit issueSlide23
Free operant responding: What is the response unit?
Mowrer & Jones,1945:
What should be counted as the response unit - single responses or the unit of responses required for reinforcement?
Free-operant
Intermittent reinforcemet, e.g., FR4Slide24
Response unit
FR4
Reinforced responsesSlide25
PREE
Total responses
Total responses /
reinforcement ratio
Reversed PREESlide26
Nevin: PREE is an artefactSlide27
PREE: Alternative analyses
Nevin, 1988: Behavioral momentum
”RPREE” is the rule – the response is stronger following CRF
in free-operant responding (but not in discrete-trial experiments)
following extended trainingExtinction performance
Traditional measure: Number of responses Nevin: Slope of the extinction curveSlide28
PREE
RPREE
SHORT
LONG
Absolute
number
of
responses
Relative
to initial
ext
response
level
Nevin, 1988Slide29
PREE vs. RPREE – important variables
Dependent measure
No. of responses vs. relative change
Type of situations
Free operant vs. discrete trialComplexity of situationOne vs. more schedules (e.g., multiple schedule)DesignBetween groups vs. within subjectsSlide30
PREE typically observed
Measure
Number of responses
Situation
Discrete trial
Schedule
Single
Design
Between-groups manipulation of reinforcer rate
Other
CRF schedule must be 100%Slide31
PREE: My interests
Interaction PREE & Reversed PREE
Cognition (verbalization) related to behavioral PREESlide32
The experimental situation
”Computer responses”
presented
Left, right
Subject responsesrecorded Left, rightSlide33
The experimental situation
Task
Complete a four-response chain of responses started by the computer
E.g.: Computer: L R
Subject: R LInstructed task: Identify and apply the functional rule(s)”Obtain as many correct answers as you can.”
Rules (depending on experiment)”Repeat computer sequence””Reverse computer sequence”Feedback (visual, autitory) for correct answer; nothing happens if answer is incorrectSlide34
The experimental situation
Manipulations (between
groups and/or within groups)
Rule
Reverse (typically used)
Repeat
Contingency
CRF (100%)
PRF (20-60%)Slide35
The experimental situation
Reward rate manipulated
Between groups
Within subjects (multiple schedule)
Discrete trial situation; fixed number of trials180 acquisition trials40 extinction trialsSlide36
Conventinal PREE; operant responding; students; Svartdal, 2003, Exp. 4Slide37
Reversed & conventional PREE; operant responding; students; Svartdal, 2000
Reversed PREE
Purpose: Explore the relationship between PREE and RPREE
PREE vs. RPREE: Contradiction or compatible effects?
MethodIndependent groups: PRF and CRF
Within: CRF and PRFSlide38
Svartdal, 2000 ctd.
Multiple schedule, alternating
Group 40/40
Half trials (signalled): 40%
Half trials (signalled): 40%Group 80/80
Half trials (signalled): 80%Half trials (signalled): 80%Group 80/40Half trials (signalled): 80%Half trials (signalled): 40%
PRF
”CRF”
”CRF”
+
PRFSlide39
PREE
80%
40%
* No. of responses: RPREE
* Relative change: No differenceSlide40
Svartdal, 2000 ctd.
Relationship between schedule components
Simplest assumption: Modulation between component schedules:
60% + context = 60%
reference 60% + context = 100%
reduced persistence 60% + context = 20% increaced persistenceSlide41
Performance of a 60% schedule depending on
other schedule = 100%, 60%, or 20%
Svartdal, 2000Slide42
Svartdal, F. (2000).
Persistence during extinction: Conventional and Reversed PREE under multiple schedules.
Learning and Motivation, 31,
21-40.Slide43
Cognition in PREE
Currently:
Strong
cognitive arguments to interpret conditioning in terms of cognition
Classical conditioning: Lovibond & Shanks, 2002Operant conditioning: Shanks & St John, 1994
Implicit learning doubted: Shanks, 2005Extinction: Lovibond, 2004Basic argument:CONTINGENCY CONSCIOUS APPREHENSION BEHAVIORAL CHANGE
CONTINGENCY CONSCIOUS APPREHENSION NO BEHAVIORAL CHANGE
Large number of studies supporting this assumptionSlide44
Cognition in PREE
So, since the behvioral PREE is very robust, a ”cognitive PREE” must be easy to measure
Basic prosedure:
Behavioral acquisition under 100% vs. 60% reinforcer rate
Measurement of verbalized PREESlide45
Cognition in PREE
Prediction of persistence:
”How likely is it that you will continue
responding if reward no longer appears?”
Several experiments have
demonstrated
no
sensitivity
to learning history in
predictionsSlide46
3 extinction trials;
immediate behavioral
sensitivity
No difference
in predictions
Svartdal & Silvera, in prep.Slide47
Cognition in PREE
Retrospective judgments:
”How many responses did you emit after
reward no longer appeared?”
Subjects are very accurate in descrbing their own behavior, including their own extinction persistence Slide48
Cognition in PREE
Svartdal, F. (2003).
Extinction after partial reinforcement: Predicted vs. judged persistence.
Scandinavian Journal of Psychology,
44, 55-64. Slide49
Meta-cognitive PREE?
We all have long experience with various contingencies
Maybe a ”meta-cognition” evolves:
Uncertain outcomes
PersistCertain outcomes Quit Slide50
Meta-cognitive PREE?
Scenarioes presented to subjects, manipulation
Reliable outcome vs.
Unreliable outcome
Persistence judgments of behaviorSlide51
Meta-cognitive PREE?
Naive students: No effect of
outcome manipulationSlide52
Meta-cognitive PREE?
Psychology students
(have read about PREE)
Naive studentsSlide53
Meta-cognitive PREE?
Svartdal, F. (2000). Persistence during extinction:
Are judgments of persistence affected by contingency
information?
Scandinavian Journal of Psychology
, 41, 315-328. Slide54
PREE: Theory
Mowrer & Jones: Diskriminasjonshypo- tesen
PRF:
Læringbetingelsene
ekstinksjonsbetingelseneGeneralisering til ekstinksjon
CRF:Læringbetingelsene # ekstinksjonsbetingelseneLiten generalisering til ekstinksjonSlide55
PREE: Theory
Amsel: Frustrasjonshypotesen
PRF:
Forventning om belønning
frustrasjon når belønning uteblirFrustrasjons-cues assosieres med læringssituasjonen
Under ekstinksjon: Frustrasjon pga uteblitt belønningLæringssituasjonen ekstinksjonssituasjonenCRF: Frustrasjon oppstår ikke under læringLæringssituasjonen # ekstinksjonssituasjonenSlide56
PREE: Theory
Capaldi: Sequential hypothesis
PRF:
Ikke-belønnede trials blir signal på at belønning snart vil følge: … N N N R N N N R …
Dvs.: Det opparbeides en forventning om belønning når belønning uteblir
Under ekstinksjon: Mange responser pga forventning om belønningCRF:Ingen erfaring med uteblitt belønning under læringUnder ekstinksjon: Få responserSlide57
PREE: Theory
Status:
Diskriminasjonshypotesen står svakt
Amsels hypotese står rimelig sterkt
Capaldis hypotese står ganske sterktNevins modell: Ingen hypotese i vanlig forstand
Discrete-trial-situasjonenCapaldi og Amsel dominerendeFri-operant-situasjonenSvak teoretisk forståelse