Year 1 Results Jay P Greene Heidi H Erickson Angela R Watson and Molly I Beck AEFP 43 rd Annual Conference Overview Randomly assign classes of 4 th and 5 th graders to receive 3 culturally enriching field trips throughout the school year and measure the impact on student social ID: 755861
Download Presentation The PPT/PDF document "The Effects of Multiple Art Experiences ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The Effects of Multiple Art Experiences on Long term Social Emotional Outcomes of Urban Elementary School Students: Year 1 Results
Jay P. Greene, Heidi H. Erickson, Angela R. Watson, and Molly I. Beck
AEFP 43
rd
Annual ConferenceSlide2
Overview
Randomly assign classes of 4
th
and 5
th
graders to receive 3 culturally enriching field trips throughout the school year and measure the impact on student social-emotional and academic outcomes.
Results:
Increase desire
to be cultural consumers
Increase in Math and ELA test scores
Increase survey effort
Null effect for art participation and social perspective takingSlide3
Motivation:
Field trips are a long standing tradition for schools
Increase in schools canceling field trips
(
Ellerson
& McCord, 2009)
Teachers report decline in arts education and field trip, particularly among disadvantaged students
(Government Accountability Office, 2009)
Principal reported pressure from accountability standards and tight budgets
Cultural institutions report fewer student groups attending and that
a
dult attendance at art institutions is also declining
(
Rabkin
&
Hedberg
, 2010)Slide4
Previous literature
Limited rigorous research on the benefits from field trips
Observational studies
Students involvement in the arts is associated with higher academic
performance
(
Ruppert
,
2006;
Jægar
and
Møllegarrd
, 2017)
Meta-analysis on arts integration programs find 4 percentage point increase in achievement
(Ludwig, Boyle, & Lindsay, 2017)
Student who attend multiple cultural institutions experienced academic benefits in the short term
(
Lacoe
, Painter, & Williams, 2016
)
Gold standard studies
Students
randomly assigned to receive a field trip to an art museum experienced an increase in tolerance, critical thinking, and desire to consume art
(
Greene et al., 2014; Bowen, Greene, &
Kisida
, 2014;
Kisida
, Greene, & Bowen, 2014)
Student randomly assigned to attend live theater demonstrated higher levels of content knowledge, tolerance, and social perspective
taking
(Greene et al, forthcoming
)Slide5
Research Question
Do students
experience social emotional and academic benefits
from multiple field trips to cultural institutions
?
What we add to the literature
Experimental design
Multiple trips to 3 different art institutions
Large, urban school district serving primarily minority and low income students
Survey and administrative data for multiple years
Hypotheses
Expect positive gains in social emotional constructs such as social perspective taking
Expect positive gains in student desire to consume arts
Expect no significant effect in academic achievementSlide6
Research Design: Randomized Control Trial
4 elementary schools in large urban school district
Randomly assign 4
th
or 5
th
grade students to serve as treatment or control
School 1
4
th
Grade-
Treatment
5
th
Grade- Control
School 2
4
th Grade- Treatment5th Grade- Control
School 3
4
th Grade- Control5th Grade- Treatment
School 4
4
th
Grade- Control
5
th
Grade-
TreatmentSlide7
Research Design: Randomized Control Trial
Treatment: Field
trips
to Atlanta Symphony Orchestra, Alliance Theater, and High Museum of Art
Control: Business as usual, field trip to
1 of
the 3 art partners at Woodruff Art CenterSlide8
Data
4 Elementary schools, 4
th
& 5
th
grades, in a large urban school district
~550 Students
Student Surveys
Interest in art consumption and participation
Social
Perspective Taking
Effort
Non-response and careless answers
Administrative records
Demographics, end of year test scores, courses, attendance, discipline recordsSlide9
Pre-Treatment
Comparisons of Treatment and Control Groups
Variables
Control (mean)
Treatment (mean)
Difference (C-T)
Observations
Demographics:
Age in years
10.91
10.88
0.03
429
Female
53.54%
57.17%
-3.63%
530
Non-white
99.37%
98.83%
0.54%
534
Students with Disabilities
17.73%
15.03%
2.70%
468
Attendance
2016
95.09%95.48%-0.40%468201795.57%95.72%-0.15%532Transferred within year20165.88%6.85%-0.97%46820174.29%6.57%-2.28%532Any Disciplinary Infraction10.56%8.22%2.35%5342016 Testscores, combined-0.36-0.29-0.07457ELA-0.40-0.29-0.11455Math-0.31-0.21-0.10455Science-0.29-0.20-0.09455Social Studies-0.30-0.330.03454Baseline Effort MeasuresCareless Answers0.170.030.14*534Non-response0.240.130.11534Social Perspecitve Taking0.000.14-0.15*529Composite Cultural Consumption-0.130.18-0.31***539Composite Cultural Participation-0.040.060.10**534Recall Previous Field Trip68.89%75.21%-6.32%526*** p<0.01, ** p<0.05, * p<0.1Slide10
Research Methodology
Randomized Control Trial: Compare students who were randomly assigned to receive 3 art related field trip to control students who received one
Average Treatment Effect
is a vector of student characteristics
Average test score, combined Z-test in prior year tested subjects
Age
Gender
Student with Disability
Attendance prior year
is a fixed effect for each school
Slide11
Treatment
Effect on Student End of Year Standardized Exams
(1)
(2)
(3)
Composite Test Score
ELA
Math
Treatment
0.124***
0.123**
0.107*
(0.046)
(0.053)
(0.060)
Composite Test Scores 2016
0.841***
0.770***
0.787***
(0.023)
(0.028)
(0.033)
Female
0.022
0.111**
-0.070
(0.047)
(0.052)
(0.060)SWD 2016-0.163*-0.236**-0.066(0.093)(0.100)(0.108)Attendance 20160.985*0.5091.315*(0.550)(0.598)(0.675)Constant-1.017**-0.692-1.191*(0.511)(0.555)(0.634) Observations441441441R-squared0.7690.6930.630Fixed effects for the four participating schools are included in each model. Age is not included in these models because age could not be calculated from administrative records. Standard errors clustered at teacher level in parentheses *** p<0.01, ** p<0.05, * p<0.1Slide12
Treatment Effect on Student Effort, Cultural Consumption and Participation, and Social Perspective Taking
(1)
(2)
(3)
(4)
(5)
Careless Answers
Non-Response
Consumption
Participation
SPT
Treatment
-0.231***
-0.156*
0.340***
0.084
0.113
(0.084)
(0.077)
(0.105)
(0.119)
(0.131)
Careless Answers 2016
0.399***
(0.054)
Non-response 2016
0.025
(0.056)
Cultrual Consumption 20160.487***(0.042)Cultural Participation 20160.551***(0.046)SPT 20160.330***(0.056)Composite Test Scores 2016-0.120***-0.191***-0.0350.0280.025(0.042)(0.037)(0.050)(0.058)(0.046) Observations379379379379374R-squared0.2250.0820.3160.3840.131Fixed effects for the four participating schools, student age, gender, SWD status, and prior year attendance are included in each model. Standard errors clustered at teacher level in parentheses *** p<0.01, ** p<0.05, * p<0.1Slide13
Summary
L
arge, statistically significant, positive effect on tests scores, student effort, and art consumption
Positive but statistically insignificant effect on Social Perspective Taking and art participation
Year 2
Added school engagement questions to survey
Larger sample
6 new elementary schools
N
ew cohort at original 4 schools
Treatment in 4
th
grade in yr. 1 will get 3 more fieldtrips for total of 6Slide14
Thank YouQuestions or Comments
Heidi H. Erickson
hh018@email.uark.eduSlide15
Supplementary InformationSlide16
Sample Items: Art Consumption
If your friends or family wanted to go to an art museum, how interested would you be in going?
How interested are you in visiting an art museum?
Visiting art museums is fun.
I plan to visit art museums when I am an adult.
Art is interesting to me.
I feel like I don’t belong when I’m at an art museum.
I feel comfortable talking about art.
I would tell my friends that they should visit an art museum.
Would you like more art museums in your town?
Do you think your friend would enjoy a field trip to an art museum (such as the High Museum of Art)?Slide17
Sample Items: Social Perspective Taking
How often do you attempt to understand your friends better by trying to figure out what they are thinking?
How often do you try to think of more than one explanation for why someone else acted as they did?
Overall, how often do you try to understand the point of view of other people
?
How
often do you try to figure out what emotions people are feeling when you meet them for the first time?
In general, how often do you try to understand how other people view the situation?Slide18
Descriptive
Statistics for Constructing Outcomes Scales
# of Items
Cronbach's Alpha
Social Perspective Taking
Fall
7
0.76
Spring
7
0.77
Cultural Consumption
Cultural Consumption Composite
Fall
29
0.91
Cultural Consumption Art
10
0.86
Cultural Consumption Theater
10
0.81
Cultural Consumption Symphony
90.89Cultural Consumption CompositeSpring290.93Cultural Consumption Art 100.90Cultural Consumption Theater 100.86Cultural Consumption Symphony 90.90Cultural Participation Cultural Participation, CompositeFall120.82Cultural Participation, Art 40.84Cultural Participation, Theater 40.77Cultural Participation, Symphony 40.75Cultural Participation, CompositeSpring120.83Cultural Participation, Art 40.90Cultural Participation, Theater 40.85Cultural Participation, Symphony 40.76Notes: The scales are constructed by averaging each item in the scale. Each item in the scales was standardized before averaging all items in the scale. The entire scale was then standardized again. Slide19
Study
consent rates by treatment and control
Consent Forms Received
Enrollment
Overall Consent Rates
Treatment
307
352
87.22%
Control
252
340
74.12%
Overall Consent
559
692
80.78%
Enrollment represents students were who eligible to participate in the study. The treatment and control group both consist of two groups of fourth graders and two groups of fifth graders. Slide20
Treatment
Effect on Cultural Consumption
(1)
(2)
(3)
(4)
Composite
Symphony
Theater
Art
Treatment
0.340***
0.217**
0.535***
0.116
(0.105)
(0.095)
(0.101)
(0.092)
Cultrual Consumption 2016
0.487***
(0.042)
Symphony Consumption 2016
0.547***
(0.039)
Theater Consumption 2016
0.304***
(0.055)
Art Consumption 20160.478***(0.037)Composite Test Scores 2016-0.035-0.072-0.0580.044(0.050)(0.046)(0.053)(0.057) Observations379379379379R-squared0.3160.3430.2250.322Fixed effects for the four participating schools are included in each model. Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1Slide21
Benefits from Field Trips
Based off past literature
Cultural consumption and participation
(Greene et al, 2014; Greene et al, forthcoming)
Social Perspective Taking
(
Gehlbach
et al., 2008; Greene et al., forthcoming)
New to our project
Administrative Records
Test scores
, attendance, course selection, grades, discipline records
Survey effort as a proxy for student engagement
Careless answers and item non-response
(
Zamarro
et al., 2016;
Hitt
, Trivitt, Cheng, 2016)Hypotheses
Expect positive gains in social emotional constructs such as social perspective taking
Expect positive gains in student desire to consume arts
Expect no significant effect in academic achievement