by Brooke Culclasure PhD May 3 2016 Introductions Presenter is Dr Brooke Culclasure research director in the Riley Institute at Furman University and principal investigator of numerous studies across SC ID: 743074
Download Presentation The PPT/PDF document "An Evaluation Primer: A Plan for Measur..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
An Evaluation Primer: A Plan for Measuring Program Quality and Outcomes in After School and Summer Programs
by Brooke Culclasure, Ph.D.
May 3, 2016Slide2
Introductions
Presenter is Dr. Brooke Culclasure, research director in the Riley Institute at Furman University and principal investigator of numerous studies across S.C.
Introductions of session attendees/program affiliation/description of current evaluation activitiesSlide3
Format of Presentation
Discuss why evaluation is importantDiscuss why quality is important, related to evaluation
Discus past outcomes evaluations and their link to quality
Discuss available tools to measure quality
Discuss QSA
and how it has been used
Breakout groupsSlide4
Why is evaluation important?
Provides us with useful information to help us know our strengths and weaknesses, how to improve our program, and better serve our students
Helps us highlight successes and be able to tell our success stories
Provides us with solid data to help us compete for resourcesSlide5
Why is quality important?
In after school and summer learning programs, research
has
shown that
o
nly
high-quality programs produce positive outcomes for
youth
Many studies have looked at test scores, engagement, and other factors…BUT programs rated as low quality do not show impact
Studies by scholars such as
Vandell
, Noam and others all focus on quality as key in determining outcomesSlide6
How do we assess program quality?
Becomes obvious, then, that we must focus on quality
Numerous tools available, some are self-assessment and some are conducted by external evaluators
There are 3-4 widely used tools, we will go into detail about one today
Talk first about why self-assessment is a good choiceSlide7
How do we assess program quality?
Why Self-Assessment
?
It helps program providers to understand overall program quality
and
identify steps needed for
improvement
It is a low-stakes
process
It can focus on program needs and priorities while keeping areas outside the focus from
slipping
Low costSlide8
How do we measure quality? Four widely used tools
Youth Program Quality Assessment (PQA)
A Program Assessment System Training (APAS):
O
nly assessment that helps afterschool programs link quality to youth outcomes!
Dimensions of Success
O
bservation Tool (DOS):
STEM focused
Quality Self Assessment (QSA)Slide9
Quality Tools
What is the QSA Tool
? (NY State Network)
The QSA Tool is organized around 10 elements of
quality
It uses a set of indicators to help program providers determine their strengths and areas in need of
improvement
It is structured to lead to an action plan for increasing program
quality
The tool was developed by experienced program and TA providers to be relevant and
reasonableSlide10
What Makes a Quality Program?
Ten Essential Elements of a Quality Afterschool Program
Environment
and Climate
Administration and Organization
Relationships
Staffing and Professional Development
Programming and Activities
Linkages Between Day and Afterschool
Youth Participation and Engagement
Parent, Family, and Community Partnerships
Program Sustainability and Growth
Measuring Outcomes and Evaluation
Source: NY State Network for Youth SuccessSlide11
QSA Tool vs. Evaluation
QSA Tool
:
provides structure for dialogues around outcomes and quality.
strengthens communication between stakeholders.
helps build a collective vision.
uses data gathered to inform practice and build capacity.
is often facilitated by program staff and/or other stakeholders.
Formal Evaluation
:
takes a more formal approach to assessing programs to determine what their outcomes are.
often uses data gathered for accountability and compliance.
has higher stakes and less gray areas.
is often facilitated by an external observer.Slide12
Cycle of ImprovementSlide13
Breakout Groups
Divide into breakout groupsReconvene for discussion/reporting back
Questions: Please contact Brooke Culclasure, 864-294-3236,
brooke.culclasure@furman.edu