QRIS Standards Learning Table - PowerPoint Presentation

Slide1

QRIS Standards Learning Table

Session #4: Efficiency: Streamlining QRIS

using

your State Knowledge and

Data-based Experience

Slide2

Introductions and Updates

Introduce the state team (Name, title, agency)

AL, CA, CT, GA, HI, NV, OR, VIUpdate us on what your state team has been working on in the development of your QRIS since our last call.If a certain resource or idea has been particularly helpful, tell us about that. What is your current, most pressing challenge?

2

Slide3

Homework Discussion

AL

, CA, CT, GA, HI, NV, OR,

VIWhat did your state consider in the development of QRIS standards? What type of data are you collecting to inform future revisions?How is your state using research to inform your selection of standards?

Slide4

Overview – of Presentation Today

Data systems and standards

Using data for decision-making in QRIS design and revision

Oregon experience using dataNAEYC experience using dataData efforts (national)KY – slides and notes at the end as a resource

Slide5

QRIS

D

ata

Systems Support ImplementationOnline application (provider portals for uploading documents, connecting to relevant resources)Data import from other systems (regulation, registry, onsite assessment reports, etc.)Calculating ratings, relationship between standards/policies and program participation and levels of quality

Supporting the QI/TA functions

…Data!

Slide6

Use Data to Eliminate Criteria

If your state data show that all or most providers meet a criterion (no variation by level), consider dropping it.

Or move the criterion to Level 1.

Or if it’s an essential element defining quality, keep it, but don’t use it to determine ratings.

Slide7

Use Data to Move/Revise Criteria

Suppose

your state data show that

very few or no providers meet a criterion.If it’s not an essential element of quality, consider dropping it completely. If it is an essential element of quality, considermoving the criterion to the top Level

or

m

oving it into the CQI section of your QRIS

focusing TA and PD on improvement on it, and not including it in ratings until practice has advanced.

Slide8

Use Data to Find ‘Predictor’ Criteria

With research partners, explore the relationships among criteria.

Is there a set of items that consistently are met?

It is possible to determine statistically if one of them is a “predictor” (if it’s met, very highly likely that the others are also met)

Slide9

Use Data to Revise QRIS

Suppose the data shows that programs in your state QRIS are meeting many criteria (but not all) in the block above where they are now.

Use criteria level data

from the programs currently participating in QRIS to model how programs might score in alternative rating structures – points or hybrid. KY has done that (as resource at the end)

OR – will tell us about OR’s use of research to inform QRIS development

Slide10

Oregon’s Process to Streamline QRIS Standards

Slide11

Brought together two groups

Vision, Mission and Guiding Principles

Slide12

Workgroups Charge

Merged indicators of quality together with intensive input from the Standards Workgroup. (Dec - March)

Reviewed input to the standards. (Jan-Aug)

Provided final recommendations based on input. (May-Sept)

Slide13

Input to the Standards

Gather input from as many interested parties as possible.

Give interested groups both access and time to provide input.

Seek input in a variety of ways.Work for a balance between achievability and perfection. Remember TQRIS isn’t a silver bullet.Considered Recommendations in the larger context of whole system.

Few and powerful

Understandable, relevant and intuitive

Measureable and feasible to monitor

Progressive/distinct among the levels

Goals of the State

Slide14

Sources of input to the

Standards

Development

Standards Workgroup of Statewide PartnersResearch from Oregon’s Quality IndicatorsResearch from Oregon Program of Quality Field TestMonitoring Learning Labs with North CarolinaEarly Learning Guidelines including the Head Start Child Development Early Learning Framework and Birth to Three Early Learning Guidelines

Race to the Top Grant Feedback

Cost Modeling from national TQRIS experts

Cultural and Linguistic Competency Technical Assistance from Build Foundation

Oregon’s

Licensing

Regulations

Slide15

Focus Group input to the

Standards

Development

Focus Groups of 250 child care and early education providers and programs across OregonFocus Groups of 13 Child Care Resource and Referral agenciesFocus Groups of Oregon’s licensing specialists

Focus Groups of health and nutrition specialists across Oregon

Focus Groups of child care union members

Focus Groups of Oregon’s Professional Development

Committee

Slide16

NAEYC

Accreditation Reliability

and Validity Study

Why NAEYC Accreditation is important and caninform QRIS developmentFindings of note in re QRIS and accreditation

Validity

: Meaningful and significant differences in the percent of criteria met in several standards (Teaching, Relationships, Assessment of Child Progress) between programs that achieve accreditation and those that do not.

Content

: Strong positive relationship between meeting lead teacher qualifications and meeting higher proportion of criteria in Relationships;

Content:

O

n

overall diversity and cultural

competence criteria, significant difference

between programs that achieve accreditation

(91% met) and

those that

do not (77% met)

Slide17

NAEYC Accreditation as a Mark of Program Quality

Kyle Snow, Ph.D.

Senior Scholar and Director

Center for Applied Research

National Association for the Education of Young Children

Research

Policy

Practice

Slide18

Goals

Short Overview of NAEYC Accreditation

What do we know about Accreditation

NAEYC Accreditation & QRIS Congruence

Slide19

About NAEYC Accreditation

NAEYC Accreditation is a meaningful tool for program quality improvement for programs

serving children birth through kindergarten.

Developed in the early 1980s

A comprehensive system review and reinvention was fully implemented in fall 2006.

In 2010 an independent review of the site visit and decision protocols was completed validating these processes.

Slide20

A Portrait of Accredited Programs

Program Affiliations:

College/University 5.6%

Employer-Sponsored 7.1% Faith-based Institution 9.5%Head Start 31.7% Hospital 2.4% Migrant services 1.6%

Military Installation 2.4%

Public School 19.8%

US Government Facility 3.2%

Parent Cooperative 11.1%

Indian Tribe .8%

Alaskan Native Village .8%

Corporate Structure

:

 

Non Profit 60.3%

Public Agency 19.0%

For Profit 19.0%

Not stated 2 1.6%

Special Populations:

None 47.6%

Migrant workers 4.8%

Teen parents 23%

Homeless families 17.5%

Other: 19.0%

(incl. 13.5% low income)

As of 11/24/12, there are

6,748

accredited programs serving

592,675

children

Slide21

Quality Improvement

About NAEYC Accreditation

Meet and Maintain

Standards

Becoming a

Candidate

Becoming an

Applicant

Enrollment in

Self-Study

1

2

3

4

4-Step Process

Site Visit

Self-Assessment

Slide22

NAEYC Program Standards and Criteria

NAEYC

Program

Standards

1 – Relationships

2 – Curriculum

3 – Teaching

4 – Assessment of Child Progress

5 – Health

6 – Teachers

7 – Families

8 – Community Relationships

9 – Physical Environment

10 – Leadership

and Management

Standard

Topic

Criteria

Indicator(s)

Sources of Evidence

Slide23

NAEYC Program Standards and Criteria

Possible Outcomes:

Accredited

DeferredDeniedTo be accredited:80% of all assessed criteria in each standard70% on all criteria assessed in each group

All Required Criteria

Slide24

NAEYC Accreditation - Recap

Programs strive to meet NAEYC program standards

Programs self-assess

Assess programs against 10 standards that are research basedPerformance based upon multiple indicators and multiple sources of evidenceProcess allows for self-assessment and NAEYC performance feedbackProcess includes quality indicator and improvement systems

But – does it

really

define quality, can programs attain it, can they maintain it, and can it be monitored?

Slide25

What do we know about Accreditation?

Reinvention and Criteria validation

During

field tests for reinvention, NAEYC (2005) reported significant correlations between criteria (at the standard level) and Early Childhood Environment Rating Scale (ECERS) scores among 70 early

childhood programs.

The strongest relationships were found

between overall quality and program

standards for

relationships,

curriculum, and

teaching.

Validation studies

Sachs

and

Weiland

(

2010): schools

engaged in accreditation scored higher on subscales of the

ECERS-R, and children had higher

scores on the Peabody Picture –Vocabulary Test (PPVT-III)

compared

to peers in programs not accredited (even after controlling for initial PPVT scores).

State-level data within QRIS systems

PA Keystone

STARS program

(OCDEL,

2010) showed significant correlations between accreditation and environmental ratings of program quality (ECERS, ITERS,

SACERS)

Slide26

What do we know about Accreditation?

Trend

Briefs

(http://www.naeyc.org/academy/primary/trendbriefs) communications intended

to share data on programs seeking accreditation and to connect the findings to early childhood research trends.

Releases to date:

Teaching: Accreditation of Programs for Young Children Standard 3

Assessment of Child Progress: Accreditation of Programs for Young Children Standard 4

Relationships: Accreditation of Programs for Young Children Standard 1

Supporting Cultural Competence: Accreditation of Programs for Young Children Cross-Cutting Theme in Program

Standards

Upcoming:

Family Engagement:

Accreditation of Programs for Young Children Cross-Cutting Theme in Program

Standards

Slide27

What do we know about Accreditation?

Trend Briefs:

Data source:

Sample included 130 programs receiving accreditation site visits between September 2009 and July 2010.Data captured on all 417 NAEYC criteriaComparisons between accredited and not accredited programs’ performance on all criteria

Slide28

What do we know about Accreditation?

Trend

Briefs

- Selected findings:Relationships (NAEYC Standard 1)Differences are noted in terms of programs’ means of dealing with challenging behavior, but even more so in the degree to which programs provide a “predictable, consistent, and harmonious” classroom. Teaching (NAEYC Standard 3)

Programs differ primarily among criteria that assess the use of scaffolding strategies in the classroom.

Assessment of Child Progress (NAEYC Standard 4)

Programs accredited by NAEYC demonstrate a planned, intentional use of child assessment and communication of assessment results: using assessments to improve instruction and program design, and to effectively communicate assessment results to other teachers and families.

Slide29

What do we know about Accreditation?

Trend

Briefs

- Selected findings:Supporting Cultural Competence (Cross-Standard)Many of the same criteria that prove the most challenging overall also

differentiate between programs that became accredited

and

those that did

not.

Differences in how programs can connect

with diverse families and

engage

them in the child’s program

Differences in programs’ ability

to understand, and respect, diversity in family values, especially when they may differ from those of the teacher.

Differences in hiring diverse staff and ensuring staff receive training that includes

working with diverse

families.

Differences in providing

children with varied and deep experiences to support their own cultural

competence.

Slide30

What do we know about Accreditation?

Some data to suggest valid indicator of quality

Need more validation studies and data

Analysis of Accreditation data show differentiation between programs accredited and those not accredited, even when all attempt to reach same criteriaFuture analyses can identify performance clusters, possible examine program performance pre-self-study to site visit to examine potential for quality improvement processes

Slide31

Accreditation and QRIS Congruence

State recognition of accreditation within QRIS ratings

Some states use NAEYC Standards for specific areas

Alignment of program standardsStreamlining for programs that meet accreditation standardsAccreditation Facilitation (Program Quality Improvement) Project models

Slide32

Accreditation and QRIS Congruence

State QRIS systems include accreditation in various ways:

Not recognized

Awarding additional points towards rating (overall or in specific areas, varying by system)Enter at top (or near-top) ratingSome combine accreditation with ERS visitsSome differentiate accrediting bodies

Slide33

Accreditation and QRIS Congruence

In what ways can states benefit from NAEYC experience through accreditation in designing and implementing QRIS systems for program quality recognition and improvement, and in communicating with families?

Slide34

Data Can Facilitate Cross-State

Sharing

and

ComparisonWhat data elements does your system need?Are there common definitions of data elements? National data efforts to be aware of…

Slide35

Common Education Data Standards

Early Learning is one domain in the overall P-20 data model

https

://ceds.ed.gov/Default.aspx

Slide36

Quality Initiatives Research and Evaluation Consortium (INQUIRE)

INQUIRE

supports high quality, policy-relevant research and evaluation on quality rating and

improvement systems (QRIS) and other quality initiatives by providing a learning community and resources to support researchers. The INQUIRE Consortium also provides input and information to

state

administrators and other policymakers and practitioners on evaluation strategies, new

research

, interpretation of research results, and implications of new research for practice.

Child Trends

helps to facilitate INQUIRE activities

Slide37

INQUIRE and Data

QRIS/QI Data Elements

workgroup of INQUIRE

worked with US Department of Education group focusing on Common Education Data Standards (CEDS) to create a recommended list of data elements, which is out now

for public comment.

developing a list of recommended data elements for QRIS and

Quality Improvement

purposes

will be developing a set of data elements, especially for child care state administrators and CCDF reporting

Slide38

Questions, Reflections, Comments?

Slide39

Homework for January 17, 2013

Effective Cross-Sector QRIS: Challenges and Opportunities

Cross-sector

QRIS means one that aims for participation by most group early care and education providers, regardless of funding stream or auspice. At a minimum, this includes child care centers and family child care homes, Pre-K and Head Start, i.e., all publically supported and licensed settings, but not informal caregivers.  

A

survey monkey link

will be emailed to you for use in completing the homework questions. –

Due January 4

th

(for

January 17,

2013 webinar)

Slide40

Homework Questions for 1.17.12 Session

Do you have a plan to include a cross sector approach in the QRIS? Why did you make that decision? Identify the phase in plan for different sectors (i.e. Are you beginning with ‘all in’ or phasing in over a few years)?

What challenges have you experienced in your efforts to develop and/or implement a cross-sector QRIS

?What successes have you had with cross-sector QRIS?How do license-exempt centers (e.g.

preK

programs located in public or private schools) participate in your QRIS? Have you created an 'equivalent' standard for licensing

?

What have you learned about strategies for effectively engaging the support systems of other sectors (e.g. the Head Start T/TA system or early intervention training) in QRIS supports

?

Have you tried to engage monitoring or accountability systems from other sectors (such as collaborating with Head Start or

PreK

monitoring

)?

Have you worked with systems like early intervention, child welfare, and others to ensure that they understand QRIS and prioritize child placements in higher-quality settings

?

Slide41

Thank You

NCCCQI

does not endorse any non-Federal organization, publication, or resource.

Follow-up Contacts:

OCCQualityCenter@icfi.com

dmathias@buildinitiative.org

tcamillo@Brightstars.org

a

nne.walsh.mitchell@gmail.com

l

ouise.stoney@gmail.com

www.qrisnetwork.org

dawn.a.woods@state.or.us

ksnow@naeyc.org

National Center on Child Care Quality Improvement

Slide42

Presented with permission from Child Trends (2012)

Slide43

Presented with permission from Child Trends (2012)

Slide44

Presented with permission from Child Trends (2012)

Slide45

Presented with permission from Child Trends (2012)

Slide46

Presented with permission from Child Trends (2012)

Slide47

Presented with permission from Child Trends (2012)

Slide48

Presented with permission from Child Trends (2012)