Update on Family Outcomes Work - PowerPoint Presentation

Update on Family Outcomes Work Don Bailey and Melissa Raspa ECO Advisory Board Meeting March 7, 2012

Goals for Today Provide update on family outcomes APR data Introduce ECO family outcomes and experiences measurement system self-assessment toolDiscuss future of family outcomes Early Childhood Outcomes Center

Part C Measurement Approaches ToolsNCSEAM survey (n = 25)ECO Family Outcomes Survey (n = 24)FOS-R (n = 3)State-developed survey (n = 6) Methodology Census (n = 44) Sampling (n = 11) 3 Early Childhood Outcomes Center

Part C Population Surveyed Timing All families Families > 6 months Total Point in time 251237At exit369At IFSP meeting325IFSP or exit213Total332154 4 Early Childhood Outcomes Center

Part C Response Rates Overall response rate = 36% Tool ECO: 41% NCSEAM: 31% State: 31%Methodology:Census: 37%Sample: 34%

Part C Response Rates DistributionIn person: 46%Multiple methods: 37%Mail: 27%Return In-person: 75% Multiple methods: 36% Mail: 27% 6 Early Childhood Outcomes Center

Part C Representativeness Criteria States (N) States (%) Race/ethnicity 41 75 Geography (district, county)1731Gender1425Child’s age (at referral, at survey)1324Disability/eligibility category611Length of time in services611Income357 Early Childhood Outcomes Center

Part C APR Trend Data Early Childhood Outcomes Center

Part C Trend Data – Indicator 4A 9 Early Childhood Outcomes Center

Part C Trend Data – Indicator 4B 10 Early Childhood Outcomes Center

Part C Trend Data – Indicator 4C 11 Early Childhood Outcomes Center

Part C APR by Survey Tool 12 Early Childhood Outcomes Center

Part B Measurement Approaches ToolsNCSEAM (n=43)Adapted NCSEAM or ECO (n=8) State (n=8) Methodology Census (n=21) Sampling (n=35) 13 Early Childhood Outcomes Center

Part B Response Rates Overall response rate = 24%DistributionMultiple methods: 30%Mail: 17% Return In-person: 75% Multiple methods: 36%Mail: 27 14 Early Childhood Outcomes Center

Part B APR Trend Data 15Early Childhood Outcomes Center

Next Steps APR analyses for FFY 2010-2011 Examine data qualityAPR by response ratesDevelop template for states to share family outcomes dataProvide TA to states on as-needed basis 16 Early Childhood Outcomes Center

Family outcomes and experiences measurement system Self assessment tool Early Childhood Outcomes Center 17

Purpose Provides a common language and organizing structure for discussing family outcomes and experiences Assist states in developing a measurement system that captures various aspects of family outcomes and experiences, such asbenefits that families receive from program participationfamily satisfaction with services perceived helpfulness of the services family involvement with the service delivery system 18 Early Childhood Outcomes Center

Framework and Self-Assessment Framework Set of components and quality indicatorsProvides the structure for the self-assessment Self-assessment Scale that provides criteria for levels of implementation within each quality indicator Rating assigned based on level of implementation within each indicator 19 Early Childhood Outcomes Center

Partner States Connecticut IllinoisMinnesotaTexas20 Early Childhood Outcomes Center

Framework Components 7 key areas of a measurement systemQuality indicators16 specific items that further describe each of the componentsElementsVarious examples of what constitute quality at the indicator level 21 Early Childhood Outcomes Center

22 Purpose Data Collection and TransmissionAnalysis Reporting Using Data Evaluation Cross-System Coordination Components Quality Indicators PurposeState has articulated purpose(s) of COMS. Data Collection and Transmission 2. Data collection procedures are carried out efficiently and effectively. 3. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment. 4. State's method for entering, transmitting, and storing data is effective and efficient. Analysis 5. State identifies accountability and program improvement questions related to child outcomes. 6. Local programs identify accountability and program improvement questions related to child outcomes. 7. State agency analyzes data in a timely manner. 8. Local programs analyze data in a timely manner. 9. State agency ensures completeness and accuracy of data. Reporting 10. State agency interprets, reports, and communicates information related to child outcomes. 11. Local programs interpret, report, and communicate information related to child outcomes. Using Data 12. State agency makes regular use of information on child outcomes to improve programs. 13. Local programs makes regular use of information on child outcomes to improve programs. Evaluation 14. State evaluates its COMS regularly. Cross-system Coordination 15. Part C and 619 coordinate child outcomes measurement. 16. Child outcomes measurement is integrated across early childhood (EC) programs statewide. 17. Child outcomes measurement is aligned with state’s early learning guidelines/standards. 18. State has a longitudinal data system to link child outcomes data from EC program participation to K–12 data. Elements a. State has… b. State has… c. State agency.. d. Representative.. e. State agency… f. State …… g. State provides… h. State has..

Self-Assessment Provides guidance to states on what constitutes a high quality family experiences and outcomes measurement system.Assists states in setting priorities for improving their measurement system Provides information to assist states in advocating for resources for systems development 23

How Does it Work? Each quality indicator has multiple elements Evidence for the extent of implementation for each element is provided Each element is rated as NY = Not Yet or Don’t know IP = In Process IF = Fully Implemented NA = Not applicable (only where applicable) The quality indicator is given a rating based on the ratings of the elements Early Childhood Outcomes Center24

Self-assessment Scoring 25 Implementation of Elements Quality Indicator Score All elements are fully implemented 7 Nearly all elements are fully implemented and the rest are in process 6 Most of the elements are fully implemented and the rest are in process. 5At least one element is fully implemented and the rest are in process4All of the elements are in process 3 Some of the elements are in process 2 None of the elements are yet in process 1

Challenges with Family Framework and Self-assessment Going beyond the APR: Family outcomes and experiencesBalancing expectations: Reality vs. ideal Scoring systemDifficulty in reaching 7 Not applicable items Early Childhood Outcomes Center 26

Next Steps Finalize framework and self-assessment tool and post on ECO web site National conference call to introduce to broader audienceProvide TA to states on as-needed basisEarly Childhood Outcomes Center 27

What is the future of family outcomes?

Context Family-centered (FC) practices are endorsed by many professional organizations as central to high-quality practice Institute of MedicineAmerican Academy of PediatricsAmerican Hospital AssociationAmerican Speech-Language Hearing Association Division for Early Childhood

We know…. Meta-analyses of research provides strong evidence that FC practices lead to many benefits for families and children

Dunst , Trivette, & Hamby (2007) Meta-analysis of 47 studies of family-centered practices in a variety of settingsHigher use of FC practices was associated with:Satisfaction with services* Self-efficacy beliefs* Parenting behaviors Personal and/or family well-being Social support*Child behavior and functioning*Strongest associations

Pitrowski , Talavera, & Mayer (2009) Meta-analysis of 13 articles evaluating Health Steps, a pediatric care model based on strength-based primary prevention the assumption that child well-being and family-centered care are necessarily linked supporting the role of parentspositive, caring, and enduring relations with parents.Parents in HS were More satisfied with services Exhibited more desirable parenting practices More satisfied in their role as parentsInteracted more sensitively and appropriately with child

Broom & Enriquez (2009) Reviewed 9 randomized trials of family-centered interventions in treating children with Type 1 diabetes Family-centered practices were associated withImproved health outcomes for childrenReduced diabetes-related conflict in familiesImproved family relations

Two studies using the National Survey of Children with Special Health Care Needs When families perceived a strong partnership between the child’s family and the child’s service providerchildren’s health outcomes were improved children received more services families were more satisfied with services

Also Our NEILS analyses and our ECO papers show that family-centered practices are related to family outcomes

Epley , Summers & Turnbull (2011)

But research also shows that FC practices are not widely implemented Medical Expenditure Panel Survey and National Survey of CSHCN children found that only about 2/3 of families reported getting family-centered care A hospital survey found that only about half used FC rounds in pediatric in-patient settingsMinority families generally report fewer FC services and less satisfaction

Dunst (2012) argues that EI has not evolved in tandem with what we know

So, if….. Professional organizations widely endorse FC practices, and… Research shows that FC practices make a difference…What should we do about the future of FC practices and family outcomes in early intervention and preschool programs?

Use of Family-Centered Practices We don’t know at a state or national level the extent to which programs are actually using family-centered principles and practices We don’t know what factors predict or are associated with variability in implementation of F-C practicesNEEDED: A comprehensive description of the status of F-C practices in EI and preschool programs, and a clear understanding of why some programs are more family-centered than others

Family-centered principles, when integrated across all program activities, ought to lead to family outcomes. Referral and intake Determining eligibility Child assessment Family assessment Team meetings and decision-making Service provision and coordination Parenting practices Community supportUnderstand child’s abilities and special needsKnow rights and advocate effectively Help child develop and learn Have support systems Access the community Family-Centered Principles Program Activities Family Outcomes Shared philosophy Families as partners Focus on strengths Family choice of goals and services Collaboration and coordination of service Effective communication Flexibility Community-based

What is it about F-C practices that leads to better outcomes? With the exception of the NEILS study and other smaller studies, we don’t know the full range of outcomes that F-C practices can affect The pediatric literature has far more studies of this than EINEEDED: Sophisticated large-scale studies modeling how and which F-C practices influence a variety of outcomes

The desirability & applicability of F-C practices & family outcomes in preschool Nothing is known about F-C practices in preschool programs for children w/disabilities The ECO family outcomes have never been accepted or seriously discussed for preschoolWe have some evidence that family outcomes are less likely to be achieved with older children NEEDED : Stakeholder opinions and consensus regarding the desirability of FC practices & appropriate family outcomes in preschool

F-C practices and child outcomes Is helping families a good outcome in and of itself, OR must we show that this results in benefits for children? We have danced around this topic for 20 years, with no resolutionNEEDED: Either we need consensus on the argument that family benefit is a worthwhile outcome OR we need to show that F-C practices and family outcomes are important to maximize child benefit

Potential Year 5 Family Activities Help states use framework to improve systems for family outcomes assessment Targeted TA to states with low family helpfulness ratingsCollect, compile and analyze data from states that have FOS data that can be linked with child outcomes dataExplore with one or two states or programs use of FOS in preschool programs