A praxis based framework for evaluation - PowerPoint Presentation

Slide1

A praxis based framework for evaluation

Theory, Practice

& Impact

in Widening

Participation

Access to Higher Education and

Student Success Summit 2017

Annette Hayton, University of Bath

Marian Mackintosh

, University of

Bath

Emily Warwick, Oxford Brookes University

Increases in rates of progression

Trends in young participation in higher education: core results for

England. HEFCE 2010/3

Figure

10:

Trends

in young participation rate for areas classified by HE participation rates (POLAR3 classification, adjusted)

Monitoring

Participant

characteristics are monitored to assess and demonstrate success in attracting students who fulfil the targeting criteria.

Process Evaluation

Did they have a good time?

Were they safe?

Can organisation be improved?

Did they like the lunch?

Was the session engaging?

How was it for the staff and ambassadors?

Tracking: HEAT

Bath admitted 600 students who participated in outreach with other HEAT member universities

Twice as many

participants

from low progression areas

went

on to university compared with the average for LPN students in the counties surrounding the university.

Participants in

Bath

outreach

activities were much more likely to go to a high tariff university than disadvantaged students

nationally

Participants in Bath outreach activities

were

awarded an average of two grades higher in their GCSEs

than

students

in the same schools who had the same attainment at KS2 

who had not taken part.

Impact Evaluation

Evaluation of an HE outreach activity means assessing the impact of the activity on its participants, measured against its intended objectives.

Footnote iv

The

NERUPI framework provides a very rigorous theoretically-informed methodology for linking WP aims and objectives to impact evidence

Overcoming feelings of fearExperiencing university life and a sense of belongingConfidence in their capacity to achieveChallenging perceptions and overcoming doubts

Living away from home and meeting new peopleDeveloping the capacity for academic and learning challenges

Enhancing and contextualising subject knowledge

Developing the capacity to make informed choices

WP activity: a transformative experience?

Experiential impact

Transformational effect on learners

Pedagogy

Active learning and critical pedagogies

Content

Accessible yet challenging content

Designed to maximise the impact of Widening Participation interventions providing:a robust theoretical and evidence-based rationale for the types of intervention that are designed and deliveredclear aims and learning outcomes for interventions, which enable more strategic and reflexive design and delivery

an integrated evaluation process across multiple interventions to improve data quality, effectiveness and impact

The NERUPI Framework

Bourdieu – Field, Capitals and HabitusCultural model of Widening Participation that locates interventions within a wider field of engagement

Capitals - expressions of differences within an unequal social system – enable/restrict engagement with educationIntellectual capital (Scientific capital) - subject expertise

Academic capital - understanding of rules/customs within academySocial capital – social connections

Habitus – situated cultural identity/set of dispositions

Bourdieu’s Theories

Field of

HE Progression

The NERUPI Framework

SOCIAL AND ACADEMIC CAPITAL

HABITUS

SKILLS

CAPITAL

INTELLECTUAL

& SUBJECT CAPITAL

PROGRESSION CURRICULUM

STUDENT

IDENTITIES

SKILLS CURRICULUM

KNOWLEDGE CURRICULUM

KNOW

CHOOSE

BECOME

PRACTISE

UNDERSTAND

Develop students' knowledge and awareness of the benefits of higher education

Develop students' capacity to navigate Higher Education sector and make informed choices

Develop students' confidence and resilience to negotiate the challenges of university life

Develop students' study skills and capacity for academic attainment

Develop students' understanding by contextualising subject knowledge

Aims and Objectives Level 3

Framework enables us to share aims and develop activities withParticipantsParentsSchool teachersWP outreach workers

University academic and professional support staff

Reflexivity

Residential Summer School Evaluation

Increased

u

nderstanding,

subject knowledge and skills are evaluated through assessment of group projects by academic staff

Engagement

with the summer school and university life is evaluated by surveys (including open-ended questions) interviews, focus groups, ambassador feedback and observations

Knowledge of HE and how to choose a course is evaluated through surveys, discussion and also ‘quizzes’

i.e

testing

Before coming to this summer school I

was very apprehensive about going to university as I was not sure how different life would be there compared to the life I live currently. This summer

school has made me more confident that I can fit into university life and has given me advice about applying to university that I would not have received otherwise.

Residential Summer School Evaluation

‘They

all grasped the

concept and then

went over and above that - showing knowledge I would associate with

undergrads’

Academic staff completed a post-event questionnaire

Project was

completely

different from

any

school projects and so this insight into engineering was really useful and furthered my interest. I also learnt new formulas and concepts that involved some of the chemistry from school.

.

How we approached itEmbedding into new activities vs pre existing

ones

Making it part of planning and evaluation processes

Getting buy-in from the team & wider

university

Embedding NERUPI

Brookes Engage

Increase students’ awareness of HE opportunities, and sense of belonging within the University.To improve participants’ confidence in academic skills and knowledge of the university application process: to make them successful students Provide students with an understanding of industries they are interested in, with work shadowing, placements or contacts in those industries.

Create a peer group of like-minded studentsShowcase the links between Brookes courses and employability

Provide students with access to high-quality enrichment activities

Links to all aspects of Level 3 of the Framework

Brookes Engage: Aims

Brookes Engage:

Evaluation

Methods: Pre and Post Evaluation Diary Study Control Group

Quote from Diary Study summarising Brookes Engage:

“

A lot of the workshops from Brookes Engage have stuck with me, including note taking and University structure, equally ideas including staying organised have really helped me remain under control of my assignments and work, and I still find myself using the mindfulness techniques to calm down after stressful times which has helped on more than one occasion.

All in all really thriving in the University environment and cant wait to see what the future holds.”

Objective from framework:

Develop students’ confidence and resilience to negotiate

the challenges of university life and graduate progression

Extend the levels to Student Success and ProgressionApply the Framework in different contextsUse with a wider range of methodologies

Explore the how the ideas underpinning the framework can be applied to cross-cutting topics e.g

. Ambassadors

Invite more universities to join NERUPIEvent and website launch on the 7 February

Hayton, A and Bengry-Howell, A (2016) Theory

, evaluation, and practice in widening participation: A framework approach to assessing impact London

Review of Education, Volume 14, Number 3, November 2016, pp. 41-53(13)

Next Steps