Times Higher Education Teaching Excellence Framework and Relative Teaching Performance TEF recap Data available now What might relative teaching performance look like What next TEF Technical consultation for Year Two ID: 810673
Download The PPT/PDF document "Nicki Horseman Lead HE analyst" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Nicki
Horseman
Lead HE analyst
Times Higher Education
Slide2Teaching Excellence Framework and Relative Teaching Performance
Slide3TEF recap
Data available now
What might relative teaching performance look like?
What next?
Slide4TEF - Technical consultation for Year Two
‘build on existing high standards’
‘allow for diverse forms of excellence to be identified and recognised’
‘clear judgments about excellence for students, employers and other stakeholders’
Slide5Assessment based on:
Context/metrics
Data will be averaged over three years; also presented by individual year in total
TEF Year Two will use existing NSS / DLHE; will take account of non-respondents in Year Three
Additional evidence
- 15 pages submitted by institution
Slide6Institutional Metric presentation to the assessment panel
Slide7So what data is available now (with benchmarks)
Aspect
TEF proposal
THE
Teaching Quality
NSS Q1-4 Teaching on course;
NSS Q5-9
Assessment and feedback
NSS Q22 Overall
Satisfaction
Learning
Environment
NSS Q10-12 Academic support;
Non-continuation (T3)
Projected Outcomes (T5)
Student Outcomes and Learning Gain
Employment
(DLHE); potential highly-skilled jobs metric
Graduate Destinations (DLHE)
Slide8Teaching Quality - NSS - Q1
(Staff are good at explaining things)
v.s
. Q22
Slide9Teaching Quality - NSS - Q1
(Staff are good at explaining things)
v.s
. Q22
Q22 Overall satisfaction
Q1
Source: HEFCE NSS 2015 Summary Data Taught
Slide10Teaching Quality - NSS - Q1
(Staff are good at explaining things)
v.s
. Q22
Q22 Overall satisfaction
Q1
Guild HE
Source: HEFCE NSS 2015 Summary Data Taught
Slide11Teaching Quality - NSS - Q1
(Staff are good at explaining things)
v.s
. Q22
Q22 Overall satisfaction
Q1
MillionPlus
Guild HE
Source: HEFCE NSS 2015 Summary Data Taught
Slide12Teaching Quality - NSS - Q1
(Staff are good at explaining things)
v.s
. Q22
Q22 Overall satisfaction
Q1
University Alliance
MillionPlus
Guild HE
Source: HEFCE NSS 2015 Summary Data Taught
Slide13Q22 Overall satisfaction
Q1
Russell Group
University Alliance
MillionPlus
Guild HE
Teaching Quality - NSS - Q1
(Staff are good at explaining things)
v.s
. Q22
Source: HEFCE NSS 2015 Summary Data Taught
Slide14Q22 Overall satisfaction
Q1
Russell Group
University Alliance
MillionPlus
Guild HE
Loughborough
Swansea
Teaching Quality - NSS - Q1
(Staff are good at explaining things)
v.s
. Q22
Source: HEFCE NSS 2015 Summary Data Taught
Slide15Learning Environment - T3
(continuation after first year)
v.s
. T5 (projected outcomes)T3 (continuation after first year)
T5 (projected outcomes)
Source: HESA PIs 2013/14
Slide16Learning Environment - T3
(continuation after first year)
v.s
. T5 (projected outcomes)T5 (projected outcomes)
Russell Group
Source: HESA PIs 2013/14
T3 (continuation after first year)
Slide17Learning Environment - T3
(continuation after first year)
v.s
. T5
(projected outcomes)
T3 (continuation after first year)
T5 (projected outcomes)
Russell Group
University Alliance
Source: HESA PIs 2013/14
Slide18Learning Environment - T3
(continuation after first year)
v.s
. T5
(projected outcomes)
T3 (continuation after first year)
T5 (projected outcomes)
Russell Group
University Alliance
Guild HE
Source: HESA PIs 2013/14
Slide19Learning Environment - T3
(continuation after first year)
v.s
. T5
(projected outcomes)
T3 (continuation after first year)
T5 (projected outcomes)
Russell Group
University Alliance
Guild HE
MillionPlus
Source: HESA PIs 2013/14
Slide20Learning Environment - T3
(continuation after first year)
v.s
. T5
(projected outcomes)
T3 (continuation after first year)
T5 (projected outcomes)
Source: HESA PIs 2013/14
Slide21Learning Environment - T3
(continuation after first year)
v.s
. T5
(projected outcomes)
T3 (continuation after first year)
T5 (projected outcomes)
Loughborough
Swansea
Source: HESA PIs 2013/14
Slide22Student Outcomes - Employment metrics
Graduate destinations
HESA PI Employment
Guild HE
Source: HESA PIs 2013/14; DLHE
Slide23Student Outcomes - Employment metrics
Graduate destinations
HESA PI Employment
MillionPlus
Guild HE
Source: HESA PIs 2013/14; DLHE
Slide24Student Outcomes - Employment metrics
Graduate destinations
HESA PI Employment
University Alliance
MillionPlus
Guild HE
Source: HESA PIs 2013/14; DLHE
Slide25Student Outcomes - Employment metrics
Graduate destinations
HESA PI Employment
Russell Group
University Alliance
MillionPlus
Guild HE
Source: HESA PIs 2013/14; DLHE
Slide26Student Outcomes - Employment metrics
Graduate destinations
HESA PI Employment
Source: HESA PIs 2013/14; DLHE
Slide27Student Outcomes - Employment metrics
Graduate destinations
HESA PI Employment
46%
12%
Source: HESA PIs 2013/14; DLHE
Slide28Student Outcomes - Employment metrics
Graduate destinations
HESA PI Employment
46%
12%
‘take account of … students’ characteristics and the subject they study’
Source: HESA PIs 2013/14; DLHE
Slide29Benchmarks and significance
Benchmarks – given the characteristics of the students recruited to an institution, what is the expected performance based on the performance of the rest of the sector
No widening participation or geographic factors
Benchmarks involving entry standards – potential issues
Factor
NSS
Non-continuation
Employment
Subject
Entry qualifications
Age on
entry
Ethnicity
Gender
Disability
Slide30Student outcomes relative to benchmark
Graduate destinations distance from benchmark
HESA PI Employment E1 distance from benchmark
Source: HESA PIs 2013/14; DLHE
Slide31Student outcomes relative to benchmark
Graduate destinations distance from benchmark
HESA PI Employment E1 distance from benchmark
Swansea
Loughborough
Source: HESA PIs 2013/14; DLHE
Slide32Who performs well against benchmark?
– Learning Environment and Teaching Quality
Slide33Who performs well against benchmark?
– Learning Environment and Teaching Quality
Slide34Student Outcomes and Teaching Quality
Slide35Student Outcomes and Teaching Quality
Slide36Significance tests
(3 standard deviations
)
Institutions which are significantly above benchmark assessment in:
Three out of three aspects:Loughborough, Swansea
Two out of three aspects (and neutral on 3
rd
):
Aston, Coventry, Heriot-Watt, Huddersfield, Lancaster, Nottingham Trent
Slide37An overall picture
How to give an overall picture of teaching excellence as framed by the TEF?
Using three indicators as before:
Q22 as a proxy for
Teaching Quality T5 projected outcomes as a proxy for
Learning Environment Graduate destination as a proxy for Student Outcomes and Learning Gain
Working at an overall level without breaking down into detailed cohorts - one year’s data
Relative teaching performance measures – performance relative to benchmark – to ‘take account of differences in student characteristics and mixture of subjects taught’.
Z score each metric to form a common scale, combine with equal weights and scale from 100
Slide38Absolute Teaching Performance
Key:
Russell Group
,
University Alliance
,
MillionPlus
,
Guild HE
Slide39Relative Teaching Performance
Key:
Russell Group
,
University Alliance
,
MillionPlus
,
Guild HE
Slide40Relative Teaching Performance (using E1a)
Key:
Russell Group
,
University Alliance
,
MillionPlus
,
Guild HE
Slide41Relative Teaching Performance
Key:
Russell Group
,
University Alliance
,
MillionPlus
,
Guild HE
Slide42Relative Teaching Performance
Outstanding
Excellent
Meets expectations
Swansea
Loughborough
Slide43Market share of non-UK Undergraduates
(First Degree; order is relative teaching performance)
Outstanding
Excellent
Meets expectations
Source: HESA (THE WUR data)
Slide44Market share of non-UK Masters
(order is relative teaching performance)
Outstanding
Excellent
Meets expectations
Source: HESA (THE WUR data)
Slide45Non-UK Undergraduate market share by region
(order is relative teaching performance)
Key:
London
,
Midlands
,
N England
,
N Ireland
,
Scotland
,
S England
,
Wales
Outstanding
Excellent
Meets expectations
Source: HESA (THE WUR data)
Slide46Non-UK Masters market share by region
(order is relative teaching performance)
Outstanding
Excellent
Meets expectations
Key:
London
,
Midlands
,
N England
,
N Ireland
,
Scotland
,
S England
,
Wales
Source: HESA (THE WUR data)
Slide47REF 2014 with intensity
TEF
Russell Group
Teaching Excellence and REF
Slide48REF 2014 with intensity
TEF
Russell Group
University Alliance
Teaching Excellence and REF
Slide49REF 2014 with intensity
TEF
Russell Group
University Alliance
MillionPlus
Teaching Excellence and REF
Slide50REF 2014 with intensity
TEF
Russell Group
University Alliance
MillionPlus
Guild HE
Teaching Excellence and REF
Slide51REF 2014 with intensity
TEF
Russell Group
University Alliance
MillionPlus
Guild HE
Teaching Excellence and REF
Slide52REF 2014 with intensity
TEF
Loughborough
Swansea
Kent
Bath
Surrey
Teaching Excellence and REF
Slide53Summary
Teaching Excellence Framework proposes an alternative perspective on quality
Slide54Summary
Teaching Excellence Framework proposes an alternative perspective on quality
Highlights diverse institutions
Slide55Summary
Teaching Excellence Framework proposes an alternative perspective on quality
Highlights diverse institutions
‘Non-aligned’
Interpretation and communication
Midlands
Northern England
Clarity and transparency of judgements