Current Debates about Assessing Student Learning - PowerPoint Presentation

Slide1

Current Debates about Assessing Student Learning

Natasha Jankowski, PhD.Director, National Institute for Learning Outcomes AssessmentResearch Professor, Education Policy, Organization and Leadership, UIUC

@NILOA_web@njankow

Slide2

NILOA

NILOA’s mission is to discover and disseminate effective use of assessment data to strengthen undergraduate education and support institutions in their assessment efforts.

● Surveys ● Web Scans ● Case Studies ● Focus Groups ● Occasional Papers ● Website ● Resources ● Newsletter ● Presentations ● Transparency Framework ● Featured Websites ● Accreditation Resources ● Assessment Event Calendar ● Assessment News ● Measuring Quality Inventory ● Policy Analysis ● Environmental Scan ● Degree Qualifications Profile ● Tuning www.learningoutcomesassessment.org

Slide3

www.learningoutcomesassessment.org

Slide4

Turn to a neighbor and share

What are you most frustrated about when it comes to assessment?Now…What was your favorite teaching or learning moment?Why are these mutually exclusive? Assessment and teaching and learning?

Slide5

Slide6

Slide7

Slide8

Debates on the Purpose Why do we do assessment? What is the value and purpose of engaging in assessing student learning?

Slide9

ValueInstitutions of higher education are increasingly asked to show the value of attending, i.e. impact in relation to cost; employment – what is the value of a degree and what does it represent in terms of learning?

Public and policy makers want assurance of the quality of higher educationRegional and specialized accreditors are asking institutions to show evidence of student learning and instances of useWe have questions about our own practice to inform internal improvements

Slide10

Purpose Statement for BU

Slide11

Slide12

Accreditation Under Fire

Accreditors are increasingly being questioned on their value and role, and they have their own regulations to which they are held accountable in order to be recognized as an accrediting agency

Slide13

Principles of Local PracticeFocus on improvement and compliance will take care of itself.

Slide14

Slide15

Slide16

2

017

NILOA Prov

o

s

t SurveySample: All regionally accr

edited, under

g

r

adu

a

t

e

deg

r

e

e

-g

r

a

n

t

i

ng i

n

s

t

i

tu

t

ions (n=2,7

81

)

An

n

ounced

via in

s

t

i

tu

t

ional

mem

b

e

r

sh

i

p

o

r

g

ani

z

a

t

i

ons,

w

e

b

s

i

t

e, n

e

w

s

l

e

t

t

e

r

,

ma

i

l

i

ng

Onli

n

e

and

paper

29

%

r

espon

s

e

r

a

t

e (n=

811

)

Slide17

Institution-level

Concrete, clear proficiencies students are to achieve -- reference points for student performance common to all undergraduates across all majors. 82%

of campuses have SLO statements

Slide18

Learning Outcomes are Increasingly Aligned

At 50% of campuses: all programs have PLOs and align those PLOs with ILOs

Slide19

Current Activity

77% of institutions are currently involved in mapping curriculum62% facilitating faculty work on the design of aligned assignments

Slide20

Findings

1. Institutions are trending towards greater use of authentic measures of student learning, which is consistent with what provosts indicate are most valuable for improving student outcomes. 2. Majority of changes made and uses of evidence of student learning occur at the program- and course-level.

Slide21

Slide22

VALUE Institute

Slide23

But what about programs?

Program-level survey:

Randomly selected 3-5 departments from all regionally accredited degree-granting institutions and sent them a survey with a 30% response rate (n= 982)

Slide24

Program survey findings

At the department/program level, the primary driver in assessment activity is faculty’s interest in improving their programs—followed by accreditation, both institutional and specialized.A diverse array of assessment methods are employed at the program level across institutions, the most frequently used being capstone experiences, rubrics, final projects, and performance assessments

The primary use of assessment results at the program level is for program review—followed by instructional improvement and institutional accreditation.

Slide25

Program Survey Findings Cont.

Although differences exist among disciplines in terms of the amount of assessment activity, the most notable variations are in the types of assessment methods they use.

Slide26

Model Differences

Slide27

Institutional or Program Improvement Model

Slide28

Assess, Intervene, Reassess

Slide29

Student Learning Improvement

Slide30

Slide31

Assessment as a Process...

Is trying to get us to think intentionally about our learning design

Slide32

Assessment Cube of Misunderstandings

Definitions

Purposes/Value

Uses/Questions

Levels/Focus

Slide33

Three Schools of ThoughtMeasurementCompliance (Reporting)

Teaching and Learning (Improvement)

Slide34

MeasurementBuilt upon scientific principles or empirical research, objective, rational, validity, and reliability The Multi-State Collaborative: A Preliminary Examination of Convergent Validation Evidence

~Mark Nicholas, John Hathcoat, & Brittany BrownTesting and standardizationMust be measureable Argue narrowing of curriculum Goal driven Focused on processInterventions

Pre/postComparisons

Slide35

VALUE report

Slide36

VALUE report

Slide37

ComplianceDocumenting institutional quality assurance through reporting frameworksIs assessment destroying the liberal arts? ~Karin Brown

BureaucracticLaboriousTime consumingSeparated from teaching and learningAdd onAccountability and quality assurance Reporting and archiveLots of data collection, minimal use

Slide38

ACCREDITATION/ PROGRAM REVIEW

Slide39

Slide40

But where are the students…?

Slide41

Teaching and LearningFocus on pedagogy, understanding of student experience, informing program improvement, embedded in curricular design and feedback, builds student agency

Does continuous assessment in higher education support student learning? ~Rosario HernandezDriven by faculty questions regarding their praxis – is what I am doing working for my students?Improvement orientedFocus on individual studentsStudents as active participants – not something done to themFormativeFeedback

Collaborative Assessment for learningAdaptive and embedded

Slide42

TransparencyAwareness of Learning Outcome Statements

Slide43

IMPROVEMENT

Slide44

Involving students

Slide45

Involving students

Assessment is not something we do to students it is something we do with students.

Slide46

Additional Points of Contention

Slide47

Academic Freedom

Slide48

Does It Improve Learning?

Slide49

Cost of Assessment

Slide50

https://blogs.illinois.edu/view/915/639769

Slide51

https://blogs.illinois.edu/view/915/623545

Slide52

QuestionsEmail: niloa@education.illinois.edu njankow2@Illinois.edu

http://www.learningoutcomesassessment.org www.assignmentlibrary.org www.degreeprofile.org