Flexible and Online Learning Modes and Student Progress in First Year – FINAL PRESENTATION - PowerPoint Presentation

Slide1

Flexible and Online Learning Modes and Student Progress in First Year – FINAL PRESENTATION

for the

Common Unit

Committee and

Office of Learning and Teaching

Research Fellow Menzies Bill

Tyler

Professor Charles

Webb

Dr.

Nicola

Rolls

Sharon Bridgeman

Dr.

Malcolm

FlackSlide2

Project Aim & BackgroundTo

investigate the impact of flexible modes of learning and attendance in the first year of university

:

By exploring student responses to

fleximode

pedagogies and the effect of these on their academic

progress

Using data from student surveys, Analytics records of usage, collaborate use records and student grades

Data gathered

and analysed

over three years 2013- 2015 for

common units

and

16

course-specific, core First Year units with large

enrolments:

Law, Business, Nursing, Science, Education

Data set:

L

earnline

14881, Collaborate 4983, Surveys 17% of populationSlide3

Previous Findings for 2013-2014 data

A

ssociation

of student success with

use

of both

Learnline

and collaborate

M

ultivariate analysis

and

counterfactual

techniques

show these associations are

statistically sound and significant

Students drawn

to

tools that are

accessible

,

keep them connected with other students and staff

and are

relevant to

assessment

including

collaborate

Learner

motivation levels

per se

not

a significant factor influencing student engagement with

Learnline

Overall qualitative student commentary on the

Learnline

experience

positive but some possible

priority areas for improvement

I

ndications of student

sub-cohort differences between levels of engagement and satisfaction

Outcomes

of an audit of units

(based on

BlackBoard

categories) for

best practice on-line pedagogy

showed 80% in accomplished or exemplary categorySlide4

This Presentation: outcomes based on full data set

S

tudent success and engagement with online learning tools

A

ssessing the strength of association and testing possible causality

F

urther work on the impact of motivation, intentions to use

t

hese tools and perceived effectiveness and ease of use

S

tudies of qualitative commentary on student experience of

learnline

Relationship

between

subcohort

engagement with on-learning and student satisfaction

Recommendation for policy and practice directions for

fleximode

teaching

Next steps for understanding the whole student engagement universeSlide5

Learnline use and success at a glanceSlide6

Multivariate test: association between LL use and grade holds up

(

n=8,970 enrolments,

Sem

2 2013-Sem1, 2015)

Adjusted for variables: study

mode, age, gender, language background, part-time or full time, BOA Slide7

Applying rigorous testing: the counterfactual Model

There are many

complicating and confounding variables including:

study mode, age, gender, language background, part-time or full time,

BOA

plus a range of other factors we cant measure or perhaps don’t know about.

A

randomised

clinical trial where Collaborate or

learnline

exposure was the

“treatment” would account for these

complicating and confounding factors

in

the population tested

Since we cannot do this, we can still satisfy the assumptions

of

a

randomised

trial. This is called the potential outcomes or counterfactual framework.

This is done by applying two statistical tricks that can cancel out the hidden biases (the things you

haven’t

or cant measure) that can lead to misleading or inconsistent findings.

In this model we try to cancel out the biases on both the grade outcome and on the students self-selection to participate in

learnline

. We then subtract the potential mean for the group exposed to

learnline

from the potential mean for the unexposed group to obtain an average effect of exposure to

learnline

. Slide8

The relationship between

Learnline

activity and grade holds under the Counterfactual model

Learnline

activity in deciles

G

radeSlide9

Collaborate use = successSlide10

Covariate confirmation: Increased levels of Collaborate activity have a

positive effe

ct on Grade Awarded in the Unit*

Adjusted for Part-time Status, NESB, Indigeneity,

BoA

Higher Ed.

Crse,BOA

Sec. Education Slide11

The Gains of Collaborate Activity holds against the most rigorous test

Percentage of Collaborate Activity

Mean

Numeric

Grade ScoreSlide12

Is collaborate an optional extra or integral?

Collaborate Effect by Unit Type and Mode of Attendance

External &

Core

External & Common

Internal

&

Core

Internal

&

CommonSlide13

Does the Motivation to Study Account for the Relationship between Perceived Effectiveness of LL and Use

Perceived Effectiveness

Usage

Motivation

X

XSlide14

Perceptions that influence usage

Usage

(intention)

Ease of Use

Sense of Social Presence

Psychological Resources

Sense of Classroom Community

UsefulnessSlide15

Student commentary on Learnline Slide16

Possible improvement areasUseability Not perceived as user

friendly, needs a more attractive interface that is easier to navigate, issues with different sites looking and being set up differently, and problems with links not working

Discussion

boards and interactive tools

Often

difficult to use and to easily keep track of discussion threads.

Wikis

not viewed very

favourably.

Need

for more consistent use across units of interactive tools facilitating student:student and student: teacher interactionRecorded Lectures and online tools Concern

that units

did not

consistently

provide

recorded classes or

if provided often not loaded in

a timely

fashion.

Concern with the quality of recorded sessions students often finding it hard to hear

information. Students

would like more consistent access to the collaborate tool across all their units

with fewer technical

glitches

that impact on access.

Assessment

Concerns

with the process of assignment submission including clarity on due dates and better acknowledgment of safe receipt of submitted assessment

items. Students

would like more guidance with requirements of assessment tasksSlide17

Cohort engagement and satisfaction: perceived use correlates with perceived effectiveness

LL engagement of student cohort pairwise comparison

Cohorts with significantly lower BA/NI ratios & areas for attention

Mode

external>internal*

internal: useability of Learnline

Gender

female >male *

male: recorded lectures and online learning tools

Age

mature age > under25 *

under 25: recorded lectures and online learning tools

Home Language

English > not English *

not English: Discussion boards and interactive tools

ATSIs

Non ATSI > ATSI *

ATSI: LR support (but strong on staff)

*

statistically significant

 Slide18

Learnline engagement and satisfaction by unit typeSlide19

Policy and practice recommendationsPromulgate causal relationship between increasing engagement of students

L

earnline

and

L

earnline

tools with academic success to the student body and academic staff

Promote and instigate the broader use of collaborate across the university

Act on the priorities for improvements for

L

earnline

useability, recorded lectures and online tools, discussion boards and interactive tools and assessment identified from qualitative student commentary obtained during this study

Sense of social presence and classroom community could be enhanced to improve perceptions of Learnline’s usefulness and useabilityEncourage a review of all online units against best practice online pedagogy and make amendments to online instances as requiredSlide20

Student Engagement Universe

Towards A Comprehensive Framework

Pro Vice-Chancellor Academic

v3b

, 19 April 2016Slide21

Formal Learning that is:personalised (recognises prior learning)accessible (offered via flexible & dynamic modalities)connected (to peer, professional & industry networks)

targeted (tailored learning packages)

supported (caring staff, connected peers, services)

And that is:

relevant (to industry, further study and context)

r

ecognised (duly credentialed and transferrable)

value for money and effort

What do students want?

Quality Assurance

Curriculum Design and Learner EngagementSlide22

Video conference

2

nd

Site

Main Site

Bb Collaborate

Mobile devices

Learner

Teacher

Room

CHANGES IN STUDENT-TEACHER ENGAGEMENT METHODSSlide23

Interact

with

CDU staff

Access course and unit

i

nformation

Submit assignments

Interact with non-CDU networks

Access and move proprietary resources

Access broad information

and support services

Interact

with

student peers

LAYER 1: STUDENT ENGAGEMENT NEEDS

Access

and

move OER resources

Submit and access enrolment information

ESTABLISH AND DEVELOP PERSONAL PRESENCE

Undertake proctored exams

CONTENT

ASSESSMENT

INTERACTION

MEMBERSHIP

Access grade information

Check work for text matching

Create (and

co-create) and

move resources

Apply

for

advanced standingSlide24

Social Learning

Media

Virtual

Classroom

LMS (+Mobile)

CDU Website

Campuses

Library & Bookshop

ePortfolio

LAYER 2: STUDENT-FACING SYSTEMS

WIL

Specialist Teaching Facilities

Virtual Reality

Academic Integrity (

eProctoring

, Text

M

atching etc.)Slide25

LAYER 3: BACKGROUND ARCHITECTURE

Asset storage

and servers

Student

Portal

Common Technical Standards (

Caliper

?)

Identity, Roles

& Permissions

Data Warehouse, Querying & Reporting

Common

User Device Standards (BYOD?)

Universal Authentication

Universal Analytics

Integrated Support