Benchmarking as a best practice in assessment - PowerPoint Presentation

Slide1

Benchmarking as a best practice in assessment

Student Affairs Assessment Council

Wednesday, October

28,

2015Slide2

What is Benchmarking?

Benchmarking is the “

continuous systematic process for evaluating products, services, and work processes of organizations that are recognized as representing best practices for the purposes of organizational improvement

.”

-

Spendolini

, 1992Slide3

What can benchmarking do?

Justify programs/services within student affairs

Improve quality

Demonstrate affordability

Develop strategic plans

Formulate policy

Aid in making decisions Slide4

Types of benchmarking

Internal

: making comparisons within an organization

Competitive

: examines performance against peer or competitor organizations

Functional

: looking at high performing processes across the industry

Generic

: looks at organizations outside of one’s field/industrySlide5

What are “Best Practices”??

Best practices are typically the finest examples of process, program delivery, or methods in a given area that produce the highest known quality outcomes.

-

Palomba

& Banta, 1999Slide6

Various types of best practices

Usually determined by those meeting and exceeding a

list of criteria

Can also be referred to as

performance indicators

Can also be referred to as

benchmarks

Can also be referred to as

standards

Whatever you choose for the benchmarking project you undertake—define your term and be transparent.Slide7

A note about performance indicators

An indicator is “a relevant,

easily calculable

statistic that reflects the overall condition of an enterprise.” –

Ewell

, 1997

Easily calculable = easily calculable across institutions

Note that learning/developmental outcomes generally cannot be evaluated by performance indicators such as retention rates, graduation rates, and faculty-to-student ratio.

Indicators do not inform anyone as to the cause of the value found in the program—nor do they indicate how to improve.

USE indicators as measures, but we must incorporate student learning/development outcomesSlide8

Benchmarking is….

Comparative

Data intended to be public (for better or worse)

Intended to learn about what you can improveSlide9

Considerations to be made prior to benchmarking

Are you going to share the information publicly? If so, with whom?

Are you benchmarking services and processes or student learning and development?

Is there national data that can help benchmarking data be more legitimate? Examples include:

NSSE

College & University Counseling Center Directors Data Bank

EBI surveys

American College Health Association Survey

National Association of Colleges & Employers Career Services surveys

Which institutions allow you to compare yourself in a meaningful manner?Slide10

UC Peer Institutions

University of Connecticut

University of Pittsburgh

Syracuse University

Temple University

University of Illinois @ Chicago

George Mason University

University of Alabama @ Birmingham

University of South Florida

University of HoustonSlide11

Steps in the benchmarking process

Adapted from

Upcraft

&

schuh

, 1996

Define the problem

Make sure benchmarking is appropriate

Determine what to benchmark

Choose who should be involved in the benchmarking project

Select comparable organizations

Determine what information will be collected

Determine how the information will be collected

Analyze the data

Take action

Assess the action takenSlide12

1. Define the problem

In student affairs, problems can be identified through:

Student use studies

Student satisfaction studies

Student needs studies

Reviews

Change in resources

Realignments/reorganizations

Change in law, policy, procedure

Crisis or emergency situations

Feedback from various constituents Slide13

2. Make sure benchmarking is appropriate

Ask yourself:

Can information from other organizations help my organization and help me achieve my outcome?

Must do preliminary investigation; you’ll need to find out if other organizations have been successful in whatever you are researching.Slide14

3. Determine what to benchmark

Remember:

Product, service or process

Product = educational programSlide15

4. Choose benchmarking team

People directly affected by process should be involved from the very beginning

Process should involve staff (or students) who deal directly with the problem

If staff (or students) are involved from the beginning, there will be greater ownership of the results and a greater likelihood that solutions will be implemented.Slide16

5. Select comparable institutions

In general, use peer institutions

Can also use:

Programs, services, processes which are similar to your own

Reputations for quality programs, services, or products

Valid information to offer

Hearsay v. Evidence

Reliable information to offer

Can’t always just look at websites; need to ask for data/assessment results

Leadership which values benchmarking

Don’t forget…help other institutions out! You may be calling them in a year for info.

It is important to note that despite comparable characteristics (institution size, academic functions) student affairs divisions may be VERY different. Try, as much as possible, to compare oranges to oranges. Slide17

6. Determine what info will be gathered

Determine what information is needed to improve the program, service or process

Develop a format/protocol which provides a framework within which information can be gatheredSlide18

7. Determine how information will be collected

Telephone interviews

Personal meetings/site visits

Surveys

Document/publication review

Archival informationSlide19

8. Analyze the data

Code = look for themes

Be sure to ask yourself....

“what didn’t they tell us…and why?”

Be sure to address the problem identified initially

Include specific recommendations AND solutionsSlide20

9. Take action

Easiest action = those that save resources, or require little to no additional resources

More difficult action = when significant resources are required to solve the problem

Discuss this with leadership PRIOR to benchmarking process. If no resources are available, then what?Slide21

10. Assess the action taken

Did the action taken actually solve the problem?Slide22

Keys to successful benchmarking

Needs to be done correctly, or not at all

Process should involve those who are directly affected

Process should have support and commitment of the leadership of the organization

Use organizations that are comparable, willing to participate, and can offer reliable/valid information Slide23

Examples

http://

www.case.org/Samples_Research_and_Tools/Benchmarking_and_Research/Benchmarking_Resources/CASE_Benchmarking_Toolkit/Alumni_Relations_Benchmarking_Template.html

http

://

www.enhancementthemes.ac.uk/docs/case-studies/uk-and-international-case-studies-of-practice-in-student-support.pdf?sfvrsn=18

http://www.planning.salford.ac.uk/__

data/assets/pdf_file/0020/20657/Examples-of-Benchmarking-Reports-in-Higher-Education-Membership.pdfSlide24

References

Alstete

, J. W. (1995).

Benchmarking in higher education: Adapting best practices to improve quality

. San Francisco:

Jossey

-Bass.

Ewell

, P. T. (1997b). Identifying indicators of curricular quality. In G. J. Gaff, L. J.

Ratfcliff

and Associates,

Handbook of the undergraduate curriculum: A comprehensive guide to purposes, structures, practices, and change

. San Francisco:

Jossey

-Bass.

Palomba

, C. A., and Banta, T. W. (1999).

Assessment essentials: Planning, implementing, and improving assessment in higher education

. San Francisco:

Jossey

-Bass.

Spendolini

, M. J. (1992

). The benchmarking book

. New York:

Amacom

.

Upcraft

, M. L., &

Schuh

, J. H. (1996).

Assessment in student affairs: A guide for practitioners

. San Francisco:

Jossey

-Bass.