Normalization of Board Marks for Admission to Engineering C - PowerPoint Presentation

Slide1

Normalization of Board Marks for Admission to Engineering Courses

Findings and recommendations of the Joshi CommitteeSlide2

The Issue

Students leaving high school look for various career opportunities, including in Engineering.

Many courses, many examinations.

A lot of load on students.

High stakes.

Students turn to specialized coaching classes.

General education neglected.

Coaching distorts outcome of merit assessment examinations.

Institutes of Technology/ Engineering Colleges suffer from this distortion.Slide3

MHRD’s Response

Eliminate multiple entrance examinations by clubbing them together

JEE (Main) for all engineering institutions/colleges,

Another specialized examination for

IITs

.

Take into consideration Board performance for engineering entrance decisions

As an eligibility condition for admission to

IITs

,

As a component in composite score for merit lists of

NITs

and other institutions/colleges.Slide4

Committees Formed

Committees arising from deliberation in Council of IIT’s

Damodar

Acharya

Committee (8 March, 2010):

Inadequacy of present admission system brought into focus.

T.

Ramsami

Committee (11 November, 2010):

Feasibility of utilizing Board marks (in the form of percentiles) recognized.

Committee arising from deliberation in Council of NIT’s

S.K. Joshi Committee (13 August, 2012):

Findings presented here.Slide5

Objectives of Joshi Committee

Specified Terms of Reference

Validating the normalization formula using actual results of various Board and refining it based on its validation.

Detailed objectives

To evaluate implementation methodologies and effectiveness of various possible schemes,

To validate the chosen scheme for its consistency and application for JEE (Main) 2013,

To identify and enlist relevant issues, which are not covered under the scope of current study, for proper implementation of the scheme. Slide6

Inputs Received

by Joshi Committee

Inputs obtained by formed by the Chairman of CBSE (implementing agency) from

A ‘

Core Committee

’ formed by

Chariman

-CBSE, comprising experts from ISI, IIT’s and other prominent institutions,

Glenn Rowley of Australian Council for Educational Research

(ACER),

Jim

Tognolini

and Jon

Twing

of Indian Centre for Assessment, Evaluation and Research

(CAER).

Further analysis/validation by members of the Joshi Committee, including those participating from the ‘Core Committee’.Slide7

Decisions of Joshi Committee

Objective 1: Evaluation of methodologies/schemesSlide8

Issue: How to Equate

Board

Marks

Framework provided by NIT Council

:

60% weight on JEE-Main, 40% weight on Board scores.

Discarded option

: Use of linear transformation

Adjustment only for mean and standard deviation ,

2012 Boards data showed board-to-board difference in score distribution – even after such transformation.

Accepted (with modification) option

: Use of Board percentiles

Percentiles of different boards treated at par,

Recommended by

Ramasami

Committee.

Modification

: Use of transformed Board percentiles

Modification of

all

percentiles to bring them to JEE (Main) scale

No change in relative ranking,

Makes Board scores ready for combination with JEE (Main) score.Slide9

Issue: How to Address

D

ifference

of

D

ifficulty Levels

of

Subjects

‘

Rasch

model’ explicitly take into account difficulty level of a subject .

Specialized computational methods based on such models are still at developmental stage.

Such model can ‘compensate’ for differences of difficulty levels of subjects

within a board, but

not across boards.

Use of this model also amounts to moving away from percentiles, recommended by

Ramasami

Committee.

Joshi committee does not recommend the use of such a model.Slide10

Issue: How to Address

D

ifferences

in

Marking

P

attern

of

Different

S

ubjects

Possible solution: Equate Board subject marks by using percentiles BEFORE aggregating.

Possible difficulty

Much heavier computational burden,

Greater chances of unforeseen hitches,

Greater need for communication with several boards to resolve confusion,

Time frame for computation is short,

Data gathering/validation mechanism is not yet in place.

Joshi Committee Decision

: Equate Board marks AFTER aggregation.

Saving grace

: Usual aggregates are widely used (and accepted) for determining ranks within Boards.Slide11

Issue: Which Subjects

to

Aggregate

Number of subjects should be as large as possible (emphasis on school education as a whole).

Most boards have at least five subjects.

Mathematics and Physics are REQUIRED for JEE-Main.

Joshi Committee decision

is to use five subjects:

Physics,

Mathematics,

Any one of Chemistry, Biology, Biotechnology, Computer Science,

One language,

Any subject other than above four.Slide12

Issue: Basis Group

for

Normalization

Discarded Option

: All students

All students do not have appropriate subject combinations.

Discarded Option

: Passed students with appropriate subject combinations

Pass percentage varies from board to board,

Truncating at pass-mark would create board to board disparity.

Chosen Option

: All students with appropriate subject combinations.Slide13

Issue: Nature of Calibration

with JEE (Main)

Marks

Normalize Board aggregate marks to make their distribution match

JEE (Main) aggregate marks of

all appearing candidates (Option 1)

JEE (Main) aggregate marks of

candidates from that board only (Option 2)

Choice between the two options were made on the basis of additional data analysis for validation.Slide14

Decisions of Joshi Committee

Objective 2: Validation and fine tuning of chosen schemeSlide15

Assumptions Behind

the Options

Assumption behind Option 1

All boards have same merit distribution.

Assumption behind Option 2

Different boards have different merit distributions.

This difference can be measured (and adjusted for) by the performance levels of students of different boards in JEE (Main)/AIEEE.Slide16

Risks of Adjustment through JEE (Main) Performance

Students of some boards perform poorly in JEE (Main) / AIEEE.

This disparity may be due to

Poor merit/ability,

Non-alignment of board examination pattern with JEE (Main) / AIEEE

(rank correlations indicate this),

Lack of instruction in English and Hindi

(only available languages for JEE-Main / AIEEE),

Less access to coaching,

Load of an extra subject in board (for some boards).

All these effects are confounded.

If performance disparity is attributed only to merit disparity, confounding factors are ignored.

Solution may be worse than the problem.Slide17

Findings from Analysis of 2012 Data

Option 2 requires different treatment of Board percentiles; Option 1 does not.

The differential treatment of Board percentiles under Option 2 can be quite extreme:

80

th

percentile of Maharashtra Board equated with 50

th

percentile of CBSE;

Topper of Maharashtra Board has normalized Board score 331; Topper of Jharkhand Board has 274;

A CBSE student with AIEEE marks 130 and Board percentile 93.1 has a rank of about 18,000; a Maharashtra Board student with that profile has a rank of about 34,000.

This amounts to penalizing the Maharashtra Board student for poor AIEEE performance of peers from that Board.Slide18

Key Findings

Vastly different treatment of percentiles of different Boards (Option 2) would not be fair in the presence of confounding factors.

Representation of various boards in different sections of the merit list

Option 2 changes the present (2012) pattern substantially,

Option 1 has less drastic impact.

Option 1 would produce more equitable performance across boards.

Option 1 would be the right choice.Slide19

Decisions of Joshi Committee

Objective 3: Issues relevant for implementationSlide20

Issues and Actions

How to implement the selected method

Algorithm and Flowchart provided.

Operational issues

Timelines for Processing and Analysis provided.

Other issues

(no action within purview of Joshi Committee)

Collection of data,

Formatting of data,

Validation / Authentication of data,

Adherence of time frames for data delivery.Slide21

Further Recommendation

A Core group may be formed by CBSE for implementation of the normalization scheme

focussing

on

Data Collection,

Nature of Data,

Validation of Data,

Timeline for Data Collection,

Data Processing.