The
normalization used in NIT admissions 2013
The way it started
8 March 2010: Consequent to the decision in the Retreat of IIT Directors
and Chairpersons held at Manesar on 4 February 2010,
HRD Ministry forms a Committee
headed by Damodar Acharya
(Director, IIT Kharagpur) to look into the
"streamlining and rationalizing" of JEE, GATE, JMET, JAM, etc. The
committee submits an interim report emphasizing the need to change the existing
system of multiple examinations that has encouraged proliferation of coaching
and has apparently failed to produce a merit list that can correlate with good
performance in engineering programmes.
10 September 2010: Council of IITs decides to set up (see minutes,
page 9) T. Ramasami Committee, an extension of Acharya Committee, to assess the examination and admission
system to engineering programmes, with a view to
reducing the burden on students on account of multiplicity of entrance
examinations (JEE, AIEEE etc.).
The way it developed
11 November 2010: Ramasami Committee is formed. The Committee draws up a
plan for an alternative test scheme that utilizes Board scores and scores of a
national level examination for all engineering admissions. T. Ramasami (Secretary, Department of Science and Technology)
approaches ISI regarding making board scores comparable.
September 2011: ISI provides input, saying that treating percentile
scores of students in their respective boards at par would be the only logical and fair way to normalize
different board scores.
14 September 2011: Ramasami Committee makes a
proposal to the Council of IITs, seeking to replace the present admission
system by an alternative test scheme. According to this scheme, there would be
two common national examinations for engineering admissions, including an aptitude
test and an advanced test, and IIT admissions would be based on a criterion
(possibly from one of the six suggested options) that gives weightage to the
common national examination as well as Class XII board marks normalized on the
basis of percentiles (as proposed by ISI). The Council accepts this
recommendation in principle, (see minutes,
page 8) and asks Ramasami Committee to finalize its
report, which can be placed before Central Advisory Board of Education and
State Education Ministers for a final decision so that the new system could be
put in place by academic session 2013-14. The recommendations for the common
entrance examination for IITs and NITs are communicated through a press
conference held after the Council meeting.
November 2011: Ramasami Committee submits report,
with ISI input in appendix. In this report,
ISI support is used in respect of normalization of board scores only, and not in
specifying the roles of common national examinations. The proposal for a common
admission test for engineering admissions, based on national examination as
well as Class XII normalized board marks is ratified separately in the Council
of IIITs and the Council of NITs (see minutes
of joint meeting of councils, page 2).
22 February 2012: The alternative test scheme proposal of the Ramasami Committee is accorded "in-principle"
approval by the State Education Ministers (see minutes
of subsequent Council meeting, page 2).
The way it ended
11 April 2012 : Prior to implementation, the Hon'ble Minister of MHRD discusses the decision with the Chairmen, the Boards of Governors
and the Directors of IITs. He
also discusses separately with All India IIT Faculty Federation, while a "Core
Committee of senior academics" interacts with "the faculties of some
of the IITs" on the same.
25 April - 5 May 2012: The Senates
of seven older IITs take divergent views regarding IIT admissions, ranging
from complete support for the Ramasami Committee
proposal approved by the Council of IITs, to retention of the then prevailing
admission system with minor modifications.
12 May 2012: Council of IITs forms consensus to use Board percentiles as
well as JEE-Main (aptitude test) scores only
as cut-off, and to use JEE-Advanced (continuation of the IIT-JEE conducted
till 2012) alone while preparing the merit list for 2013 admissions. The
Council also opines that all other Centrally Funded Technical Institutions
(CFTI) including NITs would prepare merit list by using Board percentiles,
JEE-Main and JEE-Advanced with 40%, 30% and 30% weights, respectively. There is
no final decision though. It
is decided that there should be discussion with "faculty representatives,
two each from the seven older IITs" before any final decision is taken
on the matter.
28 May 2012: The Councils of IITs, NITs and IIITs hold a joint meeting
and watch a detailed power point presentation on a proposal for procedure of
admission to CFTIs, along with "the rationale, eligibility criteria, the
mythology of selection and governance structure that were proposed". After
an "open house" discussion, they reach
unanimity on the undecided consensus view of the Council of IITs to use
different criteria for IITs and other CFTIs. There is also a consensus that the
Joint Admissions Board of the IIT system should have complete control over the
conduct of JEE-Advanced, while a separate Joint Admissions Board with
representatives from IITs, NITs, other CFTIs and State Governments should
conduct JEE-Main. The Central Board of Secondary Education (CBSE) could provide
administrative support for both the examinations.
June 2012: IIT Faculty and alumni sustain pressure on the government to
ensure that there is no common entrance test, with some threatening
to go for a public interest litigation and others threatening to go on hunger
strike on the issue, emulating an earlier hunger strike in protest against
non-implementation of the recommendations of the Sixth Pay Commission.
27 June 2012: Council of IITs formally
decides to set JEE-Main rank among top 150,000 as cut off for eligibility
to appear in JEE-Advanced, and 80th percentile
in Class XII Board examination as eligibility to gain admission to IITs based
on JEE-Advanced performance. It decides to conduct JEE-Advanced through an
exclusive Joint Admission Board and a Joint Implementation Committee.
Epilogue
An analysis of the admissions data of
IIT-JEE 2012 reveals that a disproportionately small number of students
from villages and lower income groups registered for that examination. Among
the candidates who registered, those from the higher income group had four
times higher success rate in comparison with those from the lower income group.
Candidates from cities had higher success rates than other groups. The findings
strongly point towards the effect of coaching. Many students lacking access to
coaching apparently do not even register. Overall, the IIT system ends up
picking more than half of their students from a single board that represents
only about 6% of the students passing Class XII. Keeping faith in an admission
system that relies on such a narrow supply base requires a lot of faith.
Before the final decision on the matter of admissions was taken by the
Council of the IITs, the
Hon'ble Minister, HRD, had written to it:
"Coaching institutions have gradually replaced our secondary schools. ... We
are creating an army of children adept at cracking examinations but can they
think critically?" On the other hand, the All
India IIT Faculty Federation had reminded the Council that the majority of
the Senates of the seven older IITs want continuation of the present system of
preparing the final merit list on the basis of a version of IIT-JEE, and
demanded that "the collective majority views of the various senates must
be upheld". The ultimate
decision of the Council to reverse its earlier decision and to side with
the Faculty Federation took place after it became clear that the twin imperatives
of giving "due weightage to the performance of the students in class XII
board examination" and addressing "the concerns of the IIT
fraternity" (see minutes
of joint meeting of councils, page 2) cannot be reconciled.
The
normalization used in NIT admissions 2013
The story of the NITs
The way it started
18 November 2011: The Council of the NITs accepts the recommendation of
the Ramasami Committee, and records the consensus
view that a judicious mix of school and national level test performance will
ensure that the multiplicity of tests today and dependency on coaching would
get reduced largely (see minutes,
page 6).
28 May 2012: The Council of NITs, in a joint meeting with the Councils
of IITs and IIITs, reach unanimity on the consensus
opinion of the Council of IITs, that the NITs would prepare merit list for
2013 admissions by using Board percentiles, JEE-Main and JEE-Advanced with 40%,
30% and 30% weights, respectively (see minutes,
page 3-4; see advantage of using
percentiles rather than percentages). The consensus view is that a Joint
Admissions Board (JAB) with representatives from IITs, NITs, other CFTIs and
State Governments should conduct JEE-Main. A Joint Interface Group (JIG),
appointed each year by the JAB should implement the normalization of Board
scores, while administrative support should be provided by CBSE.
4 July 2012: The Council of the NITs decides to exclude JEE-Advanced from
its admission criterion for 2013. Instead, it decides to put 40% weight on
Board scores "normalized on percentile basis", and remaining 60%
weight on performance in JEE-Main, and to set up a committee to look into issues relating to normalization of the Class XII Board
marks on percentile basis (see minutes,
page 2).
13 August 2012: As per the decision
taken by the Council of NITs, MHRD forms a committee chaired
by S.K. Joshi, Chairman BoG VNIT Nagpur, which
includes directors of several NITs, Chairman-CBSE and some other experts. The
Joshi Committee is asked to validate the normalization formula using actual
results of various boards and refine it based on its validation. It is
authorized to "co-opt outside experts for dealing with the issues of
normalization of class XII board marks on percentile basis across various
boards". The committee does not have any representation from any board
other than CBSE. The CBSE, which has been
administering the AIEEE examination till 2012 (continued from 2013 as JEE-Main),
is identified as the nodal agency for implementation of the normalization
procedure for all boards.
The way it developed
September 2012: Before S.K. Joshi Committee holds its first meeting, Vineet Joshi
(Chairman-CBSE) takes the initiative in formulating a normalization procedure.
He engages Jim Tognolini and John Twing
of Indian Centre for Assessment Evaluation and Research (CAER), who prepares a report containing suggestions on how to normalize
board scores. The report starts with the premise that some boards have more
"able" students than other boards, and suggests that the performance
of students of different boards in the AIEEE/JEE-Main be used to differentially
adjust the scores of these boards. Since CBSE students perform better than
students of other boards in AIEEE/JEE-Main, this suggestion would benefit a
CBSE student at the expense of a student with the same percentile at Class XII
examination of another board.
September/October 2012: Chairman-CBSE uses a write-up based on
this report to shore up support for the principle of using AIEEE/JEE-Main
scores to make differential normalization of scores/percentiles of different
boards. He forms a Core Committee
consisting mostly of statisticians, and circulates this write-up as a
background note to be used as the suggested basis for arriving at a justifiable
normalization procedure. He also receives some preliminary
advice to this effect from Glenn Rowley of the Australian Council for Educational Research (ACER), and some
supporting data analysis from Neeraj Mishra and Amit Mitra of IIT Kanpur.
30 October 2012: S.K. Joshi Committee holds its first meeting. Jim Tognolini of CAER, appearing as special invitee, briefs the
Committee on the merits of using AIEEE/JEE-Main scores for "tracking"
Board scores/percentiles are pointed out. Joshi Committee, which does not
include a statistician, decides to
include the Core Committee (formed by Chairman-CBSE) in all its subsequent
meetings.
31 October 2012: The Core Committee, however
does not support the approach promoted by the Chairman-CBSE. In its input to the Chairman-CBSE, the flaws in the
principle of using AIEEE/JEE-Main scores for "tracking" Board
scores/percentiles are pointed out. It is argued that this approach entails the
strong and unverifiable assumption that the presumed difference between the
"ability" distributions of two boards is adequately captured by the
performance patterns of the students of those two boards in AIEEE/JEE-Main. If
this approach is used, a student may be penalized for weak performance of other students of the same board in
AIEEE/JEE-Main. Neither the Council of IITs nor the Council of NITs had
expressed sufficient faith in the JEE-Main score as a measure of relevant
"ability" so as to use it solely for engineering admissions. Further,
use of AIEEE/JEE-Main for tracking board scores effectively gives it more
importance than the 60% weightage decided by the Council of NITs. Based on
these arguments, the Core Committee recommends direct use of the board
percentiles.
30 November 2012: Joshi Committee holds second meeting jointly with the
Core Committee. Members have an open discussion on the pro's and con's of Chairman-CBSE's argument of using AIEEE/JEE-Main
scores for "tracking" Board scores/percentiles and the Core
Committee's recommendation of using Board percentiles directly. It is decided that JEE-Main scores would not
be changed, and that the board percentiles would be converted to this scale for
comparability. Chairman-CBSE suggests that the transformation of the board
percentiles be done differently for different boards, by using the JEE-Main
score distribution of students of each board for "tracking". Thus, a
procedure based directly on board percentiles transformed to JEE-Main score
scale (Procedure 1) and another based on different board percentiles
transformed to board-specific JEE-Main score scale (Procedure 2) are
short-listed. It is decided that data of 2012 Board and AIEEE examinations be analysed for studying the two procedures.
December 2012: CBSE sends JEE-Main 2012 scores as well as Class XII exam
scores for six boards (including CBSE) to various members for analysis. The
same data are also sent to Jim Tognolini of CAER.
28 December 2012: Joshi Committee meets. Members from ISI present results
of detailed analysis which shows that the "tracking" approach
(Procedure 2) results in gross distortion of the percentiles of different
boards. In particular, a student at the 80th percentile of
Maharashtra Board receives a smaller normalized score than a student at the 50th
percentile of CBSE. Other problems of this method are also pointed out. The
Committee unanimously decides to
adopt the method based on direct percentile (Procedure 1), and draws up a
structure for the final report. It decides to hold the final meeting by the end
of January 2013, and to check the analysis one more time in the mean while.
Chairman-CBSE skips the meeting, which is held at his office.
29 January 2013: Joshi Committee deliberates over draft report that
incorporates all the analyses carried out till that point of time. It is decided that finalization of the report
will take place over next few days, and an informal meeting may be called for
this purpose. Chairman-CBSE suggests that the logic behind the chosen method of
normalization (Procedure 1) be brought out more clearly in the report.
The way it did not end
2 February 2013: Sandeep Sancheti
(Convener, Joshi Committee) meets Chairman-CBSE at the latter's office to
discuss clarifications in the final report. They are joined by Debasis Sengupta of ISI Kolkata
by teleconference. Chairman-CBSE requests another round of face to face
discussion in the presence of Jim Tognolini of CAER.
The way it almost ended
11 February 2013: Convener and Chairman of Joshi Committee, together with
Abhay Bhat and Debasis Sengupta of ISI meet Jim Tognolini and John Twing of CAER
in the presence of Chairman-CBSE at the latter's office. The problems with the
use of AIEEE/JEE scores for tracking of Board scores (Procedure 2) are
discussed in detail, and the arguments in favour of
adopting the simple percentile based approach (Procedure 1) are sharpened.
Specifically, it is shown
on the basis of simulations that
Procedure 2 is expected to increase dramatically the dominance of CBSE students
at the top of the merit list, in comparison with the earlier scenario (with
merit list based on AIEEE scores alone).
18 February 2013: Joshi Committee circulates its report among members
(with copy to Additional Secretary, Technical Education, MHRD,
who had formed the Committee), with plan to finalize and submit it the next
day.
19 February 2013: Joshi Committee finalizes its report. In the evening,
it is asked by Amita Sharma (Additional Secretary,
Technical Education) to make a presentation to the Education Secretary before
finalizing the report.
20 February 2013: Convener and Chairman of Joshi Committee present the
findings of the Committee before the Education Secretary and the Additional
Secretary, Technical Education. Chairman-CBSE is also present. The presentation is
not completed for paucity of time.
4 March 2013: Convener and Chairman of Joshi Committee once again present
the findings of the Committee before the Education Secretary and the Additional
Secretary, Technical Education. Chairman-CBSE and Debasis
Sengupta of ISI are also present. Subsequently the report of the Joshi Committee is
officially submitted.
The way it started again
9 April 2013: Council of NITs meets to discuss Joshi Committee report.
Members of the Joshi Committee, together with some members of the Core
Committee, are invited to attend. The meeting is chaired by Hon'ble
Minister Pallam Raju
(Ministry of Human Resources), who is the Chairman, Council of NITs. The Hon'ble Minister is assisted by Amita Sharma (Assistant Secretary, Technical Education and
Member-Secretary, Council of NITs). The Convener of the Joshi Committee
makes a presentation
based on the report of the Committee. Chairman-CBSE is allowed to present some
new material, previously not shared with the members of the Joshi Committee,
and to make a fresh case for the differential adjustment of percentile ranks of
students of different boards based on their JEE-Main scores, which can be used
for "tracking". This is indeed the proposal that had been considered
carefully and rejected by the Joshi Committee before the Committee arrived at
its clear and unanimous recommendation. The surprise proposal produces utter
confusion. Chairman-CBSE asks NIT Council members to ask themselves what
decision they would take if the students concerned had been their own children.
The meeting ends with a decision to expand the Joshi Committee to review the
"new material" flashed by the Chairman-CBSE, together with the report
of the Joshi Committee.
10 April 2013: Joshi Committee is expanded,
with a mandate to advise the Chairman of the Council of NITs on "either options" (the
procedure recommended by the Joshi Committee and that promoted by the
Chairman-CBSE) "based on least departure, equity and transparency".
The newly introduced criterion of "least departure" (presumably from
status quo) is in direct contradiction of the decision
of the Council of the NITs to depart from the earlier practice of putting
100% weight on AIEEE/JEE-Main and 0% weight on Board scores.
The way it developed again
15 April 2013: MHRD circulates a presentation
prepared by Chairman-CBSE (in support of his proposal) based on additional
analysis of the Board scores and AIEEE scores, along with another presentation
prepared by ISI (containing detailed rebuttal of the arguments of Chairman-CBSE;
summarized here), and invites
comments from the members of the expanded Joshi Committee.
16-26 April 2013: Members of the expanded Joshi Committee deliberate over
email on the "options" and the presentations circulated. Sunil K. Sarangi (Director, NIT Rourkela) sends two messages providing detailed reasons why the
Joshi Committee's recommendation should be accepted. Debasis Sengupta
of ISI sends some clarifications
to help the committee understand the issue. Anil Kakodkar
(Chairman, BoG, NIT
Jalandhar) opines that in order not to deviate
from the 60-40 weightage decided by the Council of the NITs, the Joshi
Committee recommendations should be accepted. The procedure favoured
by Chairman-CBSE receives support from Rajat Moona (Director General, C-DAC, Pune), the evident fallacy of whose arguments are immediately pointed
out by Debasis Sengupta.
Samir K. Brahmachari (Director General, CSIR, New Delhi) endorses the views expressed by Anil Kakodkar in favour of accepting
Joshi Committee recommendations. Prem Kalra (Director, IIT Jalandhar) sends an opaque message in favour
of the procedure promoted by Chairman-CBSE, but his misquote of Anil Kakodkar
draws a swift contradiction from the latter. Meanwhile, Chairman-CBSE
remains silent, choosing not to
refute any of the counter-arguments
that exposed the flaws in his analysis
that had precipitated the deliberations of the expanded Joshi Committee in the
first place. Thus, the acceptance of the Joshi Committee recommendations
appears to become inevitable. Finally a meeting is called
in order to "discuss the observations / inputs received from the Committee
members". Ashok Misra (Chairman, Joint Admission
Board of JEE-Main) expresses his inability
to attend. Anil Kakodkar also expresses his inability
to attend, while reiterating his views and
saying that it is important that a decision is reached without further delay. Debasis Sengupta asks that in order to avoid delay,
members be given advance copy of any new material to be perused in the meeting.
29 April 2013: A handful of members of the expanded Joshi Committee meet
in the office of Amita Sharma, (Additional Secretary,
Technical Education), who chairs the meeting in the absence of S.K. Joshi. The Chair begins the
meeting by asking everyone to be flexible and strive for a consensus, without
realizing that when the scientific merit of one procedure over
another is beyond doubt, advocacy of flexibility amounts to interference in
reaching a scientific decision. She also reminds everyone that it is the fate of
"our children" that they are dealing with, as if the thought of a
certain group of children could alter their objective choice. She makes no
attempt to build consensus on the only logical option left to the Committee,
namely acceptance of the Joshi Committee recommendation. She deviates from the
terms of reference of the committee by permitting S.S. Mantha
(Chairman AICTE), a special invitee, to float a completely new proposal for
normalization that had never been circulated among the members. This proposal,
which also unfairly favours CBSE students, is readily
accepted by the Chairman-CBSE. The meeting ends without consensus. The members
are asked to provide their comments on the fresh proposal before 30 April, with
a promise that a written version of the proposal would be circulated by email
before the deadline.
30 April 2013: The
promised circulation of the new proposal does not take place. Debasis Sengupta of ISI prepares
a description of the proposal (as far as it could be understood from the
deliberations in the meeting of 29 April 2013) and circulates it, while pointing out
its obvious flaws, including the fact that it does not even conform to the
NIT Council decision on this matter. He also points out the series of maneuvers
on the part of the Education Department (and specifically the Additional
Secretary, Technical Education) to subvert the process of reaching a scientific
decision based on objective analysis.
1 May 2013: Additional Secretary (TE) admonishes Debasis
Sengupta of ISI for getting into a personalised discourse and making groundless allegations.
She reminds Sengupta that she is in the "ISI
Board" (possibly meaning ISI Council). When asked, she does not clarify
which part of the message is not a professional view, or why the fact of her
being in the "ISI Board" is relevant for arriving at a decision on
the normalization issue.
The way it eventually ended
6 May 2013: The JEE Interface Group (JIG) set
up, by the Joint Admission Board of JEE-Main, for the implementation of the
normalization of Board scores
decides to use a simple average of the
normalized score recommended by the Joshi Committee (Procedure 1) and the
normalized score promoted by the Chairman-CBSE (Procedure 2) for the 2013-14
admission to NITs. ISI learns about the final
normalization formula from press
reports.
Epilogue
The procedure adopted by JIG has expectedly created much confusion, and has been alleged
to unduly favour CBSE students. An opportunity to
make judicious use of board scores for engineering admissions has been missed.
The misconceived procedure is even fueling demands
for a complete rollback of the new admission system. A reversion to the
earlier system is likely to perpetuate dependency
on coaching, which the Council of the NITs had wanted to change.
The normalization formula adopted by the NITs for 2013
admissions
Suppose
*
A0 is the
aggregate score of a student in JEE-Main;
*
B0 is the
percentage score of that student in the Class XII Board examination, and P is
the corresponding percentile rank.
(Recall that the percentile rank of a student is the percentage of fellow
students with score below the score of that student.)
Suppose
*
B1 is that
aggregate score in JEE-Main, which corresponds to the percentile P among all
JEE-Main candidates;
*
B2 is that
aggregate score in JEE-Main, which corresponds to the percentile P among all
JEE-Main candidates from the Board of the concerned student.
Linear interpolation may be used, where necessary, to compute B1
and B2 from P. Then, according to the normalization formula adopted
by JIG,
*
the normalized Board score
of the student is Bfinal = 0.5*(B1
+ B2), and
*
the composite score of the student is C = 0.6*A0
+ 0.4*Bfinal. This is the score used for
preparing the merit list.
The scores B1 and B2 are computed from B0
by using the relation between the scores and the percentiles, for three sets of
scores:
*
the set of Class XII Board
scores (including B0) for the Board of the student concerned;
*
the set of JEE-Main scores
of all candidates;
*
the set of JEE-Main scores of all candidates from
the Board of the student concerned.
The percentile-to-score relation of the first and the third set vary from
Board to Board. This is illustrated through the accompanying figure.
It is seen that a common percentile P in the two different Boards leads
to the same value of B1 but different values of B2.
The Joshi Committee, with support from ISI, had recommended the use of B1
alone as the normalized Board score.
The Chairman-CBSE had promoted the use of B2 alone as the
normalized Board score.
The following is a selected part of a hypothetical example given in a newspaper
article to illustrate the normalization procedure.
