A Brief Proposal to Reforming & Strengthening JEE
How Some Talented Teenagers do not Get Selected in IIT-JEE?
(All Disclosures from the past 4 JEEs are done by IITs under RTI Act)
IIT JEE (Joint Entrance Examination) is being conducted for admissions in IITs flagship B.Tech. and 5y M.Sc. degree courses for the past 5-decades. This examination is considered the most competitive examination in the world; however, this examination has been completely black boxed and nothing was revealed to candidates regarding their marks, model solutions etc. With the enactment of RTI Act 2005, JEEs started opening up.
This is a brief exposure in the direction of the recent disclosure by the NRI Nobel Laureate Prof. Venkatraman Ramakrishnan that he could not make to IIT JEE, coupled with many disclosures in recent past, in respect of IIT-JEEs, that the candidates’ selection is bungled up, (knowingly or unknowingly) with multifaceted errors by which many talented teenagers have been excluded.
Scenario One : He may be one among the following highly talented 994 students (a few samples included below), who were excluded in spite of scoring huge marks in IIT JEE 2006:
Sl. # Maths Physics Chemistry Aggregate Result
(total marks in each subject were 184)
E1. 36 116 127 279 Not Qualified
E2. 101 96 54 251 Not Qualified
E3. 91 94 53 238 Not Qualified
E4. 75 104 52 231 Not Qualified
And many other such 994 HIGH scoring candidates were Not Qualified.
Though, in the same examination, and in the same General category, in lieu of the above 994 candidates, the following 994 low scoring candidates were Illogically and Illegally declared qualified for admissions in IITs, by ignoring around four thousand of the candidates who were superior to the following candidates:
Sl. # Maths Physics Chemistry Aggregate Result
(total marks in each subject were 184)
S1. 37 48 69 154 Qualified
S2. 45 56 55 156 Qualified
S3. 45 56 55 156 Qualified
And many other such 994 LOW scoring candidates were Qualified
In view of the above: The question is: Can selection decision be so ad-hoc to completely ignore the merit and performance? In spite of IITs many ex post facto desperate attempts, in the past 4 years, IIT could not yet justify how the high scoring candidates, as given in the Table (E1 to E4), were excluded.
For qualifying, a student is required to score some minimum cutoff marks. The cutoffs were precisely calculated to be 7, 4 and 6, respectively, in Mathematics, Physics and Chemistry, by the cutoff determination procedure, which was submitted by IIT under an affidavit in Calcutta High Court. Cutoffs as used by IITs were 37, 48 and 55, which cannot be calculated by any of the cutoff procedures submitted so far.
On being challenged, IIT, instead of correcting errors, took the stand that admission decision cannot be questioned, which was upheld by a single-bench of the Hon’ble Court at Calcutta. On an order from a Division Bench of the Hon’ble High Court, experts from ISI Calcutta and IISc Bangalore independently worked-out the Chemistry cutoffs to be Six by the procedure as disclosed by IIT in the affidavit-in-opposition. The Hon’ble High Court finally disposed-of the appeal, in Jan. 2010, by declining to interfere as the matter was not within their domain due to not having the expertise and not within the scope of the judicial review.
Cutoff calculation is based on simple “mean” and “standard deviation” of statistics, which can be found in school level books. The latest attempt to explain the cutoffs, in August 2009, by a Committee of two IIT Directors’ (Bombay and Guwahati) came up with a third procedure in which they took recourse to the “iterative process”, which is used to increase the cut-offs “with every iteration” to get the desired number of candidates, to finally fit the predetermined cutoffs through a tailor-made formula. While determining the cut-off of one subject through the iterative process, the faulty implantation of this procedure excluded the candidates who had high marks in other two subjects before arriving at the final cutoffs, and thus affected the cutoff calculation of those two subjects. This is done without even admitting the departure in the procedure. By correct implementation of the third procedure, the cutoffs calculations are precisely arrived at {42, 44, 51}. If this was the procedure then many of the students who are already studying in IITs, would be disqualified. Open question is:
Can any one among Millions in whole IIT system (including faculty, alumni, and students) refute the subject cutoffs to be {7, 4, 6} by IIT’s own procedure, which was affirmed in an affidavit in Calcutta High Court and on IIT’s own data, in place of official cutoffs at 37, 48 and 55, in JEE 2006? Why does IIT force victims to go to courts on the matter in which IITs are world leaders?
Is this the accountability of whole IIT System? Is it solely to delay the matter to make it infructuous? The cutoffs for 4 consecutive JEEs, having identical questions/examinations patterns and the examinees’ population, were:
Year Max. Marks
in one subject Subject cutoffs Aggregate
Maths Physics Chemistry Cutoff
2006 184 37 48 55 154
Correct calculation Ver. I -8 -3 -6 178
Ver. II 7 4 6 178
Ver. III 42 44 51 151
2007 162 1 4 3 206
2008 162 5 0 3 172
2009 160 11 8 11 178
Scenario Two : In contrast, in subsequent JEEs (2007 to 2009), it was revealed that students with as low as 5% (Sl. No. 7) marks in one of the three subjects were declared qualified. This is in complete contrast when students with as high as 29% marks (lowest in any one subject) were rejected. It was revealed that a student (Sl. No. 1 below) with as few as 10% marks in Mathematics was placed with very high All India Rank of 1234 in JEE 2009, which enabled him to be admitted in any of the IITs. Others with poorer performance, have been selected for IIT admissions as shown in the Table below:
Sl. No. Maths Physics Chemistry Aggregate Result, AIR in CML
In JEE 2009 (total marks in each subject were 160)
1. 16 134 119 269 Qualified, 1234
2. 12 88 85 185 Qualified, 7306
3. 77 15 88 180 Qualified, 8113
4. 109 107 13 229 Qualified, 3061
5. 92 85 11 188 Qualified, 6869
In JEE 2008 (total marks in each subject were 162)
6. 10 106 76 192 Qualified, 5308
7. 122 8 66 196 Qualified, 4999
8. 96 109 15 220 Qualified, 2903
The selection of above students with very poor marks in one of the subjects, is contrary to the very basic policy of JEE selections, which states that a student should have basic knowledge of all the three subjects for which a student should score marks above the cutoffs in each of the three subjects. The cutoffs, in subsequent JEEs, fell to amazingly low levels and thus frustrated the very purpose of inducing subject cutoffs.
Selection of students with very low marks in one of the subjects, is contrary to JEEs own stated selection criteria. If such students were not qualified by not having reasonably good cutoffs (not as high as of JEE 2006) their places would have been taken by those students who have scored respectable marks, in each of the three subjects, as per the stated criteria.
It may be noted that the Physics cutoff in 2008 was Zero. If cutoffs are to be as low as Zero, then why to have the cutoffs? Simply allow every candidate scoring above zero to be considered for merit list. Why to have a sword hanging on every JEE aspirant’s head by a tender thread, when it has no effectiveness? Why is it to put every JEE aspirant in undue stress, which ultimately results in bungling?
Scenario Three : Disclosure of question papers and model answers have revealed that there were serious evaluation errors in questions as well as some questions were directly picked from out-of-syllabi undergraduate (UG) level books. Chemistry paper of 2006 has 11 marks of questions wrongly evaluated and 12 marks of questions directly picked from UG standard books. In question paper of Mathematics 2008, it was revealed that 18 marks (11%) of questions were wrongly set/evaluated. It is more disturbing to know that, in the same examination, a student with a score of 10 just marks (6%) in Maths was found eligible to be admitted in prestigious IIT Kharagpur and other newer IITs.
If the questions are wrongly evaluated and/or made open, it always penalizes talented students who spend enough time and try to come up with a correct and unambiguous solution. Similar is the case when out-of-syllabi questions are directly picked up from UG level books; in such cases students who attend coaching classes, are benefited, because coachings have expertise in these matters and help students to memorize such likely to be questions. As reveled by the Nobel Laureate that he possibly did not get through JEE because he did not go for coachings due to a conservative look.
Scenario Four : It was disclosed that many seats including in General category were lying vacant in spite of high demand for every seat in IITs through JEEs. This is due to the reason that IITs do admissions only once, though it is done multiple times elsewhere. In 2009, for the first time, IITs did second round of admissions by which some of such vacant seats were filled. It was revealed that many of such vacant seats as well as supernumerary seats were created and filled through back-doors by IIT Kharagpur till 2005; beneficiaries include a current IIT director and one the most controversial Chairman, JEE 2006. In 2006, seats high in demand, were shown artificially filled. It was also revealed that all the selected faculty wards in 2006, from IIT Kanpur and Kharagpur, scored amazingly high marks in Chemistry. When directed to open up the answer scripts of such high scoring faculty wards, it was revealed that the scripts were destroyed in undue haste and against their own norms. A ward of an IIT Director (and Ex-Chairman JAB) was caught impersonating in JEE. It was also found that wards of power professors (Directors, Deans, Heads, and closer to management) are routinely admitted in a few IITs; most of the admitted wards through JEE in IIT Kharagpur belong to this category. Such phenomena cannot occur without knowing personal identities of the candidates, which implies such a prestigious examination has no sanctity. Due to such bunglings, many genuine candidates were deprived off their admissions in IITs through JEE.
The above is not the end. There are instances where IITs can come clean and adopt effective transparency. For example, perhaps IIT-JEE is the only examination which does not disclose the marks at the time of result declaration. IIT-JEE does not disclose evaluated ORS. IIT-JEE does not disclose the model answers after the examination on the same day. IITs disclose the marks and model answers after almost 3-4 months of examination conduct when the whole admission process is complete (including second round of admission); even for the apparent errors, IITs take refuge “admission decision (even if flawed) cannot be questioned.”
A Brief Proposal to Reforming & Strengthening JEE
Issue I: Subject cutoff marks vary widely from Zero (too low) to 55 (too high) across 4 JEEs having similar patterns of questions and identical set of examinees. The dismally low cutoffs render subject cutoffs redundant. This is due to considering all students who appear for the examination for cutoff calculation, though only two percents make it to JEE’s main Merit list.
Proposal: Cutoffs should be stable across the years though they may vary marginally due to differing levels of question difficulties. If cutoff calculation is based on a percentile factor, say 75 percentile (Minimum of Top 25% students), it will ensure cutoffs to be stable, meaningful and have a large enough pool of candidates available for final selections. For example, the following table gives the subject cutoffs based on 75 percentile:
Table I: Proposed cutoffs based on Min of Top 25% (at 75 percentile)
Year Subject Cutoffs Candidates above all these cutoffs
Maths Physics Chemistry
2009 17 12 18 44,901
2008 26 19 26 30,310
2007 25 26 32 29,870
2006 22 20 25 28,839
Issue II: Errors in question setting and evaluation. There was wrong evaluation of 11 marks of Chemistry questions, in JEE 2006, as revealed by disclosure of question papers and model answers. Another disclosure of JEE 2008 revealed that a student, in general category, with 10 marks in Mathematics was eligible to get a berth in IIT Kharagpur while there were reported evaluation errors of 18 marks in Mathematics.
Proposal: Disclose model answers after the examination on the same day (current practice is to disclose model answers after 3-4 months of examination when all admissions are completed); invite public to send feedback regarding correctness within a week and then freeze the answers for final evaluation. This is the practice used in some states, e.g., CET in Karnataka.
Issue III: Possible rejection/tampering of ORS (Optical Response Sheet) if filled with HB pencil. Marking of ORS by HB pencil has a possibility that the filling can be dim/faint, and thus the ORS is rejected by the scanner. Additionally, there is a possibility of tampering. JEE does not adopt any additional safeguards to avoid tampering. It has been revealed that many selected faculty wards scored exceedingly high marks.
Proposal: Additional safeguards should be taken in terms of marking ORS by black ball pen. Backside of ORS can be coated with a carbon film to ascertain any tampering and a carbon duplicate of the ORS can be retained by the candidate as a proof. The current practice of empowering JEE Chairman, with the raw and coded data must be abandoned; personal identity of the candidate should not be revealed.
Issue IV: JEE administration promotes coaching institutes. There are many acts of IITs which essentially promote coaching. For example, coachings display model answers on same day of the examination, and thus boost their image in public, while IITs do this in next 3-4 months. Candidates login to their web-sites and thus become soft target. Similarly IITs do not disclose marks at the time of result declaration; students with low ranks or not-qualified are trapped by coachings. IITs announce an Extended Merit List (EML) of many tens of thousands of candidates, whose cutoff was simply 11% in 2009, which is too trivial to score. Coachings use these numbers to boost their performance which is hardly an achievement; this is misnomer.
Proposal: Disclose model answer on examination day, provide marks with result declaration. Rename EML to ‘ranks’, assign a rank to every candidate, as in AIEEE. Provide unambiguous, complete and correct information. Set conceptual/analytical questions of XII standard level.
Issue V: Heavily skewed marks distribution. There is essentially a bimodal marks distribution, a few hundred candidates score decent marks for around 8000 - 10000 seats; most others are in the other. Therefore, scoring of marks for most such students can be contributed to random selection of choices. Some questions are unnecessary complex (at times, directly picked from undergraduate books) without testing much analytical or conceptual knowledge.
Proposal: Set proper and balanced questions of XII standard; this will reduce dependence on coachings; students will concentrate more for XII examination.
Issue VI: Binary grading of multiple choice questions does not test analytical skills.
Proposal: Introduce differential grading to test students’ knowledge by multi-level grading rather in 2-levels. Each option can be assigned different marks, in positive or negative, based on its closeness to the correct solution. It will make dispersion of marks wider and improve selection quality drastically. This strategy will take care of wrong setting of questions, if any, without making it open.
Issue VII: Poor admission counseling. Many seats lye vacant in spite of cut throat competition, this is a national waste, though some of such seats were filled by second round of admissions in 2009. Information disclosure has revealed that some of such vacant seats were offered to IIT KGP’s own wards without ever qualifying JEE, till 2005. In 2006, most seats which were high-in-demand, were shown artificially filled.
Proposal: Make counseling fully on-line and publish filled-in and vacant seats on day-to-day basis. Prepare a wait list during admission counseling and fill/upgrade seats by sliding. Set deadline for students to join, else allot those seats to wait-listed candidates. It would be best if JEE, AIEEE and other counseling are combined. A student UID can provide a perfect solution.
Issue VIII: Lacks transparency. In Information age and being world leaders in IT, marks and ORS are not opened at result declaration. Model solutions are not disclosed on the day of examination. Marks and model solutions are disclosed when the whole process of admissions is complete. There are roll-back too, e.g., authenticated copy of the mark-sheets, which unearth the cutoff fiasco of JEE 2006, is no more supplied now.
Proposal: Disclose complete information at appropriate time, e.g., disclose model solutions on the day of examination, marks and ORS with result declaration so that errors, if any, can be corrected immediately. This will also enable a student to take proper decision regarding repeating the examination next year or joining an engineering program.
Issue IX: Students do not get time after XII exam, thereby, they do not concentrate much on XII.
Proposal: Defer examination by a 4 - 6 week period. There is enough time still left.
Issue X: Poor ethics in JEE administration. Admission decision (even if flawed) cannot be challenged due to signing a declaration. ORS were destroyed in undue haste. There is a pattern in some IITs that wards of powerful faculty are selected routinely. Many candidates were selected from a single room/centre. There are reported cases of impersonation.
Proposal: Change thy mindset from feudal to facilitator. Amend the declaration to have a choice of not waiving the right to challenge irregularities without having any bearing on candidature with an explicit disclaimer that not waiving the right is not a disqualification. Correct the errors, if any, instantly rather taking a legal shield; settle disputes during the period from result declaration to counseling. All records must be kept till disposal of disputes.
Do not repeat same set of people for question setting, computer experts and JEE administration. Randomly allocate examination centre. Take finger prints at the time of application, examination, counseling and admission. Strengthen vigilance.
Best is to have a single JEE examination (replacing AIEEE and all other states’) by which a student (securing a UID and a score, may be an aggregated score over three sittings of the examination spanning over a month) can seek admission in one of the engineering college
