Just when we were close to concluding that Prometric had perfected the on-ground delivery of the Common Admissions Test (CAT) after the lessons it had learned in 2009, entire testing labs at at least two test centers in New Delhi failed this Saturday, on day 8 of CAT 2012.

But instead of providing the choice of another date and center to the affected candidates, Prometric asked a substantial number of them (at least nineteen) to travel to faraway centers immediately and write their tests until 8 pm, late into the evening.

One affected candidate narrated her plight in a comment on our news report about the test failures,

This is how my experience at the Panacea Testing Centre (Mayapuri) today (21st Oct) made me feel : disillusioned and cheated. As a candidate, who has paid for the facility of choosing a date and time of convenience, what are my rights? When after an hour and a half of dilly-dallying by the center “managers” (Ha!!), you are told that due to a certain “technical glitch” (suitably vague as always), the center has been “shifted” to Dwarka and that the exam would be conducted there (which takes another hour and a half of travelling and some more waiting around, and mentally you are no longer in the zone). Didnt you charge me good money to “choose” a “time” and “place” for my exam? Arent you going back on BOTH counts? Do I not have the right to say, “NO WAY”, this is a service you are bound to provide and if not I WILL CHOOSE when I wish to take it again! I will not be carted off to some center of YOUR fancy, at a time of YOUR interest, fatigued, dazed and frustrated, to face one of THE most critical exams I will ever take! But I did all of that today, and for that, I feel ashamed and weak that I did not take a stand. Deep respect for the 10 warriors who decided to do otherwise. To me a leader takes stand and no college can teach you that!

(Original comment here)

On PaGaLGuY’s asking, Prometric did not divulge the number of candidates who were forced a changed of center and slot, instead replying, “All affected candidates have already been contacted by Candidate Care and provided with a new appointment within the testing window.”

So in effect, at least nineteen candidates (probably much more) would have been forced to compete for an IIM seat under stressful conditions at a time and place which was not of their choice, and Prometric would have them believe that they were provided with a level playing field with other candidates who enjoyed this luxury. We don’t know if there were more than nineteen, because Prometric wouldn’t say how many.

Should these and other candidates who have experienced less than fair test conditions accept that as the norm and move on? As with a lot of things, following the money — comparing the price of the CAT with that of the Graduate Management Admission Test (GMAT), its internationally-accepted standard and counterpart — suggests that they should. Read on.

At the current US dollar-to-rupee conversion rate, taking the CAT costs a candidate about $30, while the GMAT costs $250. Within the price you pay to these tests, both have to recover the cost of sourcing the questions that form the test as well as of providing trustworthy testing infrastructure and security to you, and then perhaps make some profit too. Unlike the CAT, the GMAT is hardly ever known to experience technical snags on a large scale and then respond with unsatisfactory measures.

For the sake of argument, even if one assumed (by a vast stretch of imagination) that the Graduate Management Admission Council (GMAC, the owners of the GMAT) was making a fat 100% profit on the cost, it would still be spending $125 in business expenses to deliver one GMAT test. That’s at least four times the retail price of the CAT. Think of it as the Apple iPad costing much more to produce than the selling price of a Chinese-make tablet computer. It means that either the GMAC is ridiculously overspending on its test, or Prometric is cutting corners in bringing out the CAT. The former is unlikely, as no business that is more than 50 years old would be running in such an un-optimised manner. Nor are the IIMs and Prometric incurring huge capital expenses which are invisible in the price charged to the candidates.

The Rs 1,600 you pay in buying the CAT voucher would have to fund the question creation process, the on-ground execution costs and the centers that lend out their labs, Prometric’s own business expenses and if any still remains, some profit for Prometric and the IIMs. (The IIMs make some additional money by charging b-schools for the CAT score reports, but that money does not go to Prometric.)

The more likely possibility is that Rs 1,600 is rather too tight an alley for Prometric to even attempt to wriggle out the same smoothness and quality of testing experience that the GMAT provides at our conservative estimate of $125, even if you accounted for the larger pool of questions that the GMAT has to spend on procuring due to its adaptive nature and the fact that it is manufactured in the USA at first world costs. And corner-cutting is written all over the CAT in its non-committalness on practically everything that matters to the test-taker. Here are a few examples.

While the GMAT specifies being “tested in a manner that is consistent with the applicable professional testing standards” and having “your test administered according to standard technical specifications and under standard environmental conditions” as the rights of test-taker (see here), the CAT provides you absolutely no rights whatsoever. It in fact unapologetically tells you in its disclaimers, “As with paper and pencil testing, or virtually every other human endeavor, a very small number of problems could occur that might prevent a test from being delivered and/or a result from being generated. In the unlikely event this does occur, every effort will be made to correct the problem, up to and including the administration of another test.”

The vagueness in the “every effort will be made to correct the problem” could be used to make it mean anything, including packing you off to another test center after putting you through hours of stress and throwing administration consistency of the test to the wind.

What should instead have happened, is that the affected candidates on October 21 should have at least been provided a choice between taking the test immediately at the new location and rescheduling it to a new date within the testing window. Prometric cannot claim to call the CAT a fair test without having allowed ALL candidates to take the test in the conditions they chose.

Prometric displays the same non-committalness about owning up to the precision of the CAT scoring process. Four years into the computer-based CAT, Prometric still does not publish the reliability and validity statistics of the CAT to the public as the GMAT does, and as the ‘ETS Standards for Quality and Fairness’ stipulate. In Layman terms, there is no statistical evidence to show that the CAT is a credible instrument to measure your aptitude as a management student, or whether the particular way in which the CAT tests you will produce consistent results. On being asked why Prometric did not make these standard disclosures, a senior Prometric official once told this reporter that it was because the Indian public (students, parents) wouldnt understand the implications of these disclosures and would only raise further questions out of confusion. More recent emails by PaGaLGuY enquiring about the disclosures were not responded to by Prometric.

This hesitation in making the standard disclosures more likely indicates a test so cash-crunched that it cannot support the science it claims to be based on with solid evidence. The same professorial class at the IIMs who fight against governance autonomy in order to preserve faculty freedom and academic purity at their institutes support this by simply looking away.

The computer-based CAT continues to be a change designed to make the lives of the IIMs easier, whose professors were sick of proctering paper-pencil tests every year and wanted a way out. If any part of the new format of the test was supposed to improve things for candidates, it is yet to be seen. The simple test to establish this is to ask this question: If the CAT were to be reverted to its paper-pencil format tomorrow, what compelling reasons would the candidate community create an uproar to prevent it with?

The single answer the commentor’s questions therefore is, that as a CAT exam candidate you do not have any rights or entitlements to a committed quality of service. You have paid for a low-cost exam that enjoys a monopoly as an admission gateway to your dream b-schools, and you are getting exactly what you paid for (or didn’t). You will probably get to sit for the test at the most, prayfully at the slot of your choice, but sorry, you have no further right to quality if Prometric messes up.

Isn’t it ironic that getting into b-schools that supposedly teach you the best management practices requires that you first crack a test that does not employ the best management practices in its domain?