Ever since 2008 when we at PaGaLGuY started publishing our own rankings of b-schools, the possibility of overenthusiastic b-school students rigging the rankings survey to favour their institute has been giving us the heebie-jeebies.
It’s a good worry to harbour, because if we didn’t make sure that our rankings were free of taint, why would you, the readers, follow them as religiously as you do?
That paranoia made us design a rankings algorithm which at its very core minimised the chances of any b-school student getting even the name of his b-school in his survey at all. Even if a b-school student did get a questionnaire laden with his school all over, there would be several times as many of his classmates who would not get questions about their b-school at all, thus cancelling out any avenue for rigging.
So all those who argue that schools with a larger student body can actually turn the survey in their favour should know that many of these respondents would not get the name of their b-school even once in their individual questionnaire and most wouldn’t get it enough number of times to really cause an impact. No doubt some would get a ‘favourable’ questionnaire, but there’s not much even 50 people can do considering that adjacent b-schools in the rankings are separated out by thousands of votes.
Yet, we have faced our share of attempts by b-schools to play truant with the rankings. In 2008, we made the students of one b-school do community service for trying to rig the rankings using multiple voting. In 2009, one school disqualified a bunch of students from its placements process for attempting to rig the rankings. In 2010, we finally took the unpleasant but necessary step of disqualifying two b-schools from the rankings for what was clearly a concerted and planned effort to cheat the survey.
In 2010, we added another security feature to the survey: Facebook logins. Facebook, as you all know, is the best curated repository of real identities on the Internet today. If we were surveying you for the PaGaLGuY Rankings in real life, we would meet you in person and ask you for your name, age, gender, etc. By using Facebook logins, we get hold of all this data about our respondents (unless they have blocked it out using their privacy features in Facebook account settings). Of course, all that data is private with us and you need not worry about it reaching the wrong hands at all.
But Facebook with its in-built features also helps us ascertain whether those voting in the PaGaLGuY Rankings are real people or fake Facebook IDs made solely for the purpose of voting in the rankings. Once we identify the impostors, which we do after digging through the data on an item-by-item basis.
So has rigging of the rankings survey stopped then?
Not really.
It has conclusively reduced for sure, but has taken newer forms that are more individual-driven than planned efforts of colleges. Here are some stories below from the few days and nights that we spent in cleaning up the survey data for the PaGaLGuY B-school Rankings 2012.
1. Love makes the world go round, but it also helps your b-school get a better rank, it seems. We encountered several cases of respondents who were b-school students and were getting their girlfriends, boyfriends and wives — who seemed to have nothing to do with b-schools — to vote in the rankings. How did we know about these amorous associations? It was enabled in their public profiles, so no snooping around by us.
2. In one particular case of a b-school in west India, the respondent had created two Facebook IDs, voted using both of them. Each of the profiles listed a different woman as being “in a relationship” with him, available to public view, and both of those women had voted in the rankings too! The profiles of both women seemed positively genuine.
Does one woman know about the other? We don’t know and it is not our business. But dude, you might have been pulling off a two-timing gig in real life, but you can’t two time your way into the PaGaLGuY Rankings!
So that we err in favour of the credibility of the rankings, we deleted the responses of everyone who voted as described in 1 and 2 from the rankings survey. Of course, we didn’t delete the responses of couples where each looked positively like a genuine MBA student or aspirant.
3. Knowing how tough planned rigging of of the survey en-masse by b-schools is, many b-schools students got their families and far-off relatives to vote in the survey. Family gatherings seemed to have been used to the hilt for the purpose, with all aunts, uncles and each of the seven cousins being asked to vote one after another. We could locate the ‘nucleus’ of such activity in many cases, but in some we couldn’t. Either way, like I said, we discarded these responses entirely, including that of the ‘nucleus’, as collateral damage in our mission to publish a clean rankings.
Moral of the story: The penalty for being oversmart is that you lose your own genuine response too.
And you know what the saddest part is? All, ALL, of the responses we cleaned as suspect data could be traced to specific students or IP addresses of b-schools that are in the Top 20 of this year’s PaGaLGuY Rankings. Not a single one from b-schools lower down the ranks, which you would usually expect to go unethically ballistic over getting a better rank.
I wonder what this says about the so-called ‘quality students’ of our top schools and the insecurities they harbour about the brand-tag on their CVs. Or whether in the frenzy of getting students with high quantitative abilities, b-schools are missing out on checking how likely their students are to do the right thing.
Maybe starting next year, we should start deleting one or two additional random responses from another student of the same b-school as penalty for each case of such proxy-rigging done by one student. We will decide the exact steps next year. Even if it means deleting all votes by students of that b-school, we will do whatever it takes to keep this ranking free of the smarty-pants.