Thursday, November 5, 2009

Way to improve Jobmine's matching process

Yes, yes, there are tons of ways to improve jobmine. Googling "jobmine sucks" returns tons of reasons why Jobmine lacks usability. I won't go into it here, because though important, they don't directly affect the core functionality of jobmine. Inefficient job matching process, however, does.

The matching process is as follows: during round one interviews, employers have three weeks to interview students and assign a rank between 1 and 9 (not necessarily distinct) to each qualified candidate. After the three weeks deadline, students see the jobs that they are ranked for. A rank of 1 shows up as an "Offer", and a rank >1 shows up as "Ranked". Now, students are given a weekend to rank each of the positions from 1 to 9 (not necessarily distinct). Jobmine then matches students and jobs by minimizing the combined rankings of students and employers. (More details here.)

Sounds good in theory. In practice, students have difficult choices when faced with "Ranked" for a job they want, and "Offer" for a job they don't like as much. Students know that if they rank an "offer" job a 1, they will get the job for sure. This security makes offers attractive. The situation actually happened to me once, and like many students I took the offer. Imagine my delight as I found out that my first choice job was left unfilled.

I think that the problem lies in the lack of information flow. Students don't know how other students will behave. Incidentally, there have been facebook groups set up to facilitate information flow, allowing students to share their ranking results and preferences. There were people who emailed all candidates to ask whether anyone is taking this job. Unfortunately, Jobmine's interface makes it seems as though we should be as secretive as possible. But why shouldn't we share information to achieve a collectively better result?

One solution that would work well is a rolling ranking process: accept student ranking submissions that result in an offer being taken any time throughout the weekend. The system can then update the other jobs that the student was offered/ranked for to indicate their unavailability. This change can then be reflected on other students' ranking pages: e.g. if John is offered for job X, and Mary is ranked 2nd, and John takes a different job, then Mary would see "Offer" on her rankings page.

At least one advantage of this method should be clear: non-econ majors wouldn't have to study game theory to find the optimal rating strategy, and everyone will have better chances in choosing the jobs they want the most. This is done while preserving employer's ranking privacy -- more so than the facebook group and emailing!

A less obvious advantage is that the number of jobs filled should increase. We can expect that a small percentage of students will have a large number of offers from high-profile companies, usually ones that take in a flexible amount of students. These students must reject all but one offer. I assume that many of these positions are left unfilled, as people who are ranked for these positions settle for offers, which are safer. These companies would be left with no matches, and may be disinclined to continue into second round, reducing the number of total jobs. This is clearly avoidable in a rolling process. Offers not yet taken can go to a second group of students. Offers not taken by this group would again be trickled downwards. More jobs will be filled, which is a plus for both employers and students.

You might note one potential disadvantage: employers will be more likely to be matched with students further down on their priority list. With the employers I know, it's not a problem. Students think that the difference between an "offer" and a "rank" is huge, and feel awkward working for an employer who preferred someone else. The employers, on the other hand, highlight the difficulty in choosing students. To them, everyone they ranked is good. Employers, unlike students, have the ability to not rank a student (and rightly so).

Incidentally, I think that students should be able to reject an employer as well, and in this case, a more elegant matching method exists. Currently, students can be matched with a job they don't want even if they rank it 9. They then face the choice of wasting 4 months or failing the term.

Currently, to prevent being matched with a job, you must initiate a tedious sign-off process involving the student, the CECS, and the employer. The student must show that the position is sufficiently different from the stated job description. CECS seems to think that this is the only legitimate reason a student might reject a job, but there are other reasons: You might not like your future boss. You might not like the culture, bureaucracy, or disorganization of the company. You might not like the salary. There's a reason that an interview is a two-way street, and the mere fact that CECS wants to be so strict about this shows that there are jobs students don't like. The employer has the opportunity to not rank a student, so why shouldn't it be the other way around? CECS might say that taking a job is better than being unemployed, but that should be the student's decision, not anyone else's.

CECS might also suggest that an easy sign-off process would encourage students to apply to jobs without the intent of accepting it, and prevent student accountability. This is a legitimate concern, however we should note that an application is just as much a cost to the students as it is to the employer, and students don't normally have an incentive to pay this cost. Of course there are people who apply to jobs for "practice" or other reasons - but these people are already doing it! It's easy for a serious person to figure out how to avoid being ranked. CECS is really penalizing the honest students: first years who panicked under the stress of the economy and other students who genuinely wanted the job at first.

The other problem that CECS might want to prevent is employer disappointment. It's a legitimate concern in a shrinking economy, but wasting 4 months of a student's time just comfort an employer's hurt feelings? I don't know about that. I think it's fair game to let law of demand and supply dictate which jobs will be filled. Besides, job rejection is actually a norm in society, and employers should understand that as well.

(Some people may be disinclined to make the distinction between"good jobs" vs "bad jobs". There is no dichotomy here, I agree, but I'd also invite them to just look at some jobs titles in Jobmine - "Algorithmic Trading Developer", "Parking Attendant", "Student Marketing Analyst", "Head Cook", "Chemistry Research Assistant", "Brine Maker"....)

So, with this rejection ability in place, we have a second solution to the initial problem: to make job ranking and job rejection asynchronous: let job rejection happen any time in the weekend, and keep job ranking at the end. Every time a job is rejected, update everything as before (as in John and Mary). Like before, a student who is certain that he wants to take a particular job offer can reject all the other positions except one (and in doing so passes this information to other people). But this time, a student who is still deciding between two of his offers can pass down the rest of his offers, without having reached a decision.

Of course, I haven't modeled/tested these claims, and if there's something wrong with my reasoning, please let me know.

End of Entry

10 comments:

Oliver Warbux said...

sounds reasonable enough. The one thing I've found is CECS tends to wanna do a lot more ass-kissing for employers than it does for students, which would explain the 'inequality' so to speak in terms of what we can do with our post-interview state of affairs. Giving them the 'power' to pick their 'creme-de-la-creme' of students and then let the student/jobmine-random-machine-thingy 'dictate' fate through our ranks seems to appeal to them. The concerns of student's don't really rank up high on CECS's priority list.

Unknown said...

@warbux: I think that the first method would work in favor of both the students and the employers. It's much easier to think of examples where job matches are increased, and pretty difficult (though possible) to think of a case where it is decreased. (though I understand that this is no proof... don't exactly have time to really model this)

As to the rejections, i agree that CECS probably won't like it. It looks good on the CECS to keep the job count high and the % employment high, so the first instinct is to keep as many jobs in jobmine as possible, regardless of whether students are actually willing to take these jobs. Some first year students will apply to them out of fear and panic. If you think a bit long term, though, letting students choose better experiences will help strengthening the connection with employers that are more valuable to the school: employers who hire co-ops not to do some short-term manual/busy work, but one that have long term interest in the students. These employers add much more value to the co-op process, and right now, their preference seems very low on CECS's list

NullReferenceException said...

1) Then there will be a conflict of interest where students of competing ranks would battle for an unfilled job or a filled job with lower rank, unless it is a first come first serve basis. Even then, the employer would complain about not allowing a more suitable candidate take the job. This will risk students not having a job and make them check jobmine obsessively over the weekend to a point that can affect their academic careers due to its importance.

2) Maybe CECS was encouraged BY employers to do this (sign-off)? After all, employers consider most juniors to be immature.

3) More employers appear to be contacting offered students directly before rankings open. This is their way of adapting to some of the shortcomings of jobmine... or capitalizing on it.

4) Algorithmic trading developer? That's a bad job? I have no idea what you are thinking. Go do some research. Student marketing analyst is also a valid position, not vague at all. I would say only head cook and brine maker are derogatory jobs. Please don't be ignorant.

5) Finally, a normal blog post~ yay!

Unknown said...

1) If you mean two people seeing an "Offer" for the same job (which I think that's what you mean, but I'm having trouble deciphering your words) - of course we can't do that, even if they are ranked the same! It would have to be either random or make rankings distinct - but you're right in that neither of these are elegant - the "decline" job is much better imo. As for checking aimlessly, I agree, and there are definitely ways (i.e. sending notifications) that Jobmine's ENTIRE system should adopt (e.g. in the interview process too).

2) In that case employers are being very unfair: in the real world a job application is not binding. I don't understand how immaturity have anything to do here - wouldn't an employer NOT want to hire an immature student, and thus ALLOW them to sign off the job?

3) I think employers are being smart that way. Jobmine doesn't like this, but it works much better for both the employer AND the student. The point I wanted to drive through was that increase flow of info = good for everyone.

4) I invite you to read the passage again, and you'll see what kind of a contrast I was going for.

Boggled said...

some grammar mistakes that are irrelevant (like "imagine my delight[ed]") XD

I just have a thought about the rankings. It seems that if this system is implemented, everyone would wait for everyone else to rank first and then make their choice based on the assurance - since submitting your ranking would be permanent in this case - no chance to change one's mind. Of course there are a few people who would take offers immediately and that may facilitate some action... but I'm not too sure about the rate/efficiency of the method. In the end there might be a lot of rushing to rank... which is stressful.

In the mean time, loo co-ops should follow the comradeship of laurier co-ops on fb... because really, secrecy here accomplishes nothing but self foot shooting... -_-

@william - does your comment mean that you think chemistry research assistant is a "bad" job? =P

Steven said...

good constructive post. lots of big technical words that i can't understand =P i can't offer much comment becuz i don't have coop =P

but the first thing i thought when i read this is the unintended consequences which lillz mentioned. it plays out like a nash game where most of the students would hold off decisions to wait for new information to come and the net result of the system is that no new information does come, other than the uberoverachievers who get multiple offers from multiple prestige companies. The information percolation from that is doubtful because I don't think overachieving academically is similar to overachieving on the job mkt.

And if the information benefits end up being negligible due to the unintended nash game, it becomes impossible to exert the political will to change administration's policies.

fun post tho. =)

Oliver Warbux said...

When's it all said and done, they do plan on rolling out the new version of the jobmine interface and what-have-you. Perhaps it'll come with some welcome changes, but I wouldn't keep my hopes up (the changes they made to Quest just pissed me off more than anything).

Unknown said...

about the wait-until-last-minute: this is why I like the second method better, and I think the second problem eliminates the problem as much as it can be eliminated --

the people we're concerned with are those with multiple offers, since knowledge of their offer rejection is what will benefit other people. as i understand, most of these people are either certain which jobs they want to take, or they're deciding between 2 of the offers. i rarely see people trying to decide between 3 or more offers. if they are certain about not wanting a particular job, then there's nothing for them to gain to wait. i think that most people would be willing to help out other people if it doesn't make them lose anything - so i don't think the waiting game will occur.

but you're right - i don't think that people who are unsure will tend to submit their rankings early, and nor should they. this system would not depend on them to improve the rankings

incidentally... i think that encouraging employers to talk to students before doing the actual rankings might be equivalent to a "softer" version of the second method! the idea is that if an employer can ask a student before ranking him/her, then they can give a chance for the student to reject, and thus pass the offer to someone else. In some cases this will allow for an easy way to "sign off" on jobs as well. might be easier to convince CECS to do this instead...

Unknown said...

oh @warbux, about the new jobmine (aka joberloo), i was told not to put any hopes up by someone who worked on it for co-op

NullReferenceException said...

I think you are too obsessed with offers to understand what I am trying to say in 1) which is the MAIN flaw that has not been addressed. I invite you to read the passage again, and you'll see what I was going for.

As for 4), you are trying to force the reader to determine which are the good and bad jobs. Unless you are listing good job, bad job, good job, bad job, etc. Even then a comma-separated list wouldn't do.