I sometimes want to fail more people in my classes. This is not because I am evil (although some people here seem to think so), but because I want the people who graduate from our computer science program to be truly the best in the world.
When I came to Carnegie Mellon, I was surprised at the insanely high quality of our undergraduates in Computer Science. I knew the PhD program was ranked #1, but I had no idea how awesome the undergrads were. Still, I think CMU and other top universities in the US need to fail a few more students in their classes.
The philosophy in US universities seems to be mostly one of making it really hard to get into the programs, but once you're in, the chances of graduating are really high. In fact, most rankings of American universities such as the one from US News place quite a bit of weight on four- or five-year graduation rates -- the fewer students that fail, the higher the university will be ranked. I find this counter-intuitive. While I understand that prospective students want to know that if they come here they will not be flunked, I think we all need to accept that mistakes are sometimes made in the admissions process.
In some other countries, like Guatemala where I went to high school, the philosophy is exactly the opposite. Pretty much anybody can be accepted to any university. However, a large fraction of the people who enter end up failing out. The reason this appeals to me is that rather than making a decision based on a single test score (the SAT) and a couple of recommendation letters, universities get to test students for the span of several years before giving them a seal of approval.
Should I be stricter with my grades?
Subscribe to:
Post Comments (Atom)
Theoretically yes, practically no.
ReplyDeleteOne class (yours) can't change the system that exists, so if you were to buck the trend and "go for fail," this will have a negative effect.
1. You teach 251 better than in the fall, if this became your philosophy, more would take 251 in the fall and less SCS undergrads would really learn 251.
2. For those who take it in the spring, their grades will be "unfairly" lower than those who take it in the fall, even though they probably learned more.
3. Others outside of CMU (who don't read this blog) won't understand as they assume the current system.
Sure, all those are "silly" reasons, but unfortunately they are reality.
NOT ALL OF US WHO TAKE YOUR CLASS ARE CS MAJORS. PLEASE KEEP THIS IN MIND.
ReplyDeleteWhat are you trying to optimize? You say that you want the people who graduate from the computer science program to be truly the best in the world. So does it stand to reason that you should fail the bottom 90% (assuming you can reliably measure them) in order to raise the quality?
ReplyDeleteBy the way, I'd also like the faculty to be the best in the world...
By the way, I'd also like the faculty to be the best in the world...
ReplyDeleteYes, and we do get fired after nine years if we're not. That's the deal with tenure.
NOT ALL OF US WHO TAKE YOUR CLASS ARE CS MAJORS. PLEASE KEEP THIS IN MIND.
ReplyDeleteDon't take my class :)
Makes perfect sense.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteIf you and only you start giving low grades, that wouldn't help anything really. You'd really need a bigger paradigm shift.
ReplyDeleteAlso, 15-251 is required for other majors such as Cognitive Science.
Coincidentally enough, I dropped 251 today. Hopefully I make a better linguist than a computer scientist.
ReplyDeleteWhat happened to the whole "change starts with me" philosophy?
ReplyDeleteCoincidentally enough, I dropped 251 today. Hopefully I make a better linguist than a computer scientist.
ReplyDelete:( Now you're making me look evil!
What? I thought you liked looking evil.
ReplyDeleteChange starts with you, true, but what I'm saying is unless you convince some higher power to make sure that other teacher increase their criteria, your class would transform from this annoying yet fun class to just annoying bottleneck.
Now, that is even too evil for you.
Anyways, there's already kind of a class with this philosophy...it's called OS.
ReplyDeleteAnyways, there's already kind of a class with this philosophy...it's called OS.
ReplyDeleteWhich is not required for the major here.
Now, that is even too evil for you.
ReplyDeleteIn the end, I'm a big softie.
Why fail someone??? Why not give them a poor grade (very poor). At IITs we have (AA, AB, BA, BB till DD) and any grade below that is fail. So give DD (or 4/10) to a person and pass him. If you a fail a student and he/she takes that seriously then they might become depressed or will commit suicide. Will you ever forgive yourself if you spoil someone's life because of your high standards. Saying this is easy but I am sure once you will do that you will feel very bad from inside. I sincerely hope and pray for the common good that you should not take such extreme measures.
ReplyDeleteAnyone who commits suicide after getting a low grade has really bigger problems in life than can be discussed here.
ReplyDeleteI agree with Can. Everybody fails at some point. It's part of life.
ReplyDeleteI said fail and not a low grade. And this is a serious problem in many other colleges (which I think are far far far more selective than american colleges). The expectations are so high that they are driven to extremes.
ReplyDeleteP.S. I said far more selective (this does not mean better)
I failed a class. I switched majors. Surprisingly, life goes on. I can still find jobs.
ReplyDeleteOther people have lost billions, lost their families, lost their arms. Life goes on.
It's really not the teacher's responsibility to make sure students can handle the pressure.
A teacher should not care about the majority only. If you are a good teacher and I think Luis is awesome than the well being of every student is his responsibility.
ReplyDeleteI surely care about all my students, but I also think that e.g. people should sometimes be fired from their jobs or fail in classes.
ReplyDeleteBut definitely as a last resort... If you can warn the class that now if you wont work hard then I will fail you and still someone does not work hard, than failing is justified.
ReplyDeleteI believe France has a system similar to that of Guatemala - let anyone in and chuck them out when they fail the exams. While possibly fairer, since it gives everyone a chance, it probably also wastes a lot of resources since some people who want to be a lawyers or doctors etc. end up spending a few years on a degree everyone (but they themselves) knows they will fail.
ReplyDeleteThe greatest lecturers should not feel bad to make it the toughest for the students. If you think you are one of the best, which seemingly you are, then go ahead and fail more... assuming raising the bar means failing more.
ReplyDeleteI agree and identify with this post...as an undergrad in a CS program (I'm going to grad school next year).
ReplyDeleteI see so many kids coming through our program that just want a degree to land a job. They don't really care about CS, they don't really care about learning. Some of the questions 4th year CS majors ask me disgust me--I can't believe they haven't encountered the C programming language AT ALL or a compiler.
I remember my second year that a CS major in a lab once asked me how he was supposed to run VC++ (from the lab computer) to compile his UNIX program (on ANOTHER computer). He didn't even understand that the instructions he had just followed made him connect to a machine that was different than the one in front of him.
Fair point re: not all faculty getting tenure. But that was the easiest of my points to respond to. You still haven't explained what you're trying to optimize. Is their an optimal number of graduates? An absolute competence threshold?
ReplyDeletePut in a more extreme way, why not fail all but the best student?
Also, look at the problem from a student's perspective--after all, CMU is competing with other schools for those students. Choosing a school is a major investment that's hard to back out of without significant switching cost. For a student who plans to make a good-faith effort to succeed, it is very useful for a university to pretest him or her and tell the student whether that effort is likely to lead to success.
ReplyDeletePerhaps the university could accept students more liberally, but students should favor schools that offer reliable predictors of success. In fact, schools could be judged on the accuracy of their predictions.
Daniel,
ReplyDeleteI'm trying to optimize the "quality" of our program. This not easy to define, but for example, giving somebody a degree in CS from CMU that does not quite understand some fundamental concepts makes our program lose quality.
In terms of your comment about CMU competing with other schools for the students. You do have a point, but I think the best students *like* the challenge. If we suddenly started failing more people at CMU, we may get less people coming here at opposed to our competitors, but those who would not come, to a large degree, would be the weaker students.
I'm trying to optimize the "quality" of our program.
ReplyDeleteWell, then, pass only the best student in your program every year. The quality of your graduates will be higher than any alternative strategy! Problem solved.
Do I get a prize?
No quite true. I think numbers also make a difference. Graduating the top 10 people is better than graduating just the best one, since maybe numbers 2 and 4 could go on to start the next Google, and number 7 could get a Turing award.
ReplyDeleteI think it's a no-brainer that basic competence be required for graduation. That's an absolute threshold, not a relative one. You could address that by having a pass-fail qualifying exam graded on an absolute scale.
ReplyDeleteIs your complaint that CMU is producing unqualified graduates, or that some of the ones CMU is producing aren't the best? Those may be correlated concerns, but they are quite different.
"This is not because I am evil (although some people here seem to think so), but because I want the people who graduate from our computer science program to be truly the best in the world."
ReplyDeleteIf this is your goal, I think you are approaching this entirely the wrong way. A lot of care does go into selecting students for CMU, so instead of worrying about dropping the bottom 10%, why not focus on becoming the best teacher you can be, and training the students to become excellent computer scientists.
They will be better motivated to do well and make you proud of them, as opposed to being constatnly terrified that they're going to fail.
Be the change you want to see in the world
ReplyDelete--- MK Gandhi
Luis you currently are the best teacher in SCS but you should inspire your students even more so that they achieve greatness.
The mediocre teacher tells. The good teacher explains. The superior teacher demonstrates. The great teacher inspires... Luis fails you.
ReplyDeleteNOT ALL OF US WHO TAKE YOUR CLASS ARE CS MAJORS. PLEASE KEEP THIS IN MIND.
ReplyDeleteDon't take my class :)
--
It's required for a Cognitive Science major.
So you're saying: it's ok for me to try to educate the best cs majors in the world, but when it comes to cognitive science majors, they don't have to be as good?
ReplyDeleteYes, because cogsci majors don't need to be amazing at CS. CS majors need to be amazing at CS. cogsci does include some CS, it's true, but not to an extent that cogsci majors need to be above and beyond at it.
ReplyDeletedon't fail us, please. don't make us be psych majors.
there's certainly a place for quasi-cs educations with less cs than a cs major but substantially more cs than many non-technical majors. this sort of education is genuinely useful and it would be good for the world if more people had it; and easy cs classes seem like an important part of getting more of those people in the world.
ReplyDeletethat said my favorite classes always were cs/math courses that were just too hard for me. often learned more from a tough prof than a not-tough one.
a tough but unfair professor, though, is no good.
don't fail us, please. don't make us be psych majors.
ReplyDeleteI won't fail you :) But you'll learn a lot in my class.
"In some other countries, like Guatemala where I went to high school, the philosophy is exactly the opposite. Pretty much anybody can be accepted to any university. However, a large fraction of the people who enter end up failing out."
ReplyDeleteWell, in fact, that is the system in most countries (e.g., from Europe).
It is motivated by a different philosophy: Because the schools are public, and thus paid by the tax payer, everybody (having the necessary qualification) can start any programme.
The decision who is good enough for the program is based on the success *during* the program, not based on some strange number in a strange test. In top schools (such as, e.g., ETH), 50% of the students have to leave after the first year and another 50% after the second year.
The downside: You can't charge a lot of money to the students like in US private schools.
Perhaps it's time to change the system?
That said, I am very glad for the admission standards here in the US, because that way I deal mostly with super excellent students. I think we should do both here.
ReplyDeleteThere is a collective action problem here, Luis. Lowering your grade scale without others doing it accomplishes little in terms of reducing grade inflation, but does cost you on your teacher evaluations.
ReplyDeleteA major cause of grade inflation seems to be this incentive. Teachers are evaluated by students. Students like getting high grades. A teacher that gives all high grades is likely to get a better evaluation.
Grade inflation has gotten so bad that more than half of grades at Harvard in 2007 were in the A range. The average GPA across US private schools was 3.3 in 2006.
However, as much as I think this battle needs to be fought, fighting it alone is unlikely to work. You'll just be seen as unfair for failing to conform to the norm. Changing the norm requires a broader effort.
I've noticed that I haven't met many PhD students that received their undergraduate education at top American CS schools. I suspect that these students are already getting burnt out from their undergraduate education, and have little desire to stay in school as a result. Failing more students might make this even worse by adding extra stress *even for the good students*.
ReplyDeleteI agree with Greg's point.
ReplyDeleteOne thing, however, is the usage of the term "unfair" -- you can be fair (as I think I am) and still be strict with your grades.
I've noticed that I haven't met many PhD students that received their undergraduate education at top American CS schools. I suspect that these students are already getting burnt out from their undergraduate education, and have little desire to stay in school as a result.
ReplyDeleteThat's just not true. For example, by looking at our applications for the PhD program and our enrollments, a very healthy fraction of the students are from "top American CS schools." It's true that it's less than 50%, but the US only has about 5% of the world's population. Given that in the PhD program (at least here) we want to have the best people in the *world*, I wouldn't expect the US to have more than 50% of the students. In terms of just American schools, I don't have the numbers in front of me, but I would bet that more than 50% of the people who are admitted from American undergraduate programs and enroll in our PhD come from a top 20 CS undergrad school.
"That said, I am very glad for the admission standards here in the US, because that way I deal mostly with super excellent students. I think we should do both here"
ReplyDeleteDoes super excellent and getting a fail grade do not contradict each other????
I said mostly, not 100%.
ReplyDeleteWhat makes you think that failing students will increase their quality?
ReplyDeleteIf you keep failing them, they will only try to get a grade that will satisfy "you", they will not study because they like the lecture or they want to learn.
I think teaching the importance of the lecture and showing them how fun it is, is much more motivating then any other violent method.
I bet you are who you are because you knew how to love what you do. Show them how you did it instead of forcing them to study. Your lectures must be like your games. "Making people do it voluntarily"
Oh I'm not trying to use failing as a motivation. I motivate through other ways. I'm just trying to use failing to get rid of the ones who are probably never going to do well.
ReplyDeleteIronically, the two best programmers I know dropped out of undergraduate programs much less rigorous than CMU's. If you're trying to train good programmers for professional programming jobs after college, you need a different paradigm than classroom lectures plus homework. CMU's graduates are in a hole compared to more practically oriented programs, like Waterloo's.
ReplyDeleteWhile I was a prof at CMU in the 1990s, median graduation time went from 4 to 5 years in engineering, because everyone took Luis's view of "I'll make my class really hard". Remember that yours isn't the only class someone's taking! Obviously students are going to take these factors into account unfavorably when choosing schools.
CMU's in an odd position in that it's highly ranked in CS, but is not a household name outside of Western Pennsylvania. As a result of people's brand consciousness in wanting to be from a place perceived as the "best" (as well as CMU's rather limited size and breadth overall), CMU has a tough time competing with places like MIT and Stanford or even Harvard for the "top" undergrad CS majors.
The non-evil approach is to figure out how to help the students so that they all succeed. Of course, it's easier to just kick out the ones who can't take care of themselves on the first go.
It's a disservice to fool someone into believing they can do something when they just can't or, more likely, won't. CMU should be a place that serves people who truly love the material, not a programmer farm or job launch pad. The issue isn't with professors failing students, but with society taking such issue with failure in general.
ReplyDelete"society taking such issue with failure in general."
ReplyDeleteAnd society is not kind at all.
Just my $.02
ReplyDeleteI personally am not fundamentally opposed to the idea of you failing more students to raise the overall quality of the undergraduate program at CMU. Before turning the thumbscrews though, you might want to make sure that the following is accounted for:
Assuming that failing more people will weed out the less worthy students implies that all the students enter your class with equal preparation(and hence only those with the ability and drive to be great Computer Scientists should pass). Unfortunately this is not always the case. Those students from magnet/private schools have a significant advantage, particularly in a freshman course like 251.
Of course this kind thing is the probably the department/universities responsibility it's worth considering. Certainly it is the responsibility of the lesser prepared students to catch up on material they missed in High School but it is difficult to face challenging new material while still trying to learn what one has missed from High School. This particularly tricky when many of these students are trying to catch up on the background material in multiple courses. It might be something to compensate for when you decide to turn up the heat.
coming from a system where failing 50% of the students after the first year is normal, I can say that failing a student is sometimes the best option for the school AND the student.
ReplyDeletevery often students realize that they chose the wrong major. If you didn't fail them, they will realize that much later when it's much worse, usually.
last thing: with every pass/fail system it is important that the evaluation is fair (cheating is nearly impossible) and that it is possible to repeat (at least once) a failed class/year.
Mr. Carpenter,
ReplyDeleteWhere exactly were you a professor? Of course everybody has examples of people that have dropped out of college that have done very well (e.g., Bill Gates). But I will not agree with you that college is useless or that our *computer science* graduates are bad programmers. For example, as of 2 years ago, Google had hired more people out of CMU than any other school (and that included Stanford).
Luis your blog has become very famous.. Now it seems you donot need anyone from a "Third World Country" to write posts for you.
ReplyDeleteIs the old offer still open???
If you don't allow yourself to fail students (as a university), then to maintain your reputation as a high-quality school, you must reduce the acceptance rate of high-risk pupils. So no little kiddies from the Guatemalean slums. Who knows, you may be passing up some real gems there.
ReplyDeleteWoah, what a coincidence. I'm from Guatemala!
ReplyDeleteAssuming that failing more people will weed out the less worthy students implies that all the students enter your class with equal preparation(and hence only those with the ability and drive to be great Computer Scientists should pass). Unfortunately this is not always the case. Those students from magnet/private schools have a significant advantage, particularly in a freshman course like 251.
ReplyDeleteYou're actually so wrong that I'm going to say it like this:
wwrrrooooooooooooooooooooooooooonnnnnnnnnnnnnnnnggggggggggggg
kids at CMU from public schools are here because they really care about computer science. Lower standards at home meant that kids who didn't care about CS didn't pursue it later. The problem is with the private school kids who are trained to "only accept best." Coincidence: they're also the ones whining about bad grades.
"So no little kiddies from the Guatemalean slums."
ReplyDeleteWhy the author is using derogatory language on the blog.
Just as a complete side note question: I've noticed that you are giving a talk about teaching students at 251 at the same time when the 251 lecture is happening. Not only that, to prevent students from coming to your talk instead of 251 lecture, we have a quiz on that day. Was this intentional?
ReplyDeleteI'm not opposed to failing people in CS programs in general, but I think 251 is the wrong place to do it for several reasons:
ReplyDelete- As several cogsci majors have pointed out, it has cogsci majors in it, and you don't want to apply the same CS rigor to cogsci majors as you do to CS majors.*
- Frankly, I don't think being good at 251 is that correlated with being good at anything else in CS. There is frequently not enough coverage of each section of the course (each of which, taught properly, is its own course at minimum) to give students a complete idea of what one can and cannot say in a given proof, and so scores on the homework are often more correlated with empathy with the TAs than with being good at CS (Or, if you go to office hours, I'm told, they tell you the answer).
I think OS is a much better course for this kind of thing, although obviously it tests only practical programming ability and not theoretical understanding. It does, at least, correlate pretty well (as employers have found) with being a good programmer.
Perhaps one day CS could have a few tracks, each of which has its own filter course that's supposed to fail people; an implementation-focused track might have OS as its filter, while a theory track might have a souped-up 451 or similar.
*(So you could have an honors version of the class for CS majors, or something, and the cogsci majors can take the non-honors version, or whatever, if you still wanted to do this in 251.)
"so scores on the homework are often more correlated with empathy with the TAs than with being good at CS (Or, if you go to office hours, I'm told, they tell you the answer)."
ReplyDeleteThis is a very serious accusation. If this is true than higher authorities should look into the matter.
No no calm down. By empathy with the TAs, I meant knowing what they were going to be anal about and what they were going to be very handwavy about when grading. By telling you the answer, I didn't mean actually telling you the answer, I meant they just made it much easier for you, and made it (presumably) much clearer than the question makes it what's being asked.
ReplyDeleteThis is one area where OS beats 251 hands down. In OS, the TAs are instructed to say two things: "Read the spec!" and "It's a design decision!"
This is because the spec is always completely insanely 100% bulletproof, very much unlike 251 homework questions.
I'll clarify further, for fear of being misinterpreted. I am thoroughly convinced that no past or present 251 staff have ever done anything remotely dishonest or unfair or anything like that.
ReplyDeleteI'm saying that the nature of 251 and the cursory coverage of individual topics sometimes makes it hard for students to figure out what is required, and this makes 251 grades a less useful CS-student evaluation tool.
ReplyDeleteYou're actually so wrong that I'm going to say it like this:
wwrrrooooooooooooooooooooooooooonnnnnnnnnnnnnnnnggggggggggggg
Bravo for lowering the bar on an otherwise civilized conversation.
kids at CMU from public schools are here because they really care about computer science. Lower standards at home meant that kids who didn't care about CS didn't pursue it later.
I never claimed that such students didn't care, or that they are not motivated. Rather, that such students may be spending more time on {calculus, physics, 251, etc,} to compensate for weaker backgrounds. This leaves them less time than their better prepared counterparts for any particular course. Suggesting that everyone who enters a computer science program from public school is interested and hence a self taught to a level comparable to that of a private/magnet school is blatantly false. Consider that there are some students who may have developed an interest in computer science later in their career in high school, or those who didn't have access to reliable broadband internet and other "tools of the trade" Some do, some don't, but those who didn't have access to excellent preparation shouldn't be penalized excessively provided they show sufficient aptitude and motivation to learn once they are in the university.
The problem is with the private school kids who are trained to "only accept best." Coincidence: they're also the ones whining about bad grades.
This is possibly true but unrelated to the point I was trying to get across. It was: It is possible to have two equally bright, motivated people with different amounts of exposure to background knowledge and material presented in a course like 251. Hence the university and individual professors should probably ensure that there are sufficiently many tracks to compensate for these differences before merely increasing the difficulty curve or failing percentage.
Calm down peeps :)
ReplyDeleteFirst, let's not make this about public versus private schools.
Second, I actually agree with most of the things that people have been saying here (on both sides of the question).
In reality, I do not plan to fail any more students than I already do (which is approximately 3% of the class and which is exactly the fraction of student that pretty much do no work throughout the semester). My original post was just to say that "sometimes I feel like I should fail more of them."
We work very hard to make 251 as useful and as fair of a course as possible, and the feedback we get is overwhelmingly positive, *especially* after the students have gone on to either grad school or their jobs.
For example, when you interview for jobs at Google/Microsoft/Facebook the questions they ask are EXACTLY the types of questions we ask in 251 homework. This is because these companies actually believe that ability in such problems is highly correlated with ability to succeed in their jobs.
Hmmm. Made a post and then lost it. It was too long anyway.
ReplyDeleteBack in the day when I was an undergrad, (92-96) I had many friends who were CS. This was still when it was a Math/CS hybrid, and basically a 5yr program, billed as 4yr.
Admitting MORE students simply to FAIL more students is no better than admitting SOME students and then forcing them to take an extra year to finish.
A year's tuition gone. Some students have the luxury of being able to suck it up financially, do the loan, and move on. Others do not, and they are THE kid who is going to University in the family.
I am no fan of grade inflation, I think it perpetuates the credentialism cycle that we have going in this country. If a student is doing the work, and isn't getting it, I think a poor grade will suffice. Outright failing should be reserved for people who aren't bothering to do the work at all.
-Charlotte
"If a student is doing the work, and isn't getting it, I think a poor grade will suffice. Outright failing should be reserved for people who aren't bothering to do the work at all."
ReplyDeleteYes.
And I'm not entirely familiar, are (poor) teacher evaluations weighted more heavily than say a packet of strong comments saying students learn a lot etc etc? This is relevant to the idea that grade inflation is partially due to the teacher evaluation feedback loop.
And I'm not entirely familiar, are (poor) teacher evaluations weighted more heavily than say a packet of strong comments saying students learn a lot etc etc? This is relevant to the idea that grade inflation is partially due to the teacher evaluation feedback loop.
ReplyDeleteThis is a very good question. The answer really depends on the school, and on the type of professor. Some professors are "tenure track" and others are "teaching track". For "teaching faculty" I assume teaching evaluations matter a lot. For "tenure track," (like me) the unfortunate truth is that what matters the most is research, not teaching.
I agree with this, but there's also something I find that's been irking me.
ReplyDeleteOur 15-251 class is basically very, very dependent on a curve. As such, at some point, failing more students will be "unfair" to the amount of effort that they work.
Of course, right now as it stands, the 15-251 curve is working sorta okay.
As for failing more students to increase the standard of graduates, it will work, obviously. It's just that sometimes the class isn't ultimately representative of any one's true potential. They might end up doing bad in 15-251 since they haven't taken enough math topics (which really helps a lot), etc. etc.
As for the public/private school argument, it is absurd what actually matters. The US education system is so messed up, that say going to like TJ or Stuy (the only two top magnets which don't have insanely low student populations) end up have such a disadvantage in college admissions. In the end, college ends up as a tool for trying to even people out ...
As for grade inflation, please don't cite Harvard. Many people always claim "grade inflation" at top schools, but it's more so a byproduct of students working hard. They probably deserve the grade.
As for CMU's undergrad CS program, if you compare the courses at MIT and CMU, you would realize that CMU's CS is highly more practical in the real world setting and MIT's CS is more theoretical. Look at the content of the classes~ So, don't go there either ...
Anyway, more responses later ... my laptop is dying out~
One thing I should clarify about the 15-251 curve: we never lower your grade based on the curve; we only increase it. At first, the cutoffs are 90, 80, 70 and 60. So, if you get above a 90, you get an A. Guaranteed. However, because so few people get above 90s, we "curve up" making the cutoffs be lower than 90, 80, 70, and 60. So, it's completely conceivable (though unlikely) that the whole class could get an A.
ReplyDeleteWell, most of what I can remember from courses in which I was not interested, and in which I had to work a lot, is just that I did not sleep much during some days.
ReplyDeleteSo, assuming the students who are not interested are going to forget most of it anyway, if what you want is to make your students better, one thing you can do is just teach more advanced things, and then have in the exam/assignments an easier part that makes almost all student pass.
By the way, really nice comment system.
I would like to stress again the difference between background preparation and aptitudes.
ReplyDeleteI came from a relatively poor public school in a rural area. Most of my classes were taught by football coaches, and advanced tracks in mathematics were few and far between. I made it to this school through dedication and willpower alone, and I was one of the first in my family to go to a university.
When I came to this university, I had never done, nor even seen, a formal proof.I thought I wanted to do CS because I liked to play around with computers. I had no idea it would have all of this math. 21-127 came as a massive shock to me, as I was thrown in with little understanding of any of these mathematical concepts. I had to struggle at every point, and I had to make great efforts to accommodate myself to the way of thinking that was expected here. I was constantly frustrated by my privileged peers who wrote me off as stupid and ignorant, and who groaned about the triviality of having to learn things that they had been doing since middle school.
However, when I look back at how I thought and acted just those few months ago, and I compare my knowledge then and my knowledge now, I barely recognize myself. I worked extremely hard, and I feel like I'm finally catching up with everyone else. It is true that I don't immediately recognize esoteric mathematical concepts that I have never seen before, but after I learn these concepts, I understand them.
What I am trying to get at is that broad introductory classes should be made to equalize students and order them based on APTITUDE rather than EXPERIENCE. Your "best" students freshman year may not be your "best" students graduating year. It is true that some of us will never be able to catch up to our pampered, well-prepared peers, but you must at least give us the chance.
That student struggling to get by with a C might come out of the class to do great things; so please, don't lower the cutoffs or make things impossibly difficult just because you feel that you must "filter" us to find the "best" students; you might be snuffing out an opportunity. Let the classes later on filter us out, once we have had the time to catch up.
Luis, I know you are the master of human computation, but the amount of collective thought that has gone into this comment thread shows off how good you are at getting the world to think for you!
ReplyDeleteJust to pick a nit with your curve, the fact that you have a curve in the first place suggests that you feel your absolute thresholds may be unreasonably high. Why should you have a curve at all if you believe your grades reflect competence in an absolute sense?
Put another way, is your grading on a curve an admission that you don't know how to identify competence in absolute terms, and thus fall back on letting students compete with one another for scarce resources? Perhaps that is good preparation for some aspects of the real world.
I think it's more of the default thing. Most classes grade on 90, 80, 70, 60, unless scores are too low.
ReplyDeleteIn this case, I think it allows TAs to make tests with a variety questions.
That, and I heard that Von Ahn likes to see the mean, median around 50, so it looks like a proper bell curve instead of say ... 90 80 70 60 etc. etc.
And to the other anonymous, like I said ... college is to even out the US education system up ...
~Anonymous of 7
I personally enjoy seeing that some people feel the only people who are worried about or upset about the issue of 251 and difficulty are the spoiled brats who whine for As.
ReplyDeleteLet's consider those people who might not have a good background in CS or, more relevant to 251, upper-level mathematics. Someone even pointed out that they may be from lower income families.
Hello. I am one of those people without higher-level mathematics and someone from a not-so-well-off family. I had another sibling go to college, but she went to Slippery Rock University, a college where taking out loans to cover the cost of tuition is actually feasible (I'm looking at you, CMU).
I received a large scholarship from my high school and a moderately sized one from another institution. This sounds wonderful, right? The catch? I lose both if I don't have a 3.0 at the end of the semester. The brutal reality? I can't afford to go to Carnegie Mellon without them. Period. 3.0 isn't terribly high (especially if the national average is 3.33) but if you decide that more students should have lower grades, maybe I won't be so lucky.
I can honestly say that I have given everything I possibly can to your class this semester. Unfortunately, I can't ignore my other classes either. Also unfortunately, your class counts as more than any other class I'm taking towards my GPA, and even if I am trying and learning, certain people won't see it as such if I don't produce. I don't feel I'm among the best of your CS students, but I'm sure trying. The fact that I constantly feel like it won't ever be enough is disheartening. I think CMU has the best CS program in the country. That's why I came here. I didn't find out I loved it until the summer between my Junior and Senior high school years.
Without being a total sob story (I really hope it doesn't come across as such), I am currently sitting pretty hundredths of a point above the magic marker. Furthermore, I'm giving everything I can to this class while only giving what I can to just squeak by in my other classes. Apparently, under this new system of thinking, I should head somewhere else.
Grade inflation is bad, but a change in policy like this may have real consequences for some people, and in some cases, I think it just might be unfair.
Thank you for the post. I hope you do well in 251! (Truly.)
ReplyDelete"I sometimes want to fail more people in my classes. This is not because I am evil (although some people here seem to think so), but because I want the people who graduate from our computer science program to be truly the best in the world."
ReplyDeleteThis statement to me seems to be composed of a pair of nonsequitor sentences. In order to make the general student pool better, you would like to throw out possibly promising students, rather than... making those students better.
There is no oracle certainly. No one knows how well a student will do or how serious they may be, or why they succeed or don't. If I feel a student should receive a failing a grade, it is not because I want to make the pool of students better, but because that student entirely failed to master any of the material to my satisfaction.
To me a student failing should be as rare as possible. I certainly can't control everything, however if a student is capable of learning the material, they should be able to learn it at least well enough to pass. If not then there is some deficiency in my teaching. If I lecture to a room of 500 students who, all things being equal, should be capable of doing the work and diligently apply themselves, if any of them do not it is a reflection on my inability to teach those students.
No group of students is equally and entirely capable of passing a class, and outside circumstances always take their toll... however... to me the distribution of grades in a class equally weights the abilities and performance of the students as well as the ability of the teacher.
So, to me, saying you wish to fail more students is tantamount to saying either you want to be a worse teacher, or that you want to teach less (i.e. only cover very difficult topics very quickly, and not worry about it since only the very good students will understand enough to pass).
If a student is capable of fully mastering the syllabus of topics excellently, they should merit an A. If the admissions process is so stringent that all of your students are just that good, then maybe your admissions standards need to be relaxed so you can see a wider spectrum of students.
I would however question your motives in increasing the number of failures. If you feel your students should be better, add or tweak the syllabus, or bring up a retuning of the curriculum. Simply increasing the number of failures means that only the students that are prone to do well will do well, and that you are in effect improving your program not by improving the students, but by dropping those who don't already have an intuitive grasp of the subject before they set foot in the class. These are the students that need your lecture the least.
If you want to improve the students, improve the students; don't just throw away the ones to whom the material doesn't come naturally.
Dear Anonymous,
ReplyDeleteWhile I see your point, I simply disagree with it. I want to make the students better through other means. But, I also think that, in a #1 ranked program, it's ok to remove some of the ones that are not #1 material. In a similar way, if I tried to join the US Olympic team in swimming, I'd quickly be thrown out even if my trainer was the best trainer in the world, and even if I worked my butt off.
By concentrating energy on the ones that *are* #1 material, you can teach them more things.
I understand this is a very elitist view, but that's the whole idea of the top-ranked US university programs.
How do you classify a university (for undergrad studies)?? I mean by jobs or by the fact that how many people go on to be successful later in there life etc.. I can see how they rank graduate programs (by research) but I see no criteria for undergrads.
ReplyDeleteIf its job then 251 is useless. If its mathematical prowess then too 251 is useless. If its something in between then 251 is awesome.
According to me undergrad programs should be ranked by overall achievements (jobs, service to the nation and community etc..). If thats the case then ranking programs (undergrad) by majors is wrong. The university should be ranked not its majors. And if that is the case than you are not helping the cause by making students spend all their time on a single course only.
I strongly believe that working hard on 251 makes you better at your job *and* at mathematical prowess. I'm not going to make 251 easy so you can concentrate on your wine tasting class. Go to a liberal arts school if that's what you want.
ReplyDelete"I'm not going to make 251 easy so you can concentrate on your wine tasting class."
ReplyDeleteIf that is the case than make the homework tough. Solving problems is better than wine tasting. I was thinking of extra stuff like taking other advanced courses.
Just wondering, how did CMU become a top ranked school? And who has the responsibility to maintain the rank? Individual professors?
ReplyDelete"And who has the responsibility to maintain the rank? Individual professors?"
ReplyDeleteProfessors have a very big part in maintaining the high standards of a university. As far as I know rankings are dependent on the perception of a university so if there is a famous prof (like Luis :) )we get more points.
I think failing a large number of people is probably too much of an all-or-nothing outcome: there are more flexible, fair options (see below). I'm not a CMU student (I'm a grad student at another top CS department), so I can't speak to the particulars of your class or the CMU program, but you should be cognizant of the facts that 1) students are taking multiple classes and have other (class-related) demands on their time, 2) some students will be taking the class because they want to learning the material (or need to in the case of cogsci majors) but are not CS majors and might not have the incentive to become super-stars in the subject, 3) in an intro class students will have widely different backgrounds, and 4) for 1 reason or another otherwise-good students will occasionally have a bad day and do poorly on an exam or problem set. You should try to minimize the number of failures due to these sorts of "legitimate" reasons for doing less-than-stellar.
ReplyDeleteMoreover, at a place like CMU, most students are likely to be quite good. If you were teaching at the University of Nowhere it might be reasonable to fail a large percentage of the class, but if you're starting with smart students, failing a large number of them reflects far worse on you as a teacher than it does on them. And later in life they will be judged by their grades by people unfamiliar with your specific class, and you will be hurting them by giving them failing grades in fundamental classes. In my experience as a TA, there generally are a couple of students who deserve to fail (and unfortunately, often don't), but it's usually not many.
The solution, I think, is a combination of low raw scores/high grades and non-linear grading curves. Giving hard assignments with low raw scores emphasizes that they don't understand everything, incentivizes them to work hard, and makes them uncertain as to their final grades but then making the final curve more forgiving ensures they're not bitter and aren't punished for taking a challenging class. Some of the best classes I've had have used this technique.
By non-linear grading curves, I mean, for example, making it relatively easy to get a B but very hard to get an A. By making the grade cutoffs non-linear, you can tweak the parameters to simultaneously satisfy sometimes-conflicting objectives like punishing students who don't learn the fundamentals without hurting people who had a bad day or 2, recognizing that most students did fairly well by giving them a decent but not great grade, and ensuring that only the best get rewarded with an A.
Judging the fail scale is one of the hardest things to do for any class. For a class like 15-251, which is both a requirement for a variety of courses but also is expected to be one of the early capstones for Computer Science, it's one of the hardest scales to gauge period. This is certainly not due to the staff caring only about the academic material. Indeed, were that the only factor, there wouldn't be problems like this at all.
ReplyDeleteI might be mean for saying this, but working -hard- is not the single item in the formula for passing any class. This might have been true in high school, but that naive thought will not last through college. No; college teaches you what is essential for the rest of your life - not just working -hard-, but also working -smart-. Acknowledging the necessity to work with others; getting the guts to ask for help in order to preclude slamming one's head against the wall for extended amounts of time; finding two, three or even more ways to tackle a problem, whether it be literally in homework or otherwise in managing life and time; learning many, _many_ subtle things that you certainly won't regret learning as you tackle challenges in the future.
15-251 isn't just a math course out to devour little children. It is a brutal but essential learning experience for the computer science freshmen who are set to take it in the semester. It challenges its students with overwhelming obstacles at times, pondering how they respond - and though one can respond with simple academic superiority, there are many other paths one can attempt to walk to obtain understanding. Collaboration rules are mentioned because collaboration is -expected-, so a tight leash is placed on it as a reminder. Office hours are regularly placed on a near-daily basis.
There are many, many factors in this that make it hard to gauge who out of the students one should outright fail - and it'll usually just come down to the same paradigm that most other classes would use without a thought. Indeed, it is certainly possible to see people who struggle at the beginning, but then find a routine that works for them - though unfortunately, also possible to see people who cruise well at the beginning, but take a dive towards the end. The correct intent behind Luis's wish, I believe is not one that can quite be measured by just staring at numbers. Someone may have just 'scraped past' with a C because they didn't try hard enough - or they might have 'scraped past' because they came in with little more than geometry education and worked hard (or intelligently) to learn more than anyone else could hope to learn from such a class. We want to narrow our eyes in a squint at the former, and pat the shoulder of the latter in encouragement - but it is often too hard to tell one from the other.
I believe that the curve system used right now is correct for the class - it inherently recognizes the difficulty of the class, and gives students hope even after the fire and brimstone - because they know the curve will serve to try to help them out, as long as they keep on trying to improve. The myriad of mixed signals attached to different people getting the same 'number grade' makes it impossible to quantify things about letter grades - you can simultaneously hear students griping about getting that marginal B in 15-251 next to those who are almost tearfully joyous that they passed the class with a C, and would not have to subject themselves to it again. And then again, there's the blank slate under there where the answer to 'did they just want to get the class over with and never think of it again, or did they actually learn something that would stick?' is.
I do not think more people should be failed - at least, if the class is staying the way as it is. It'll heighten the bar, but not improve how people regard the class - just how much daunting the goal is. Your goal should not be to fail more people, but to fail more intelligently. To actually catch the people who you mean to fail.
I dropped 251 today.
ReplyDeleteHSS, here I come.
The ends don't justify the means.
ReplyDeleteIf you want top caliber students to graduate there are 2 ways:
1. fail more
2. teach better
When I agree to pay nearly 50 grand to hear you speak, the agreement is that you will try your best to make me a CMU caliber computer scientist, not that you'll see if I somehow already am one. Likewise, its my job to try my hardest to do well. If a student fails, you should consider it a failure for your self in getting the student engaged and understanding.
Lets say that your salary was determined by the amount of students that do well. If that were the case and your ass were on the line, I'm sure you would try a lot harder to engage the students and many more would become CMU caliber computer scientists. How about that system?
I think the 50 grand I paid for the semester I took your class was more than enough for you to reach out and help me do well, rather than fail me. Try to realize the sacrifices people make to even take your class. CMU should make brilliant computer scientists, not just find them.
ps. i thought you were a great teacher when i took 251 and don't think you need to change your ways. I just disagree with the fail more perspective
pps. i'm from a south american country and our systems do tend to weed out more people, fail more, and give teachers a great deal of freedom. then again, there are many reasons, and perhaps this is one of them, that our countries aren't doing so well.
You're not just gracing us with your presence, we're paying for you to teach. Maybe if CMU were free, or even had a decent endowment, and could afford to select only the best (assuming that they wouldn't choose another school despite 'the great luis von ahn' being here), you could be as elitist as you apparently think you deserve to be. But (extremely high) tuition puts funds towards your research and food on your table. Its give and take.
ReplyDeleteSome, and in my opinion better, teachers stay true to the objective of teaching: That you should seek to impart your knowledge to every student to improve their lives and standings.
Some students deserve to fail but failing more students for the sake of taking the cream of the crop is a cop out to any teacher who teaches for the right reasons and actually wants to make CMU a top university. Any teacher, any school, can pass only the best students, but the best teachers (and i think you are one of them) can change students to make them "the best."
I totally agree with these last two comments. It is my job to be the best teacher I can be.
ReplyDeleteOne minor comment: although you think that your tuition pays for my salary and my research, that happens to mostly not be the case. It pays for none of my research and only about 20% of my salary.
I believe that to be a good department, you need to create an open and nourishing environment where students can be wildly creative. This will allow your current students to maximize their ability and it will also attract good students.
ReplyDeleteThe system of "admit everyone and fail the bottom 90%" works in some ways, but in my experience, it certainly doesn't encourage creativity. What creativity, when you're under pressure to trudge through the mechanical details of syllabi? Look at the countries/universities who employ/have employed that system of education, and you will see highly skilled graduates who aren't on the map in terms of innovation and impact.
Finally, the curriculum is hardly a thing that can heavily and positively influence the reputation<->good students feedback cycle. Reputation is in (large) part due to smart political thinking, good students follow good reputation, and the curriculum is one tool among many to help these good students reach their potentials.
So what you're proposing is to make the curriculum more of a bottleneck, and I think that misses a bunch of points.
Prof. Von Ahn,
ReplyDeleteNot having read the entire comment thread, I am sorry if this has already been addressed. However, I do have to ask:
Are grades really a good metric for the quality of a student?
Or, are they a better metric for the work environment of the student?
Alternatively, if I decide to sacrifice to some degree my grades so that I may pursue some of the other opportunities that CMU offers me (extra-curricular, research, or otherwise), does that make me a student of lower quality?
I have always been of the opinion that there is a point of diminishing returns when comparing the work required to achieve a certain grade compared to what I learn. While I might be able to get to increase a B to an A by spending more time on homework or studying more for exams, I would argue that the additional time is better spent on taking advantage of other opportunities.
In the end, time scheduling is an optimization problem. And so, I would worry that failing more students would end up failing those students who care more about learning than their grades. A heavy emphasis on grades has a cultural trade-off: is the "4.0 student" the one you want working on your research project? Or, the student who is passionate about research and learning?
"although you think that your tuition pays for my salary and my research, that happens to mostly not be the case. It pays for none of my research and only about 20% of my salary."
ReplyDeleteIt's not quite so easy to break it all up I'd guess, since your other funding sources are causally related to these students and their success.
I graduated from university where the computer science department was dieing and I felt that maybe on a few occasions the professors let slide some of the lowest performing students. I felt that the professors were excellent and the course material to be challenging. It did bother me that other students received a pass, because I felt this diminished the value of my own success.
ReplyDeleteYou give the example of how in foreign countries many students do not pass classes. In most foreign countries students tend to go to public universities that are virtually free. The government only wants to pay for the best students. In the US where students frequently pay for a large share of their education especially at a private University like CMU it would be unfair to fail more students.
ReplyDeleteIf students lost their scholarships because they failed a class or had a lower grade that would be a different story. As the system is setup today when students pay 50k+ a year, so the school is under some pressure to make sure that students graduate. Otherwise who would be willing to spend 50K+ a year to go to a school from which they were unlikely to graduate. If CMU was free and only 10% of students graduated that would mean the incentives would be aligned better, but to pay 200K for 4 years and stand a chance of not graduating would be stupid.
Dear Dr. Luis von Ahn,
ReplyDeleteI firmly believe that a successful professor is not the one who fails his students, but rather the one helps his students and directs them toward what they are best at. I think that only the hard working student must pass a course regardless how well or bad he/she did in the course compared to your standards, by giving the students only the appropriate grades they truely deserve.
I am wondering if I should be quoting Kierkegaard here: "The path must be made difficult, for only the difficult inspires the noble-hearted". I agree that a Computer Science major must be tough and hard, but I am personally against the policy of failing students because one student could have knowledge and skills that differ in creativity from one sub-field of computation to another. Again I think that a course mentor must assist his students to find their potential and their weak points in the course, because when a student recognises his/her weaknesses and continues to work harder on them then those weaknesses shall transform into points of strength.
By doing so, not only students get to learn more about the course and work on their weaknesses but also they develop a state of mind and confidence in working harder and challenging their peers, as well as trying to satisfy their professor standards which at first they couldnt satisfy.
Computer Science is my major of study. I am also considering to continue my higher studies in the near future in institutions as successful as Carneige Mellon University. I know very well the meaning of the word "success", and one can never be successful without having failed many times. During my college years I failed several courses, but that never stopped me from continuing the challenge and graduating with a high GPA. It was not because I failed a few courses I decided to continue the challenge, rather it was because I knew that deep in my heart I had passion for computing. I had the fire in my belly and I knew that I can be better than my peers and can exceed the expectations and the "predefined" standards of my professors. I knew I could give more.
I am against your policy of failing students rather than giving them the grades they deserve. What you are trying to achieve could not be done your way.
I think you want to fail people because you are actually a pathetic Computer Science prof that is trying to exact vengeance on students for the educational injustices done to you earlier in life. If it were up to me, I would fail YOU right now. Instead of being a true professor and inspiring students, you just want to fail them to artificially "raise" the quality of the CS program.
ReplyDeleteWell, I'm glad I get the CMU award for teaching excellence :)
ReplyDeleteHere's a CS freshman, Fall 2009.
ReplyDeleteTry me, Dr. von Ahn? :)
Well I'm late to this conversation, but I found the thread truly interesting. I don't think failing more students is necessarily a bad thing - if the student truly does not understand the material after exhausting all options then to pass them would be a disservice to both the university and the student. However, if a student is failing I think it's your responsibility (as others have stated) to ensure that you are providing the best possible resources for them and pushing them to succeed.
ReplyDeleteStudents essentially go 30 years into debt to complete the cs program at CMU, that is likely more years than they have even been alive at the point they enter your class. That isn't to say that they should get a free ride cause they got in and paid a huge fee, but as with any service that you pay tons of money for I would assume they all expect to get the best possible education you can provide- not to be written off as a lost cause or worthless cause they aren't the best of the best. The grading scale is what should be used to show the difference in their skill- Just cause D is for diploma doesn't mean it looks too great on it ;).
I've personally loved having you as a professor (I took a formal languages,automata, and computation course that you co-taught several years ago), however many other professors seemed more interested in their research than the students and it always made me wonder why those people even bothered to teach (I assume they are essentially required). Seems to me to be a great disservice to future generations if you are just going through the motions but have no heart in what you are teaching. So maybe while you are failing the students who aren't up to snuff you can fire the profs who don't honestly care to truly TEACH.
G'day Luis
ReplyDeleteWhat was your final thoughts on this matter? Did the pies influence your thinking?
Maybe you should ask Vineet Nayar (CEO of the Indian outsourcing giant HCL Technologies). Just saw a Slashdot article today where Naya stated that most American college grads are "unemployable." :P
Cheers,
Scott.
P.S. Sorry I'm a bit late to the party on this one. Only just discovered your blog and am adding it to my aggregator of choice :D
I WISH I COULD AFFORD TO GO TO CMU!!! : (
ReplyDeletebad idea. your job is not to dictate your students future, but to simply shed some more light on their path.
ReplyDeleteThe problem with this society and your post is the focus is off. Weeding out, only the best, fail more, etc. Where is the compassion? Where is the zeal for teaching, for reaching people who likely took your course because they had an interest in the topic or in the field in general. Not everyone learns the same way and not everyone conveys what they have learned in the same way. You come across as pompous and not compassionate. If you are truly a 'teacher' you should hold this position with pride and honor. It is your job to help people along, even the ones that aren't the best.
ReplyDelete