On the positive side, I think such questions were a big waste of our time so the existence of search engines is (hopefully) making middle school teachers ask questions that require a little more thought.
On the negative side, search engines are adding more work to me! When I teach 15-251 (CMU's version of discrete math for computer science), most of the classic problems (and their solutions!) can be easily found on the Web. I'm always torn as to what strategy to follow to deal with this issue:
- Ignore The Whole Thing and Do Nothing. Pros: It's easy. Cons: Kids don't learn how to actually solve the problems if they just Google for the answers.
- Police State (my current approach). Make searching for answers on the Web be considered cheating in the class and punish them harshly if they cheat. Every year, we set up "Google Traps," in which we assign a problem with a unique name like "Giramacristo's Puzzle." We then publish a Web site that has a solution to "Giramacristo's Puzzle" and make sure it's the first result in Google for that query. Since we control the Web site, we can record all IP addresses that visit it and later correlate them with students in the class. We catch approximately 10% of the students in the class cheating. Pros: It's fun! (for me), and if you do it early in the semester they learn never to search for answers again. Cons: It requires effort. In addition, it's not clear that disallowing search engines is good preparation for life -- after all, they'll be able to use the Web when they're out of school.
- Allow Searching on the Web but Change the Problems. Pros: In real life they will be able to use Google. Cons: It's hard to come up with good ways to change the problems, and inventing brand new problems every year is even harder, especially if you want them to be as good as the classics. My advisor Manuel Blum has recently been thinking deeply about this and he told me a good strategy: for most problems (at least in theoretical CS), you can change them significantly by thinking "how can I make this problem be closer to reality?"
Your police state approach is famous, though I'm sure some students already know about it and can guard against it. Moreover, Twitter is probably your enemy, as your students compare notes.
ReplyDeletePerhaps a better approach is to come up with a means of generating unique problems (perhaps through random perturbation / combination of standard ones) that test the material? You could even test to verify that their answers are not easily obtained through web search.
Why not just make homework problems worth nothing and have the course mark come entirely from in-class examinations? As an undergraduate, I can say that this has always been my preference for math-type courses. The homework is optional - if I do it it is not because it is required for marks, but because I need to make sure I understand the material well enough to perform well on the tests. Required homework is just an annoying form of hand-holding.
ReplyDeleteYour police state approach is the first thing I ever heard about 251 before taking it.
ReplyDeleteA crackpot approach heh: Use "reverse SOI" and blow up pagerank for all the pages that DON'T have anything to do with the problem when you do a common search for it. Students get discouraged when an answer isn't on the first or second page of a search.
Why not just make homework problems worth nothing and have the course mark come entirely from in-class examinations?
ReplyDeleteThis is a good suggestion, which I've considered. However, (A) at least for my class I feel that most of the learning happens through the homework so they should be rewarded for spending countless hours on it, and (B) in an exam I can only give easy problems -- what can you solve in 50 minutes? -- unless it's a take-home exam, in which case it's no different from a homework.
I disagree entirely with Nicholas. I have learned more doing Von Ahn's grueling homework assignments than from the lectures and recitations themselves. There is just something about sitting down for 8 hours a day for 2-3 days to work on an unintuitive, miserable problem, bouncing ideas off of friends, and then getting that *amazing* breakthrough and solving the problem. It ensures that you *REALLY* learn the material, because you have to make every logical step to get to the solution yourself. It is incredible to peel back your layers of assumption and get the creative juices flowing; to generate a novel solution to a strange and impenetrable problem, and then have that solution somehow, amazingly, *work*.
ReplyDeleteTests are just regurgitation. There is really no way to make a test that has the same depth as these homework assignments, so you have to revert to repeating formulas, and giving basic interpretations and simplifications of earlier problems. Granted, Von Ahn's tests are impossible in themselves, but they are not nearly as difficult as the homeworks. I feel that in the real world, we will have to solve deep, complex problems in small groups. We will have to spend many long hours solving single, monumental problems that require (sometimes entirely original) problem solving and thought.
What better way to prepare us for this than by giving us a few, huge, heavily weighted homework problems?
As for the cheating policy, I am obviously not a fan. I feel it is draconian, if not sadistic, to give a student a NEGATIVE ONE HUNDRED on a homework assignment for simply visiting a website, especially when homework assignments are weighted so heavily.
I may just be a victim of groupthink and subtle brainwashing when I say this, but this draconian policy is *probably* necessary. There are too many classes where all of the questions can easily be found on Wikipedia and rehashed with only slight modification.
Von Ahn's policy on this is more of a scare tactic than an actual "policing" of the web. He can only trap so many sites; and he certainly can't circumvent some of the larger and more well-known resources; so he must revert to creating one extremely obvious trap, and then saying that "next time, it won't be so obvious."
This is a clear bluff on Von Ahn's part... But it is an effective one. When you dedicate so much time, and so much of your grade, to a single class, working on a single assignment, all to appease some diabolically cruel professor, you stop thinking clearly. You become paranoid. Von Ahn has become something like Big Brother among us students. We are wary of doing anything which might even slightly contradict the rules. We are--sometimes literally-- constantly looking over our shoulders with paranoid delusions that Von Ahn could be lurking behind every proverbial corner.
This may be the image that Von Ahn wants to convey, and it may be necessary to *control* the behavior of his students and ensure that they do not cheat; but I was always taught that it is often ethically superior to be *respected* rather than *feared*; though it is probably pragmatically superior on Von Ahn's part to be *both* respected and feared.
I do respect him. But Jesus do I fear him.
Anonymous,
ReplyDelete1. Why are you not doing homework?
2. Your IP address has been recorded and it is currently being correlated with our course roster as well as that of all previous incarnations of 15-251.
IPs are not 100% accurate, and given that students know about this ahead of time they can easy spoof their IP. Or I could spoof my IP to look like one of your student's, thereby getting them in trouble :) Or do you not target specific people?
ReplyDeleteLuis
ReplyDelete1. I finished it early!
2. :(
Luis I am not sure blogspot allows you to record people's ip (specially when you allow people to post comments without moderation). Also, I do not understand why you so adamant on the fact that students should learn. If someone's wants to learn he will work hard, why care about people who donot want to get more knowledge.
ReplyDeletePolice state concept is quite lame, why not keep them as captives for an entire semester ???
Keeping them captive is a good idea. I may do so next semester.
ReplyDeleteIf you are really thinking in that direction, then perhaps your sponsorship rates may also increase.
ReplyDeleteAnyways have you recorded my ip and identified me???
Dear Anonymous,
ReplyDeleteI highly doubt you finished this assignment already and yes, Luis has already identified you..
by another student in 251 who doesn't want to start the hw.
make homeworks optional, the exam 4 hours long (closed book) and reuse some slightly modified homework problems in the exam (plus some new problems). only return exams on request.
ReplyDeletehave quizzes but do not tell them when. (this is *very* effective)
grade based on quizzes + exam.
I like Severin's idea except trying to get a 4 hour time slot that works for everybody is extremely difficult. I also don't know how returning exams only upon request is relevant.
ReplyDeletewhat if the student accessed one of your "trap sites" from a proxy, or a starbucks, or off their iphone?
ReplyDeleteTo bradley.yoo: What about Saturdays? ;)
We follow Severin's idea in India because of the simple fact that you cannot stop people from cheating and its very mean to fail them in the course if they are caught cheating.
ReplyDeleteI do like reading your blog, but if you want to take it to the next stade - consider changing your design. The only thing I can reccomend is you to browse free themes for blogger.com
ReplyDeleteI like my design! May I recommend you look at acquiring a spellchecker for your comments? That will bring them to the next "stade".
ReplyDeleteWhat if we made the homework seem as if it is required and is a huge portion of the grade but is actually worth 0 percent to the final grade. (TAs will need to give enough effort in grading to make it seem as if it is worth something) Then, the students who cheat will continue to put effort in cheating (which is a seirous waste of time) and people who don't cheat will do better on their tests. Since the grade cut-offs are not being released anyways, students have little chance of knowing that their grades are being evaluated this way.
ReplyDeleteI got caught in your 251 class. I'm going to say this: do exactly what you're doing. Because it works. Although you should probably just tell everyone you are going to have trap sites which would cut out that 10% cheating...
ReplyDeleteyou should also consider checking out all the books in the library on a topic that appears on hws, in that solutions to a recent homework problem was found in a number theory text.
ReplyDeleteadvice from a former student of 251
p.s. what was the link to the wiki trap you just set up?
Actually, I'm going to return to the suggestion that homework not count toward the final grade. Is it so important that the grade reflect the entirety of the course? Why not let the homework be a treat for those who care (perhaps reward them with recognition rather than grades, which would hopefully cause cheaters to feel stupid / guilty), and let the exams address the mundane problem that you have to assign grades?
ReplyDeleteI'm not entirely against this idea, but I truly feel that students learn more with my method.
ReplyDeleteLearning isn't all about quantity. Students need to be prepared to succeed in the real world, not on textbook assignments.
ReplyDeleteBack when I was a prof ('88-'96), I encouraged students to work together (especially on final projects, though I allowed it for everythgin) and turn in a joint submission for homework where everyone got the same grade. I told them they could get anyone they wanted to do their homework as long as they wrote up an original response and cited their sources. I never gave in-class tests.
Disclaimer: I taught primarily graduate classes and philosophy of language, where most of the homework was non-trivial. It was also pre-web or the early days of the web.
What makes the police state strategy preferable over the strategy of making homework insignificant?
ReplyDeleteSome of my best classes have been when the teacher didn't count homework or counted only specific homework assignments (that he warned us of in advance). Grades only came from periodic quizzes and exams.
Those who didn't care didn't do the homework. Those that cared but did poorly did the homework and studied on their own.
And those who cared but were satisfied with their grade did whatever homework they felt necessary.
Everyone learned as much as they wanted to from the course.
How to spell the word Definately
ReplyDelete